Main Menu

Recent posts

#91
Color Palette Editor / Re: Support for NES Palettes
Last post by Richard Moss - April 10, 2023, 09:05:33 AM
Hello,

Thanks for the suggestion. 192 and 64 suggested this is a simple RGB triplets palette and I logged issue #485 to implement it.

On checking the source website I found http://www.firebrandx.com/nespalettes.html. This useful page included downloadable palettes along with reference images and thus I was easily able to test my implementation using the Digital Prime (FBX) download.

The next build nightly build of 1.8 should include this type.

(PS: Requests like this normally wouldn't normally be implemented Soon(TM) let alone instantly, a) you caught me in a break b) it is probably the simplest file format possible and which is already implemented and c) reference images were available so I could instantly be sure it was correct)

Regards;
Richard Moss
#92
Color Palette Editor / Support for NES Palettes
Last post by brunobox99 - April 10, 2023, 02:17:14 AM
Good Day Richard!

I tried to open NES Palettes (*.pal) with Cyotek Palette Editor, but it says that the format is unsupported.

Can you please add support for this format? All NES palettes are 192 Bytes (64 colors).

Here are some palettes you can try: http://www.firebrandx.com/downloads/Novemeber-2017-Palettes.zip

Please let me know if I can help with anything.

Thank you very much Richard, hope you are doing fine.

Bruno
#93
WebCopy / Re: scan folder is limited to ...
Last post by Richard Moss - March 29, 2023, 06:51:43 AM
Hello,

Firstly, please stop replying to other peoples posts with unrelated questions. It doesn't help users getting emails about replies that having nothing to do with their own own questions nor does it help other users scanning boards based on topic titles.

From your somewhat vague description I assume you're talking about the Quick Scan dialog. This is intentional - it is a quick scan, intended to scan "just enough" of a website for you to make decisions about things you might need to exclude - see the documentation topic, https://docs.cyotek.com/cyowcopy/current/quickscan.html.

Basically, the two options "Maximum Depth" and "Maximum pages per host" settings only apply to the context of the Quick Scan window and these settings override the normal settings (such as Limit crawl depth).

If you do a full scan (https://docs.cyotek.com/cyowcopy/current/scan.html) or download (https://docs.cyotek.com/cyowcopy/current/scandownload.html) there are no deliberately enforced limits.

In hindsight it is confusing, I'll get the documentation updated.
#94
WebCopy / Re: Rules Exclusion Happens Af...
Last post by Richard Moss - March 29, 2023, 06:26:28 AM
Welcome to another episode of "Richard Admits To Embarrassing" bug, courtesy of your host, the one writing embarrassing bugs.

I had checked this previously (multiple times!) and couldn't reproduce, but when I was tweaking the rules so that distance limits applied only to text/html, I finally twigged onto why this is happening, wrote a reproduction test and sighed.

Basically, there are some advanced filtering options that need information from the server

* The Download all resources option is set
* Rules that do processing based on content-type
* Minimum file sizes
* Maximum file sizes
* (1.9.1+ only) Maximum distance from base URI (this version wants to know the content type so it can only apply it to text/html)

If any of these options are set, then WebCopy makes an additional request as part of rule processing  so that it can read the content-type and content-length headers. Which means it hits the logout page that you excluded with a HEAD request and thus logs you out.  :-[

I don't know when this bug was introduced, file limits were introduced in 1.7 and content type rules in 1.8 but I do know that in 1.9 I spent a very long time drawing a flow chart of crawl decision logic then rewriting all of the code to be less spaghetti and more maintainable and I definitely (re-) introduced this in that build.

Not sure if this is the cause of all of the cases in this thread but it seems a good bet given the difficulty in reproducing, please let me know if this isn't the case. I've logged issues #481 for this and it will be part of the mammoth 1.9.1 bug fixing release.

Another case of better late than never...

Edited to add it also does it when "download all resources" is set. Which makes sense (needs to know the content type!), but also means this affects pretty much every single crawl. Oops.
#95
WebCopy / scan folder is limited to 1000...
Last post by david72 - March 26, 2023, 03:05:36 PM
Hey

I found only scan folder is limited to 1000 max page only.

Is there any other way  ?

Also i don't understand Max Depth and Limit Crawl depth are they same ?

Thanks
#96
WebCopy / Re: Want to clone a website wh...
Last post by Richard Moss - March 19, 2023, 01:10:41 PM
Hello,

Belated Happy New Year! And 2023 is shaping up nicely so far, hence even replying to support tickets and forums post, months late as they mostly are.

Simply put, you probably can't. Captcha's require JavaScript, WebCopy doesn't execute it.

Regards;
Richard Moss
#97
WebCopy / Re: Only http and https URI sc...
Last post by Richard Moss - March 19, 2023, 01:08:28 PM
Necro reply, sorry.

This was logged as issue #441 and is fixed in WebCopy 1.9.1.

When using that positional argument that can either be a URI or a file, files can be relative or absolute, but URIs must be fully qualified.

Regards;
Richard Moss
#98
WebCopy / Re: Limit distance from root U...
Last post by Richard Moss - March 19, 2023, 09:37:48 AM
Quote from: david72 on March 19, 2023, 08:04:24 AMI am trying to only scan the folders without saving html files

Anyone have idea how to do the same ?
Due to huge files and deep folders , its difficult store index files

Thanks
Dav


I haven't quite worked out if this is an AI generated spam post, but decided to err on the side of caution this time instead of deleting account and post as I usually do. I did edit the post the remove the link though.

Open the Project menu, click Scan Website
#99
WebCopy / Re: Limit distance from root U...
Last post by Richard Moss - March 19, 2023, 09:35:02 AM
Quote from: hajzlik on December 09, 2021, 11:02:09 AMThere should be an option to set different limits for HTML and for non-HTML files.

Let's say you have a sitemap. You want to download all the linked pages, but not any further links.

You can limit the distance from root URL to 1, but then you end up with HTML pages without images and other content.

You can limit the distance from root URL to 2, and the other content will download, but you will end up with a bunch of unwanted HTML files (but without any images they contain, which makes them even more useless).

Hello,

Belated thanks for the feedback. This makes a lot of sense! I don't think it makes sense to add another option (or maybe add it but not expose it for now, or hide it away somewhere), but I do think it makes sense to completely ignore distance for non-HTML. I've logged that as issue #464.

Thanks again!

Regards;
Richard Moss
#100
WebCopy / Re: Limit distance from root U...
Last post by david72 - March 19, 2023, 08:04:24 AM
I am trying to only scan the folders without saving html files

Anyone have idea how to do the same ?
Due to huge files and deep folders , its difficult store index files

Thanks
Dav