Cyotek Forums

Products => WebCopy => Topic started by: Tyron on February 17, 2023, 07:12:15 AM

Title: Unable to copy website, content encoding UTF-8 is not supported
Post by: Tyron on February 17, 2023, 07:12:15 AM
Getting attached error and webcopy stops the copy. Is there a fix/workaround to this?
Title: Re: Unable to copy website, content encoding UTF-8 is not supported
Post by: Richard Moss on May 22, 2023, 06:50:19 PM
Hello,

This sounds like a manifestation of issue #471. Basically, the web server is misbehaving a little and returning a non-standard encoding. Current versions of WebCopy throw a fit at this because they can't decode things they don't know about and web servers usually return standard values like gzip, deflate, br etc.

If you download one of the 1.9 nightlies this issue is resolved so WebCopy will now ignore these content encodings and will just download the file as-is. Of course, if the web server truly is returning the data in a custom form it won't be decompressed.

If possible, can you share the URL causing the crash if known, or the website if not. I'd like to test that URL to verify the existing fix is sufficient.

Thanks;
Richard Moss