Main Menu

Encoding of URL's is invalid

Started by lswang, January 18, 2016, 06:47:57 AM

Previous topic - Next topic

lswang

Hi Richard,

Good morning.
I have downloaded the target website, it make it more quick to access the website now.
But I still facing challenges on some issues, it should be caused during the generating of local links between storage folders:
1. If the slug in the WP pages is in English, the generated link and local storage folder can be open no problem.


2. But if the page don't have a English character but a Chinese or other format of Character, it can't be open locally later:


3. I think the reason might be the Unicode issues. if you search folder carefully, you will find actually the page already downloaded but stored in a Complicated Named folder:


   Could you help to check if I can fix this problem here? thank you.

Kind regards,
Louis

Richard Moss

Hello,

I knew it was too good to be true ;) That is a bug - I'll take a look and see if I can get it fixed for the next update. Slightly odd as I do actually have tests for Unicode URL's, but I shall do some more digging.

I doubt there's much you can do to resolve the issue - save a lot of manual search and replace

Regards;
Richard Moss
Read "Before You Post" before posting (https://forums.cyotek.com/cyotek-webcopy/before-you-post/). Do not send me private messages. Do not expect instant replies.

All responses are hand crafted. No AI involved. Possibly no I either.

Richard Moss

I'm not able to reproduce this locally, my test pages that include Unicode characters in their names are processed correctly.

Assuming you saved your WebCopy project so all the link information is stored in it, can you send me the project file for analysis? (You can clear the rules and forms / passwords before sending)

If you could also send one of the HTML files that contain the "bad" characters so I can check its encoding. It seems that for whatever reason the pages aren't being saved as UTF-8 as the response headers indicate they should be.

Also, as this is a different issue from your original post, I'm going to split this topic in two so there's one thread per issue.

Thanks;
Richard Moss
Read "Before You Post" before posting (https://forums.cyotek.com/cyotek-webcopy/before-you-post/). Do not send me private messages. Do not expect instant replies.

All responses are hand crafted. No AI involved. Possibly no I either.