Cyotek Forums

Products => WebCopy => Topic started by: folamour on June 27, 2016, 06:24:29 AM

Title: how to be sure that a page as been crawled
Post by: folamour on June 27, 2016, 06:24:29 AM
hi i have make a donation because this program is awesome, i use it to populate the cache of my cdn

but yesterday when i was opening a downloaded page in browser she was displayed immediately

but its no more the case,

how to be sure that page has been crawled,

my site have 11.000 page so it's a real problem when copy stop i've to restart from the begining

hope you can help
Title: Re: how to be sure that a page as been crawled
Post by: Richard Moss on June 28, 2016, 05:32:42 PM
Hello,

By default, WebCopy will try to copy only new or changed files. If you want to force WebCopy to download everything, then either hold down Shift when clicking the Copy Website button, or trying setting the Always download latest version from the Advanced category in your project properties. This knowledgebase article (http://www.cyotek.com/support/kb/cyotek-webcopy/how-can-i-copy-only-changed-files-from-a-website) describes the different options you can use so that WebCopy only downloads new or changed files, you can use this for more information.

While scanning your website, if the main log shows entries with a yellow icon and a Status of Skipped, that probably indicates that WebCopy doesn't think the file has changed.

It's not really clear from your message precisely what issue you're having with WebCopy, so any further information you can provide would be helpful.

Regards;
Richard Moss