In Need of Immediate Help: What Am I Doing Wrong? WebCopy Keeps Logging Out

[Nippi]

I'm attempting to clone a website that's going to be deleted in less then two or three days now, and I am having massive difficulty. I have set WebCopy to login first, then start copying. I have disabled header checking. I have excluded logout pages. Yet, somehow I am STILL being logged out during the copying process and it is missing the most important pages I need. I have also tried including a form and that isn't working either. Please help!

[Richard Moss]

Hello,

Welcome to the forums and thanks for the questions. Once you've successfully logged in, WebCopy reuses the cookies for the duration for the crawl, the only way I'm currently of causing a log out is by hitting a logout link. You say you've already covered this but have you checked the results list by filtering it on common works like "log" or "sign" to see if anything else comes up? WebCopy doesn't currently offer any sort of logout detection and I don't have time to try and build anything in the short term.

Otherwise you could try adding rules to block most parts of the site to only copy a little, check that it downloads correctly, then relax the rule to copy a bit more to try and pin down where you're getting logged out.

Regards;
Richard Moss

[Nippi]

I managed to get it working, however it stopped at some point with a "System Out of Memory" exception, despite having plenty of gigabytes in memory left to continue. I will have to see if I can manage to get it working again. I still don't know what I did differently if anything!

[Richard Moss]

Hello,

You could try running the 64bit version (they are an optional feature of the setup), or potentially try the 1.8 nightlies. The 64bit version won't be constrained by the same memory limitations as the 32bit version, whilst 1.8 (finally) makes a start on reducing memory and resolving cases where it loads large amounts of data without a good reason.

Regards;
Richard Moss