Copy a personal site

[Alyssa]

Hello

I have tried Cyotek webcopy on a public site made with Google sites and it worked well.

Now I want to download my own site (which is private) for backup, but Webcopy doesn't prompt me for any user / password and just fails to copy. How I can do it, without making my personal site public?

[Richard Moss]

Hello,

WebCopy supports two distinct methods of logging into protected sites.

WebCopy will only prompt for usernames and passwords for sites which use HTTP 401 challenge authentication (the ones where your web browser will display a native username / password dialog).

Many websites use "forms" authentication where you need to enter your username and password into a form and submit that to log you in. WebCopy doesn't automatically prompt for these, you have to setup a Form with the appropriate fields in, and have that posted prior to a copy being made. Without further information, it sounds like your website is doing this. Have you tried setting up a Form in your WebCopy project with the appropriate field name and values?

Regards;
Richard Moss

[Alyssa]

Hello, thanks for your response.
I am not much into this kind of knowledge, so I don't really known how to set up the said forms, is there a way to determinate which forms Google Sites requires and how to set up?

[Richard Moss]

Hello,

I must confess I'm not familiar with Google sites. When you access your site, what happens? Do you need to be logged into your Google account to actually access it?

If your account uses two factor authenticate, then it may be very difficult to do - WebCopy creates a new cookie container each time you copy a site, which means you have to log in each time (it doesn't have access to any other cookies, i.e. those used by your web browser). Although I've never investigated this, I suspect WebCopy will be unable to access such things, as it isn't capable of providing the multiple inputs such systems require.

Regards;
Richard Moss

[Alyssa]

Well Google Sites its linked with all Google products like YouTube/Google+/etc  so if I am already logged in my Google account there's no asking for login or anything.
And I don't use 2-step authentication for logging in.

[Richard Moss]

Hello,

Even if you were using two factor authentication, it occurred to me you could use application passwords which would work with WebCopy, so that ought not to be a factor regardless.

I'll have a play around with Google sites and ensure that WebCopy can copy these.

Regards;
Richard Moss

[Richard Moss]

Hello,

Rather annoyingly, I have to conclude that WebCopy can't copy protected Google Sites sites at this time.

The problem is that their login form includes a value named GALX which is dynamically generated. Unfortunately, WebCopy currently only sends the exact information you supply it. I've logged a new enhancement to overhaul form posting to pull out existing values then apply the custom values you specify, but it'll probably be a little while before this is implemented in the product.

Sorry that WebCopy can't help you in this case.

Regards;
Richard Moss

[Alyssa]

Hello

Thanks for your concern, I have thought some alternative solution to copy a site with WebCopy.

There are 3 methods to share a google Site:

- Public
- Shared with specific user
- Link only

The third option (link only) its probably the most appropriate in this situation, as it doesn't make the site visible with a simple internet search, but only make it accessible if someone knowns the exact link to it.
So I made my site "link only" and tried to copy it, I still had problems since the internal links were not converted so, while the site's pages were saved, each link that led to another page within the site would still link into its online source.

e.g.
<a href="https://sites.google.com/site/mysite/home/test-page"  ....
instead of
<a href="test-page/test-page.htm" ...

So the backup copy isn't navigable.
(Even if honestly I am not too sure if an altered copy would be appropriate)

[Richard Moss]

Hello,

I tried copying a test site of my own and it seemed to go fine - I didn't look at every single template file but they seem to be remapping links as expected.

The only occurrences where this was not the case was those pointing to other domains (for example https://ssl.gstatic.com/) as the current builds of WebCopy only support from a single domain + subdomains.

With that said, I suspect there is a remapping bug somewhere as another user had a similar issue. I really need to find the time to test and fix this one if it exists!

Regards;
Richard Moss

[Alyssa]

Hello

Thanks for your support, I managed to backup my sites at least with some workarounds using another tool. I would be happy if Webcopy could do it too since the other tool I used isn't very user friendly (if wasn't for the help of someone I would have never configured it correctly).

Regards
Alyssa

[Richard Moss]

Hello,

Thanks for letting me know - sorry WebCopy wasn't suitable for your needs. Hopefully in the future it'll be better server by adding some extra stuff to forms support, and eradicating any lingering mapping bugs.

Once again, thanks for the feedback!

Regards;
Richard Moss