Custom attributes and link rewrites

[frhel]

Hi, big fan of webcopy here, do have a couple of questions though.

Is it possible to make webcopy crawl a URi that's defined in a custom attribute (like <img data-original="http://..." />) and then rewrite the url within that attribute just like a regular src attribute? I'm asking because I'd like to use lazyloading for my website, but in order to do so I have to leave out the src attribute and include it as a custom attribute like mentioned above.

Also, how extensively can I run this as a commandline tool? Is there some documentation available to support that sort of use?

Best regards,
Freyr

[Richard Moss]

Freyr,

Welcome to the forums and I'm glad you like WebCopy!

Currently you can't include custom attributes, this wasn't something I originally considered - makes perfect sense though. Behind the scenes, WebCopy uses XPath to query the DOM nodes to detect additional links, so it should be reasonably straightforward to extend this to allow custom expressions to be included. The problem of course, is that WebCopy is already an unfriendly tool for end users, and expecting them to write XPath expressions wouldn't be very helpful! The UI would probably need some extra thought to make it easy to use without introducing more bugs. I've logged this suggestion to include in a future update.

The last update or two included a new tool, wcopy.exe, the intent of which is to do more or less everything the GUI client can. As of yet though, I haven't added commands to expose a lot of that functionality as I haven't finished re-writing command line parsing to be a little bit more automated - this tool will end up with lots of options. Currently it can download either a single file or an entire web site, and there are a few options you can set (run wcopy.exe without any parameters to get the current list). As a workaround for the functionality that isn't exposed yet, you can configure a project using the WebCopy GUI, then pass the project file name to wcopy.exe and it will use all the settings defined in the project. Basically it's still a work in progress. The current beta version released last week includes some new code for handling command line switches by allowing you to do /switch to enable something, and /no-switch to disable it which I think will be more convenient for users, and I'll start adding much more options in the next update hopefully.

Thanks again for the suggestion regarding custom attributes, let me know if you have any other problems or suggestions with WebCopy.

Thanks;
Richard Moss

[Richard Moss]

A new update has been released which includes support for this

http://www.cyotek.com/blog/products/srcset-attribute-support-custom-attributes-300-status-support-and-more
http://help.cyotek.com/cyowcopy/webframe.html#customattributes.html

Hope this helps, if not please let me know.

Regards;
Richard Moss