Recent posts

[href]

#51

WebCopy / Re: Picture ref in <a> tag

Last post by Richard Moss - November 18, 2023, 05:08:38 AM

Hello,

The issue appears to be that the image isn't actually referred to as part of the raw HTML, but is populated by JavaScript. As I note many times, WebCopy doesn't execute JavaScript.

So using an example from your screenshot, the following is part of the raw HTML downloaded by the browser or tools like WebCopy

Code Select Expand

<a tabindex="-1" href="#" class="pushed" data-caption="1" data-deep="gallery-742680_77759" data-lbox="ilightbox_gallery-742680_77759" data-options="width:2500,height:1866,thumbnail: 'https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg'" data-album='[{"title":"","caption":"","width":"2500","height":"1866","thumbnail":"https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg","url":"https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg"}]' data-lb-index="0">
The href attribute is #, essentially pointing back to the parent page, so WebCopy ignores it.

Once the JavaScript has ran, the href is populated.

Code Select Expand

<a tabindex="-1" href="https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg" class="pushed" data-caption="1" data-deep="gallery-742680_77759" data-lbox="ilightbox_gallery-742680_77759" data-options="width:2500,height:1866,thumbnail: 'https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg'" data-album='[{"title":"","caption":"","width":"2500","height":"1866","thumbnail":"https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg","url":"https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg"}]' data-lb-index="0" data-lbox-init="true">

Unfortunately, while WebCopy can read data from custom attributes (such as data-album above), it wasn't really designed to extract bits out of them. However, by combining a couple of features, we can at least extract the images - but the a tags won't get updated with the true URL.

Firstly, you need to tell WebCopy where to find the extra URLS

• Project Properties | Advanced | Custom Attributes
• Value: //a/@data-album

(Documentation link: https://docs.cyotek.com/cyowcopy/current/customattributes.html)

As the blocks of JSON extracted by this method aren't valid URLs, we need to use URL Transforms to discard the bulk of the JSON and just keep the one attribute - I went with url in this case (again unfortunately WebCopy wasn't designed to be able to pull out multiple URLs from a single value except in some very specific places).

• Project Properties | Advanced | URL Transforms
• Add a new transform
• Expression: \[{(.*?)"url":"(.*?)"}\]
• Replacement: $2

(Documentation link: https://docs.cyotek.com/cyowcopy/current/uritransforms.html)

With the above in place "https://vicenteromeroredondo.com/wp-content/uploads/2023/01/130x97-cm.jpg" (and more!) is generated as a URL to scan by WebCopy.

Regards;
Richard Moss

#52

WebCopy / Picture ref in <a> tag

Last post by Chris - November 15, 2023, 06:14:18 PM

Hi,

It seems that all pictures that are referenced in the <a> tag are not dowloaded.

Thank you in advance for your help.

[&size=(\d+)]

#53

WebCopy / Re: Help copying a responsive ...

Last post by Richard Moss - November 11, 2023, 08:18:41 PM

Hello,

I had a look at the website and the image URLs actually have a size parameter that defaults to 1024. When viewing the page source, it was 1024, but when looking at the DOM in the browser it was 2056 - which happens to be the width of my primary monitor. I haven't looked further but it would seem clear that some JavaScript is being used to manipulate the style URLs.

So from that perspective, WebCopy is doing exactly as it should - the HTML specifies a size of 1024 and thus that is what it downloads.

However, if you want to try to mimic this aspect of the JavaScript behaviour (a reminder, if needed, that WebCopy doesn't execute the JavaScript) - WebCopy has a "Transform URLs" feature that will take a given URL and replace part of it.

I tried adding a transform (Project Properties | Advanced | URL Transforms) with the following attributes which seemed to do the trick - just set the new size as appropriate.

• Expression: &size=(\d+)
• Replacement: &size=2056

You can learn more about transforming URLs at the following documentation link: https://docs.cyotek.com/cyowcopy/current/uritransforms.html

Edit: There doesn't seem to be a need for the data-original attribute rule as this does not exist in the original source, of this page at least.

Regards;
Richard Moss

#54

WebCopy / Help copying a responsive Page

Last post by dantzu - November 02, 2023, 10:26:51 AM

I'm trying to copy a bunch of pages created with adobe express. I'm having trouble getting the full image resolution in the local copies. I only get images with max 1024px. Even though upon inspecting the page, I can see that images are served or at least are available in 2560px

(see screenshot here: https://pasteboard.co/OUmklCOvJ8BZ.jpg ).

Adding the custom attribute "//img/@data-original" to the WebCopy projekt didn't solve it.

How can I get the max resolution on the page? Any help would be much appreciated. Here's an example URL:

https://express.adobe.com/page/7Gwsq6GSTd8bl/

#55

Gif Animator / Re: Gif Animator foe WinXP

Last post by sanu66a - September 14, 2023, 03:49:17 PM

Version 0.0.6.0 and below should work on XP as they are compiled against the now discontinued .NET 3.5. Version 0.0.7.0 onward (the "nightly" builds) are compiled using .NET 4.6 and will not work on XP.

#56

WebCopy / Iamges on idex.html not showin...

Last post by gholland - August 30, 2023, 05:55:03 PM

Trying to copy infinityurns.com\index.html but the images are not being copied.
For example if I inspect the image labeled Brass Urns I see this and the image    https://www.infinityurns.com/cdn/shop/collections/Brass_Urn_180x.jpg is not copied but the Brass_Urn_10x10.jpg did copy and it's so small when I enlarge it its blurry:

<img id="944914694237" class="thumb-image blur-up lazyautosizes lazyloaded" alt="Brass Urns" src="//www.infinityurns.com/cdn/shop/collections/Brass_Urn_10x10.jpg?v=1650254596" data-widths="[180, 360, 540, 720, 900, 1080, 1296, 1512, 1728, 2048]" data-aspectratio="1.0" data-sizes="auto" data-image="" data-srcset="//www.infinityurns.com/cdn/shop/collections/Brass_Urn_180x.jpg?v=1650254596 180w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_360x.jpg?v=1650254596 360w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_540x.jpg?v=1650254596 540w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_720x.jpg?v=1650254596 720w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_900x.jpg?v=1650254596 900w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1080x.jpg?v=1650254596 1080w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1296x.jpg?v=1650254596 1296w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1512x.jpg?v=1650254596 1512w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1728x.jpg?v=1650254596 1728w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_2048x.jpg?v=1650254596 2048w" sizes="144px" srcset="//www.infinityurns.com/cdn/shop/collections/Brass_Urn_180x.jpg?v=1650254596 180w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_360x.jpg?v=1650254596 360w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_540x.jpg?v=1650254596 540w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_720x.jpg?v=1650254596 720w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_900x.jpg?v=1650254596 900w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1080x.jpg?v=1650254596 1080w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1296x.jpg?v=1650254596 1296w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1512x.jpg?v=1650254596 1512w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_1728x.jpg?v=1650254596 1728w, //www.infinityurns.com/cdn/shop/collections/Brass_Urn_2048x.jpg?v=1650254596 2048w">

#57

WebCopy / Re: Using query strings to con...

Last post by crocket - August 24, 2023, 01:38:53 PM

I was able to get WebCopy to save the files using the query string parameters. I created a new project and used the default items checked on the Local File property screen, along with enabling the "Use query string in local file names" option.

In the past, I tried numerous combinations of options on the Local File property section but nothing seemed to work. I'm not sure if I missed a specific combination of options or if the creation of a new project fixed the issue.

Anyway, attached is a screen of the Local Files property screen that worked for me if anyone else has problems with the query string naming option.

Thanks!

#58

WebCopy / Re: Using query strings to con...

Last post by crocket - August 23, 2023, 04:17:17 PM

That is what I am thinking as well.  Attached is a screenshot for the Local File properties for my project.

#59

WebCopy / Re: Using query strings to con...

Last post by Manuela - August 23, 2023, 05:07:57 AM

Use query string in local filenames should exactly do want you want, you said you have checked this out and it did not? I guess then this is a bug

[get_photo-1]

#60

WebCopy / Re: Using query strings to con...

Last post by crocket - August 21, 2023, 03:48:26 PM

I am already providing WebCopy with a complete list of files to download, so I don't think I need a rule to download specific files. On my test site, I am able to get WebCopy to download these files from the list I provide to WebCopy. The problem is that the image files that are downloaded are name get_photo-1, get_photo-2, get_photo-3, and so on.

I am trying to figure out how to get WebCopy to give these images a specific name based on the images URL. I specifically need WebCopy to name the file using the image name and the app_id value that is in the URL.

I am not very picky on the exact format of the file name, but I do need the file to have key pieces of information in the name.

Code Select Expand

https://test.site.com/api/v1/get_photo?filename=photo_12345678-1234-5678-9876-abdefghijklm.jpg&app_id=98765
In the URL example above, I need at least this info contained somewhere in the file name

• photo_12345678-1234-5678-9876-abdefghijklm.jpg
• app_id=98765

 
Thanks!