Main Menu

Recent posts

#1
WebCopy / exclude duplicate files
Last post by marvell - January 27, 2023, 08:32:56 AM
Hello. I'm trying to download the forum (all sections and posts are only on the first page of each topic), but I'm running into a cross-referencing problem on the pages.
For example:
/forum/viewforum.php?f=1 -- this page contains a link to /forum/index.php and your program downloads a copy of already downloaded sections again forum.
Example 2:

/forum/viewforum.php?f=1&start=50
/forum/viewforum.php?f=1&start=100
/forum/viewforum.php?f=1&start=150
"start" -- indicates the page number of the directory (like page 1, page 2, page 3, etc). Each catalog page includes a link to the previous page. Your program creates a complete copy of the site with each successive page.

How can I configure the program so that it does not download duplicates of already existing sections and topics on the forum?
How to prevent repeated download of the same file on different pages?
#2
WebCopy / Re: Blank error message when a...
Last post by angtucker - January 20, 2023, 10:39:17 PM
Hi there
I am having same issues did anyone get back to you on this?
Thanks!  :)
#3
WebCopy / Re: exclude of domains doesn´t...
Last post by ebuzzz - January 17, 2023, 08:39:08 AM
I'm having the exact same issue. The main reason I want to exclude domains is that WebCopy follows AddThis links on news articles pages. This AddThis link is a redirect and these redirects are done multiple times per page. There does not seem to be a way to exclude this domain in WebCopy. This results in a huge number of requests and a very long crawl time.
#4
WebCopy / Want to clone a website which ...
Last post by vkdesi - January 11, 2023, 08:46:11 AM
Hi;

First of all Happy 2023 to everyone.

I am looking to clone a website; which presents a Form and there is captcha image validation.

After submission of form; it presents list of records of useful information.

What is the best way to use webcopy to Clone such an website for offline use?

Thanks in Advance.
#5
WebCopy / Copy only sites mentioned in s...
Last post by Craftable - January 02, 2023, 04:19:10 PM
Hello, I'm new to this tool and want to copy some reference material for a project I'm working on. But I don't need the entire site and only the redirect sites mentioned by the website I inputted. If possible, I would also like to copy the redirect sites in the previous sites. Is there a way to do this?
#6
WebCopy / Copy website redirect to login...
Last post by gjimenezf - December 20, 2022, 04:44:41 PM
Hi,

Im trying to copy this website and all subpages from this root:

https://learn.comptia.org/app/certmaster-learn-and-perform-for-security-exam-sy0-601#read/section/
But as it begins it redirects to:
https://learn.comptia.org/login

I already put the credentials in both Fomrs and passwords, but as soon as it begins it says that it is redirecting to the login page and if i would like to change it, i select no, then begins copying and just copy the main webpage not the root url and its subpages from the url I want.

#7
Color Picker Controls / ColorPicker .net6
Last post by Jupklass - December 12, 2022, 02:03:15 PM
Hi
Thank you so much for this controls.
I had a free tool (https://boardgamegeek.com/thread/2490834/tool-help-your-game-prototype-tabletop-pnp)
wich use it and i ´m migrating it to .net6
I know there is no support but could it be possible to get the code and i try to migrate the controls to .net6?

Thanks
#8
WebCopy / Simple key click bypass
Last post by TheSpeaker - November 20, 2022, 03:11:52 PM
Hi Guys,

I'm a first time user, have search the forums and cant find an answer to my issue.

The website I'm trying to back up just requires a "I understand" click and I dont seem to be able to bypass that as it isnt a form.

Also when I try to scan the website the dianlog box, no matter how I resize it looks like this :

Thanks in advance for any pointers in the right direction.

Cheers
#9
WebCopy / Blank error message when attem...
Last post by whiggs - November 20, 2022, 03:46:33 AM
Not sure what the heck this is supposed to mean, but I see this whenever I try to scan a particular website:

#10
WebCopy / Webcopy running on AZURE
Last post by logos - November 16, 2022, 06:07:09 PM
Hi ,

I am exploring the possibility of runing Webcopy at command line in Azure platform without having dedicated Virtual PC .

Intension is to load webcopy.exe and run it in Azure Function app  and schedule of run according to requirement.

Similar requirement for different application seems covered
https://stackoverflow.com/questions/45348498/run-exe-executable-file-in-azure-function


Thanks