curl-and-php
Website times my scraper out frequently
From: Gil Gomes <gil_at_gilgomes.com>
Date: Mon, 08 Jun 2015 11:20:48 -0500
Date: Mon, 08 Jun 2015 11:20:48 -0500
i'm currently fiddling with my first scraper. I can get the info I want,
but the website times my scraper out frequently, so I need to make a
manual request to the website in order to copy the headers to my curl
app. Is there any way I can do this without the manual part? I'd like my
data to be always up to date and to update it now, I have to intervene
manually, wich sucks. How can I solve this? I've tried using cookiejars
but i wasn't lucky. perhaps because I'm using them in the wrong way.
The website i'm talking about is www.jumbo.pt
-- Thanks, G. _______________________________________________ http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-phpReceived on 2015-06-09