cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Website times my scraper out frequently

From: Gil Gomes <gil_at_gilgomes.com>
Date: Mon, 08 Jun 2015 11:20:48 -0500

i'm currently fiddling with my first scraper. I can get the info I want,
but the website times my scraper out frequently, so I need to make a
manual request to the website in order to copy the headers to my curl
app. Is there any way I can do this without the manual part? I'd like my
data to be always up to date and to update it now, I have to intervene
manually, wich sucks. How can I solve this? I've tried using cookiejars
but i wasn't lucky. perhaps because I'm using them in the wrong way.

The website i'm talking about is www.jumbo.pt

--
Thanks,
G.
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2015-06-09