Greetings all,

I'm using cURL to download and save the page source of different web
pages. I'm running into an occasional issue, though, one example of
which is at the following address:
http://www.fanfiction.net/s/1260679/1/

Every time I load this page in a browser, it works perfectly. In fact,
I've never received the error page I'm seeing from FanFiction.net when I
load it manually. However, for some URLs (such as the one above), it
seems to be returning the error page every time when I load it with my
cURL script, while the majority seem to work fine.

My script is currently called through applescript (I'm running MacOS
10.4, I'm not sure how to check what version of cURL that is), and is as
follows:

curl --cookie "<path to my cookie file>" --max-time 15 '<url>'

Replacing the two parameters in <> with the appropriate values, of
course. My cookie file has nothing to do with FanFiction.net, but I
don't think it's hurting, either (in fact, I've tried to solve this by
taking out both options, with no luck).

Can anyone tell me what might be different between my cURL script
request and my browser http request that causes FanFiction.net to give
me an error page (relevant text below) vs. the correct page?

The text of the error page is as follows, not that it's of much help:
"FanFiction.Net Error
An error has occurred while processing your request.
If the problem persists, please alert site admin via
http://support.fanfiction.net.
All errors are logged for administrative and security purposes."

Thanks for any and all help! :-)

-Joe
Received on 2006-07-27