cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Unable to capture required page.

From: Ralph Mitchell <ralphmitchell_at_gmail.com>
Date: Tue, 7 Sep 2004 10:43:16 -0500

On Tue, 7 Sep 2004 14:11:50 +0200 (CEST), Daniel Stenberg
<daniel-curl_at_haxx.se> wrote:
> Now, the first page that returns is basicly a single javascript that sends a
> POST. curl doesn't support javascript. You need to write a script that parses
> that page and sends off that POST (and I would suggest you include cookies and
> correct referer etc).
>
> This method is commonly used by sites that try to prevent what you're trying
> to do.

I think sometimes that just happens, without necessarily being
intentional. There's one site I've written a script for that mostly
has a normal page, but whenever they do updates, they switch the first
page to a different page. The alternate has onLoad=form.submit() in
the body tag, and there's nothing in the body except for the form with
a couple of hidden fields. When it executes, it appears to go to the
exact same page that I normally get to. Maybe it would be different
for an actual paying customer, though...

I suspect that sometimes, some of the web page generating tools try to
be "smart", and don't quite make it. I think that's definitely the
case with a Tivoli tool we use - it generates pages which are forms
with hidden variables that match the displayed text. There's a
"Return" button which is actually a submit button - even though there
is no way you can change any of the form. The server pushes out about
40kb of html and 40kb of javascript, where 2kb would be sufficient
especially considering that the javascript is doing form data
validation...

Ralph Mitchell
Received on 2004-09-07