cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Newbie's question on cURL usage

From: Cris Bailiff <c.bailiff+curl_at_awayweb.com>
Date: Tue, 05 Mar 2002 23:55:05 +1100

I have mentioned this on the list before, and I don't like to promote the
'non-curl' solution here, but you are looking for 'elza' - a scriptable web
client...

Info at:
http://www.stoev.org/

Download at:
http://phiphi.hypermart.net/pub/

You give it a little script like 'follow the third link on this page', 'put
"fred" in the username field', 'press submit' and it 'just does it', cookies,
redirects, authentication, ssl etc. etc.

Its a bit tricky to configure, but once you get the hang of it, its quite handy
for burrowing into complex sites for data. It's not fast, being perl and stunnel,
but it works...

Now to re-write it using Curl::easy...

Cris

> > Now looking at the solution, I have a little bit more understanding how to
> > use cURL -- just follow the redirect (action) chains and keep the cookie
> > information at the same time, till you reach the target. Am I right?
> > However, can this philosophy apply to every situation?
>
> As you say, follow the redirects - this is how your browser would do it, so you
> would need to have a powerful reason to do it differently, such as knowing for
> sure a different approach would work.
>
> > Also I really think all these steps are too much for beginners. Is it
> > possible to allow cURL to automate this process? We know the input (login
> > info) and output (target page), cURL could handle all the mess internally
> > instead of shifting the burden to the user. Is this technically achievable?
> > Maybe it is just a naive idea.
>
> One major reason for not even attempting to implement this mess inside cURL is
> exactly that - it would be extremely messy. For example, the page you've been
> trying to post to requires you to login via Passport.com. The Passport page
> expects the login name and password to be passed via the POST variables "login"
> and "passwd". A page *I've* been trying to login to recently expects the name
> and password to be passed via "userId" and "userPasswd". Another page might
> use "userName" and "password", or "loginname", or "name", or "id" or anything
> else the page author thinks is useful and/or humorous.
>
Received on 2002-03-05