On Fri, 1 Mar 2002, David Thiel wrote:

> I'm trying to write a website testing program in python, but I'm seeing
> some discrepancies in how curl and libcurl behave. Any guidance would
> be appreciated.
>
> Here's a short version of the offending bits, pretty simple:
>
> url = "https://foo.com/LogIn?name=bob&password=johnson"
> curl = pycurl.init()
> curl.setopt(pycurl.URL, url)
> curl.setopt(pycurl.FOLLOWLOCATION, 1)
> curl.setopt(pycurl.WRITEFUNCTION, search)
> curl.perform()
> curl.cleanup()
>
> The way this site authenticates, it just accepts the parameters given in
> the url and sets a cookie. When I try to access the site using curl -L
> https://blah, it logs in properly and returns the proper page. When I use
> this libcurl bit, if I give it the *improper* name and password

If you set VERBOSE to 1 as well, you'll get the request displayed to stderr
and it should tell you a little more about what's going on.

Also, I think this sounds as if you might have been bitten by the pycurl bug
Kjetil found the other day. What libcurl and pycurl versions are you using?

> it returns the login failure page, but when trying the correct one, it only
> displays the regular login page, with no success or failure. So it looks
> like there's a problem with either accepting the cookie sent, or the
> redirect that happens shortly after login.

You don't enable the cookie parser, so the cookie will be ignored. The
Location: following code will be exactly the same, as that is controlled
purely internally in libcurl and thus there's no difference if curl asks for
it or if pycurl does it.

> So I guess the question is, what is the correct way to use libcurl to
> achieve the exact same results as one would with curl -L?

The above snippet looks just fine!

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/