cURL / Mailing Lists / curl-users / Single Mail

curl-users

RE: cURL --cookie-jar

From: Roth, Kevin P. <KPRoth_at_MarathonOil.com>
Date: Wed, 1 May 2002 13:00:45 -0400

-----Original Message-----
From: Daniel Stenberg [mailto:daniel_at_haxx.se]
Sent: Wednesday, May 01, 2002 10:58 AM

>> I was under the impression -c had been added to eliminate
>> having to use multiple cookie parameters to make cookies "just work".

> Yes, and that's why -c also enables the cookie parser. Basicly, if you
> have no previous cookies lying around, these two command lines are
> equivalent:
> curl -b readnofileatall -c storecookieshere [URL]
> curl -c storecookieshere [URL]

OK. I've finally set myself straight, by running a few tests and
observing the results...

I had been under the impression that -c would UPDATE a cookie-jar (if
it already existed). But in reality, -c simply dumps cookies into a
netscape-style cookie file (and of course enables the in-memory cookie
parser). Silly me...

> There were no significant and important reasons why [-c shouldn't also
> read from a cookies file], mainly a few minor ones:
> o There would be no way to tell curl to not read *any* existing cookies
> but overwrite the already present cookiefile.

Sure there would - `rm cookie-file`, or "DEL cookie-file.txt" ;-)

> o I thought it played nicely with the fact that -b already read cookies,
> we only needed an option for writing as well.

True. But other tools (thinking Eliza is one) simply have a flag that
"enables cookies". It would be more browser-like (IMHO) to have curl
treat a cookie-jar as a permanent cookies file (like netscape does),
with an extra flag (as suggested below) to block reading session
cookies. This would mean that -c both reads and writes (e.g. "keeps
up-to-date") the cookie-jar file.

However, I can live with 2 parameters instead of one (as it would
probably break something to change this now) for historical reasons,
as long as the documentation is completely clear.

>> curl -b @ -c cookies.txt url://...
> While this certainly would be possible, I can only see problems with
> this approach. It is harder to document, harder to describe to people
> and I guess therefor more error prone when people will use it.

Understood. I guess what we need then is some simplification of the
--manual text, along with a cookie-use FAQ... I will probably take a
crack at the --manual updates, unless someone else wants to volunteer.

> Let me try (but keeping it plain text), please ask again if there's anything
> in this picture that seems blurry!
> [SNIP]

Thanks. Two points of clarification please.

1: If -D is used on a request with multiple URLs (or with a -L) it appears
   that only the LAST set of headers is written to the file. Is there a way
   using only curl.exe parameters to get ALL headers recorded (either in
   separate files or by appending to the one file)?

2: If "-b headers" and "-D headers" are used together, along with multiple
   URLs, I'm going to take a guess at the order of things. Please correct
   it if I'm wrong:
   - curl reads from headers because of -b
   - then the first URL is requested
   - curl writes out the headers from this request because of -D
   - then the 2nd URL is requested (I'm assuming WITHOUT re-reading
      the headers file)
   - curl writes out the headers from the 2nd request

>> This way, there are two ways to "clear out" session cookies. One is to make
>> a request for a non-existant file (e.g. /dev/null) at the *end* of a curl
>> "session"

> I don't understand how this would work. Can you elaborate?

curls current command line structure offers no way for us to instruct curl
to clean session cookies out of a cookie-jar file. This -j suggestion would
add that ability.

Here's a bit of elaboration. The example I was thinking of is this simple one.
URL 1 is the "login page" for a site, which sets a session cookie. The 2nd
page sets some other cookie, but the session cookie is still needed. This
is the end of a curl "session". When we come back and re-run this tomorrow,
we need a new session cookie, but we want to retain the value of the other
cookie that page #2 had set.

One option is to block *reading* the session cookie from the cookie-jar file,
which you understood from my previous message. The problem with this approach
is simply that it's not very "browser-like", as a normal browser would have
thrown away yesterday's session cookie at the point in time that the browser
was closed (and the memory was flushed).

So, to truly mimic a browser, we need a way to flush the cookie at the END
of our session. We could introduce another option to prevent WRITING session
cookies (which would be used on the request for page #2). Or, as I described
above, you could make a request to a URL that doesn't return any cookies
(such as a local file URL), along with the commands for reading cookies,
writing cookies, and blocking session cookies. This sounds complicated,
but it would solve the problem.

We could NOT combine the two options (have -j block both reading and
writing of session cookies), because in the case of a curl session with
at least 2 calls to curl.exe, you need to only block one or the other,
but not both, or else you'll lose the session cookie at some point.

--Kevin
Received on 2002-05-01