On Wed, 25 Oct 2000, Kevin P Roth wrote:

> (If this question has been asked before, could someone just point me
> towards the archived discussion? I looked through the past couple months
> though and couldn't find anything on this issue)

I don't think there's a precise question regarding this issue in there, no.
 
> I'm aware of the use of -D to dump headers into a file, and of -b to
> retrieve cookies from the file.

Yup

> However, there seems to be a limitation here as compared to a typical
> browser. A browser stores up cookies over time, so if the web server
> sends three different cookies on three different requests in the same
> path, each subsequent request will send the server all three cookies.

This is indeed what a browser would do. You'd have to fiddle a little to get
curl to do the same.

> However, cURL only saves the cookie(s) that was SET on the most recent
> request into the dumped headers file,

Curl sends back all the cookies that exist in the specified file. There's
nothing that prevents you from adding headers from several sessions into that
single file.

The -b cookie reading function also supports reading the same cookie name
several times and only using the last one, so if you do append the headers in
cronological order it could work.

> Plus, if I try to use the same file on one request for both reading and
> saving cookie info, it appears (I'm guessing without peeking at the
> source code) that curl blanks out the contents of that file (maybe in
> preparation for dumping headers to it) before it reads the contents of
> the file (due to the -b flag).

Yes it blanks the -D file. Still, you could easily make a script that would
append several -D dumped files into one.

> So if I needed to use this mechanism to store up all three cookies, I'd
> have to parse out the Set-cookie lines myself and gather them up in a
> separate file (imagine a multi-page request sequence that builds up a
> couple different cookies depending on the form variables that were
> posted, etc).

You don't have to filter out the Set-cookie lines, just append all the
headers. Curl's cookie parser skips the non-cookie related lines.

> I'm wondering whether there couldn't be a modification to -b (or perhaps
> a new flag altogether) that allows cURL to act a bit more like a true
> browser, by reading existing cookies from that file (without needing to
> store the rest of the headers too) and writing new cookies to the same
> file. That way I could do something like:

I've received this request before, and I do like the idea. This feature would
then store the "state" of the browsing to allow sequential invokes/uses to
pick up exactly where the last invoke left it (which would include cookies,
referer and other things).

As many other never-implemented suggestions, this hasn't been introduced
since no one has volounteered to write the code, not even partly.

> and get a different cookie from each request into the file, while still
> passing the server all previous cookies, based of course on expiration,
> path, etc...

The path and expiration code is already being used.

> I'm not familiar with the netscape cookie file format, but perhaps that
> would be the right choice for such a cookie file?

That format is just slightly better than the plain header. I would prefer to
see a much more machine readable format, in a more verbose manner that would
allow future changes without breaking backwards compatibility.

I would also like to really think things through before this is added, like
if it is a library thing or if we should design the whole thing in the client
side...

> Have I missed something obvious, or am I correct in seeing this
> limitation?

It is, although it has never been designed to work! ;-) Maybe the time to do
that has come...

-- 
  Daniel Stenberg -- curl project maintainer -- http://curl.haxx.se/