On Tue, 26 Feb 2002, Clay Loveless wrote:

Thanks a lot for your report.

> I believe I've found that the current stable version of curl has some
> cookie bugs.

Those cookies never let me sleep really good...!

I would be reaaaaally happy if you could provide us with some headers that
actually make curl go bananas as you describe. Then I could produce little
test cases in my end to work with.

If you can, I'd prefer one example for each single error discussed below.

> In addition, I've made requests to this same page with a browser and
> tcpflow to watch the comings and goings of cookies. The browser seems to be
> sending far fewer cookies than cURL is "seeing" and/or recording.
>
> In the example above, there are many duplicates in the cookiejar file for
> domains of "my.screenname.aol.com" and "screenname.aol.com". The
> "screenname.aol.com" entries are listed first in the cookiejar file, and
> have accurate values... However, the "my.screenname.aol.com" values are
> listed below the "non-my" entries, and have blank values. Unfortunately,
> those are the values that appear to be sent back on subsequent requests.

Let's call this problem A.

I think we might have a cookie recording problem here. If we first receive a
cookie named NAME for domain 'loonie.domain.boo' and then later receive
another cookie line with NAME for domain 'domain.boo' (cutting off parts of
the previous domain) this second cookie will be stored as a different one due
to the different domain property. But I figure they should actually be
treated as the same. (Cookies are a tricky business due to the lack of
standards, or rather due to the lack of sites following the actual
standards.)

I need to dig around in this. If anyone of you reading this has an idling eye
and a helping hand, just howler my way.

Does this sound like it could be what happens?

> For what it's worth, AOL's servers are pretty erratic about their cookie
> formatting... Here are a few other examples of what they send:
>
> Set-cookie: MC_MS_LDC=x; path=/ ; domain=screenname.aol.com; expires=Fri,
> 31-Dec-1980 23:59:59 GMT;
>
> (note space after / in path)

We call this problem B.

We do not strip off trailing spaces from values. This could probably be fixed
by applying this patch:

diff -u -r1.28 cookie.c
--- cookie.c 7 Jan 2002 23:05:36 -0000 1.28
+++ cookie.c 26 Feb 2002 10:27:12 -0000
@@ -135,7 +135,7 @@
       if(strchr(ptr, '=')) {
         name[0]=what[0]=0; /* init the buffers */
         if(1 <= sscanf(ptr, "%" MAX_NAME_TXT "[^=]=%"
- MAX_COOKIE_LINE_TXT "[^;\r\n]",
+ MAX_COOKIE_LINE_TXT "[^;\r\n ]",
                        name, what)) {
           /* this is a legal <what>=<this> pair */
           if(strequal("path", name)) {

> Some of their headers are "Set-cookie", others are "Set-Cookie" ...

That shouldn't matter. Headers should be treated case insensitive and I
believe curl does that.

> And about 80% of their cookie headers don't end in semi-colons, many of
> which do not get written properly to the cookiejar file (if they get
> written at all).

This is problem C.

Again, can you produce examples of this please? I did make a change rather
recently that I thought would make this work, but you seem to have proof that
I am wrong.

> I'm in dire need of a reliable cookie handler ...

Believe me, I want this fixed too.

> If there's anything else I can document to clarify this problem, please let
> me know. I know that some of these issues have come up in the past
> (according to the changelog and list archives) ... But unfortunately some
> of them appear to still be lingering.

Cookies are nasty. Really, they are. ;-/

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/