On Wed, 9 Oct 2002, Cris Bailiff wrote:

> This is starting to look like an ssl bug, either in OpenSSL or the SSL on
> the securityfocus server. I think it's shown up by the following change in
> transfer.c:

I greatly appreiciate your efforts Cris. The moment I saw this problem and my
first gdb-session made me suspect this code immediately. I changed this very
recently.

What OpenSSL versions are you guys using for this? I have an oldish 0.9.6c
and it shows this problem.

[ code snipped out ]

> Debugging, 'nread' (for the first SSL 'packet' from
> analyzer.securityfocus.com) is '122', but the printable text in the first
> packet is much shorter, only 44 bytes (ending at "-Type: text"). The rest
> are NULL.

ARGH!

> This change, from using strcpy to memcpy, allows the trailing nulls in
> k->str to be copied into the output, whereas in previous versions, of
> libcurl they would have been truncated/ignored.
>
> Looking in Curl_read (sendf.c:333), this data is exactly as returned by the
> underlying SSL_read function:
>
> nread = SSL_read(conn->ssl.handle, buf, buffersize);
>
> Breaking here, we can see that nread is 122, but buf has only 44 printable
> chars, so it looks like openssl's problem, not libcurls.

This change was necessary to handle the odd cases when the remote server
doesn't reply a single header, but instead starts with a binary chunk. It is
a violation of the HTTP spec, but a violation we can deal with and that
browsers deal with, so I think we should.

I didn't however count on OpenSSL to stab us in the back like this.

> The workaround is to change back to the strcpy, instead of the memcpy
> (which is basically what you are doing, Craig), but that's only a kludge
> which works because headers cannot contain binary data. If there's some
> underlying problem with SSL_read, it's potentially going to clobber binary
> body data as well. I think this has to be triggered by some specific
> server-side behaviour (bug? No, never!) in the specific version of IIS in
> use, otherwise we'd surely have noticed curl corrupting SSL body data
> before now!

Right, the work-around to put the strcpy() back will work for all
well-behaving HTTP replies.

> I'll see if I can get other ssl tools (or sslsniff) to show the same
> behaviour with that server (and I'll check the openssl changelogs as
> well!).

We should also try to come up with some code to try to detect if this is
happening and if so pick the proper function. I'm not sure that is possible
though.

If all else fails, it might be a good idea to write a small simple test
program that uses OpenSSL to recreate this problem and then have that one
posted to the OpenSSL dev crew.

-- 
 Daniel Stenberg -- curl related mails on curl related mailing lists please
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf