On Mon, 5 Nov 2001, john lask wrote:

Splendid work John, thanks a lot! Comments follow...

> 1. changes to files to allow curl to recognise --tlsv1 as an option

Goodie. But they weren't required for your problem to work out, were they?

> 2. correction to ssl/proxy problem

I would however like to see the implementation of this made a bit different.
I'll explain in detail below.

> 3. new makefile for vc6 for libcurl

Great! I know people will like this.

> note that 1/2 code should be tested on linux before release. I have
> tested on NT.

I'll personally try to make sure it runs fine on at least Solaris and Linux.

> so the loop reads from the buffer until a \r\n\r\n sequence is found,
> anything after that is left in the buffer i.e. for 0D 0A 0D 0A 00 the 00
> is left.
>
> one could argue that this is correct, nethertheless other clients, namely
> pavuk, etc can cope with the above acknowledgement. This is because they
> do a block read from the socket buffer, whereas the routine GetLine does
> a one character at a time read.
>
> In principle I believe that http-tunnel connect should leave the socket
> read buffer clear when it exits. For this reason I have made change #2.

Ok. I understand that this fixes your problem and that other clients do this
and it makes things work.

Unfortunately, it is the wrong thing to do!

This is my view of things. Please correct me or object if you (anyone reading
this) think differently.

We can't blindly assume that the proxy will always first send us the headers
in one chunk and then give us the rest of the reply with a separate pause.
Even if, since we're doing TCP we can't assume anything in how the stream is
being read from the proxy. Slow-start and the Nagle algorithm could very well
merge lots of data to get read at once, both the initial header and the
following data stream.

According to the standards, your proxy is doing wrong when it sends a
trailing zero byte after the final blank line. The data is supposed to start
immediately after the blank line.

That said, I don't think "fix your proxy" is a good answer. I think making a
good "work-round for bad proxies" must first detect the error and then
step-toe around it. Now we know what the error condition looks like, now we
must figure out how to be able to detect that and work-around it without
affecting all the compliant cases. To be frank, I don't currently know how to
do this (automatically).

The initial zero byte in the data stream might signal an error that could be
used to detect this situation, but I'm not sure what to do if we read it and
don't find a zero as that should then have been dealt with by OpenSSL...

Documents used as input for my conclusions:

 - Ari Luotonen's internet draft "Tunneling TCP based protocols through Web
   proxy servers". It expired in Februrary 1999, but is still available all
   over the net. One URL:
http://www.web-cache.com/Writings/Internet-Drafts/draft-luotonen-web-proxy-tunneling-01.txt
   (I don't know why this was never made an official RFC or something.)

 - RFC2817 "HTTP Upgrade to TLS" section 5.2. It mainly agrees with the above
   draft, and doesn't go into the details quite as much.

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/