Daniel Stenberg wrote:
> >So here's how the contents are not sent twice:
> >
> > 1. Server responds with 401 asking for authentication, and closes
> > the connection.
> > 2. Client never sends request body, because auth response is received
> > before the delay. Or, if client has already started, it can abort
> > after reading the whole response.
> > 3. Client opens a new connection and this time authenticates.
>
> This is ugliness. We can't do persistent connections and auth at the same
> time. But if this the only way, I will obey.

You must implement this logic anyway, for _all_ requests. Any request
can return an error code and the server may or may not close the
connection after it. Similarly as the client you may or may not
decide to close a connection when you receive an error, if you
recognise that the request body is long and not needed.

That is _entirely_ separate from 401 and auth handling.

You can do persistent connections and auth at the same time.

However, they should be completely separate logically. (With the
possible exception of NTLM with might not be HTTP compliant).

That means you issue a request, and if 401 is received, you get
credentials ready and issue another _independent_ request.

That should be done exactly the same as if the libcurl user had issued
one request and then enother.

In fact, you could code auth handling in that way: at the highest
level of the library, simply issuing one request and then another
after 401 is received.

For maximum performance, you shouldn't even try to use the same
connection for the second request. Just let the normal pool reuse
occur. That improves throughput if the libcurl user is issuing
several concurrent requests already, and you are already pipelining
them by the time a 401 from one of them is received.

> >(Note that Curl was buggy before: it didn't send the request body in this
> >case (i.e. no "100 Continue" received) but send the next request's
> >headers. That's not valid HTTP).
>
> It was buggy before and it is buggy now... This area of RFC2616 and family
> is like a maze.

It was a more severe bug before: libcurl violated the based syntax of
headers/body/headers/body/headers/body/... So all sorts of proxies
and filters could have been confused. As you found, it's not a big
problem in practice.

At least now, it is issuing valid requests... just that they may not
be the appropriate requests ;)

> >The only problem area is NTLM, which requires the connection to be
> >maintained.
>
> Yes, NTLM is the one that has caused me the most headache and since I
> prefer using the same method for all multi-pass methods, this is where we
> are now: in the wrong corner. ;-)

Do you strictly depend on all auth methods using the same connection?
Because that will not work with some servers, or through some proxies.

Note that if NTLM strictly requests the same connection to be used,
then NTLM cannot possibly work over some standard HTTP proxies. I
find it hard to believe Microsoft would be that stupid ;/

If you do get a trace of Microsoft clients and NTLM, then hopefully
one of these will be true:

    1. Some magic parameter is carried from the 401 response to the
       next request, and it doesn't really need to use the same connection.

    2. The NTLM server does close the connection for large PUTs, even if
       it doesn't for small ones.

    3. The Microsoft client doesn't use PUT, it uses something else with
       no request body for auth, just like curl uses HEAD.

> Yes, this would indeed be _very_ interesting to figure out.
>
> Any user out there who know of a Microsoft tool that do POST/PUT with NTLM?
> If so, snooping on the traffic to figure out what it does would be *very*
> interesting.
>
> Any insight on other tools/libraries that do PUT/POST and NTLM would be
> interesting as well.

I would guess .NET's SOAP client will do requests with NTLM and a
request body.

Since I don't live in a Microsoft universe or using anything other
than Word and IE for testing, I've never encountered NTLM-over-HTTP
personally. It's a shame there's no RFC (I didn't find one).

-- Jamie
Received on 2004-11-04