On Tue, 9 Nov 2004, Jamie Lokier wrote:

>> o If a 401 (or 407 when talking through a proxy) is received, then:
>>
>> if we provided credentials in our request (alas, this is the first pass in
>> the auth phase) and this is NTLM, we continue sending the whole
>> request-body
>
> This case does not occur with IIS 5.0, so is there any point coding it?
>
> Did you see David's packet trace from IIS 5.0? It doesn't send 100
> Continue _and_ it doesn't send 401 _until_ it has received the whole
> large request body.
>
> Therefore this case did not occur in the IIS 5.0 tests.

That might be true, but I don't see anything wrong in my statements above. I'm
not fixing this up to work only with IIS, I'm trying to make libcurl follow
RFC2616 more closely.

> If they've improved the server, this might occur with IIS 6.0. That's why I
> asked for a packet trace with that.

The last time we modified the auth code was when we figured out that IIS 6
behaves differently.

> Finally, if the server sends Connection: close (see David's traces) then
> that should override this NTLM test. You should fall through to the next
> case if the server sends Connection: close.

Yes

> A better strategy is to close the connection at once _except_ when we know
> the request body is quite small. Small requests are very common, and it's a
> net loss to break a connection for the sake of a few hundred bytes.

Right. I'll make a note about this for my updated plan.

>> o When sending a PUT/POST request (that hasn't a negotiated auth) after a
>> 401/407 response, we know that it will be rejected and thus we claim
>> Content-Length zero. (This seems to be what IE does.)
>
> Bad. (Unless you meant "after a 401/407 with NTLM auth type").

That's what I meant. I'll clarify in my update.

> You should only do this if the negotiation is NTLM (or any other MS type
> that depends on connections in the same way).

Yes, if this is the non-final request in a multi-pass auth method.

> Well implemented HTTP servers implementing other auth types will send 401
> without waiting, so the send-twice problem doesn't occur with them. So
> there is no need for a special hack with them.

Do we know for sure that Apache for example does this for Digest?

>> o The last step in the auth phase (the second request in Digest and
>> Negotiate, and the third in NTLM)

> A minor thing: isn't it the third in Negotiate too?

Uh, perhaps. I don't remember, but my point was more that this is for the auth
requests that aren't yet "completed".

>> It would probably make sense to somehow make this instruction
>> using the read callback. It will be troublesome for some apps to
>> deal with a rewind like this. I'm thinking for example when using
>> 'curl' to upload data from stdin...
>
> You'll also have to warn authors that the input may not be all read
> the first time (due to the early close).

Yes, this would need to be clearly documented as well.

It would be cool if someone who uses libcurl and multi-pass authencation in
their app could provide some feedback on this planned approach. It *will*
require a few modificiations in people's libcurl-useing code...

> By the way, what will you do if the server returns 401/407 to _this_
> request? Couldn't that happen if a proxy demands authentication and then
> the server does as well? Couldn't it even happen more times, if several
> proxies demand authentication?

You are right. If the proxy also demands auth we might in fact have to send
the data three times! (We only support one proxy.)

>> Optionally, we keep the current approach if some option is set, since it
>> seems to work fairly well for POST on most servers.
>
> I know real servers where that will fail with PUT. My own is one of them.

Yes, in fact I doubt there is a single one that will work with HEAD/PUT. The
case for HEAD/POST is much more likely to work.

> It would be great if IIS 6 does the right thing and sends 401/407
> immediately: then you can just tell people who don't like the send-twice
> bandwidth wastage to upgrade IIS.

I don't think it does. I don't remember the exact details, but if David can't
provide one of his excellent logs for this case I think we can dig up some
useful info from the list archives from the previous time we had to bite this
dilemma.

> I don't see any fundamental problems, just minor things to get right, and I
> think I have described them all.

To make sure we have nailed all the details currently brought into the light,
I'll update my plans and post it again.

We still have to settle on the required API changes/updates.

Then start coding.

-- 
      Daniel Stenberg -- http://curl.haxx.se -- http://daniel.haxx.se
       Dedicated custom curl help for hire: http://haxx.se/curl.html