On Sat, 9 Jan 2010, Nick Fortescue wrote:

> The reason I believe it is a bug is the following reasons:
> 1) It seems wrong and confusing that curl works with --proxy-anyauth but not
> with --proxy-ntlm when they both use the same authentication system (NTLM)
> in the end. Surely --proxy-ntlm could do the 2 request trick like anyauth
> does.

I already explained why that isn't wanted unconditionally.

There are other ways to solve it, like with a chunked-encoding POST but that's
unfortunately only a HTTP 1.1 requirement and for the first request we don't
know what HTTP version the server accepts. I haven't figured out a better way
to deal with it when we don't know the remote server's HTTP version support.

> 2) It seems wrong that when you use --proxy-ntlm a zero length request gets
> received by the website at the other end. The user never requested a content
> length zero request, so the code is doing something he didn't ask for.

The user asks for data to get sent using NTLM to the server. That's what curl
does. I don't think curl has promised that it will send the data twice to the
server. One of the times which the server will simply discard - and we know
it.

> 3) Your argument about the 100MB file seems sensible, but with the -d
> (rather than -F) option the POST is url-encoded so it is unlikely to be this
> long.

First, -d isn't necessarily url-encoded it just often is. Then I've seen many
cases where very large -d POSTs have been used and perhaps most importantly:
-d is just using the generic POST functionality of libcurl and that works like
this.

> Futhermore, even if it has to send a 100MB file twice, surely that is
> better than what it does at the moment, which is not send it at all (in this
> particular set of circumstances).

Well, your case is the exception really - or at least in my view and opinion
it is. It would've sent the 100MB if your proxy wouldn't had been insisting on
this "stupidity". If your proxy had worked like most users', then it would've
responded back with a 407 with the correct magic and curl would've sent a
second request with the 100MB of data.

I understand that you're looking at this purely from your specific case and
the problem you faced with it and I can see what you're asking and suggesting
but I'm looking at it with a much wider perspective and history.

Should curl always and unconditionally make POSTs using NTLM do the full post
in the first request, a lot of NTLM-users would ask us the reverse question
and complain on the waste. To me there is no easy way to satisfy all camps
here. (I'm not even saying "both" camps since I bet we can find more than two
if we really try.)

> Your reply says very sensibly that curl sends the data only in the second
> request. It looks to me like curl is not doing this (even if that was the
> intention) which is a possible bug.

Since your proxy sends back an error to the first request curl of course has
no point in continuing so no for you it won't. It is not an error either but
is in fact the only sensible way to act on the situation. I know that curl
does right otherwise as we test this in the test suite and we have lots of
users who use these kinds of things.

> I might try and get the source code to confirm this - at least it would help
> me understand it better.

Please go ahead, more eyeballs on the code can only be good!

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html