2010/1/9 Daniel Stenberg <daniel_at_haxx.se>

> On Sat, 9 Jan 2010, Nick Fortescue wrote:
>
> Sorry, I'm not an expert and don't understand how NTLM works, so I'm not
>> why your emphasis on _first_. However, this is my problem, maybe you can
>> suggest what I should do?
>>
>
> So you argue that curl does wrong on the HTTP level but yet you say you
> don't know the HTTP level...

I'm sorry I seem to have offended you. I appreciate all your help, which I
know you don't have to give. As a software developer, I like it when people
tell me when they think they have found bugs - I was only trying to be
helpful and precise.

 I want to simulate a browser submitting a form. I have to do this via a
> proxy. The only way I have to authenticate against this particular proxy is
> NTLM. This proxy requires authentication. A web browser can do this fine,
> without losing the form arguments. Can this be done in curl? If so how?
>

Well, I suggested a way in my previous reply that you seem to have totally
> ignored. What was wrong or didn't work with that?
>
>
I hadn't tried it yet as the problem was a problem at work, and I'm now at
home. I've just logged in remotely and tried it, and --proxy-anyauth works.
Thank you for the successful work around.

> curl doesn't "lose" the form arguments and the browser argument also often
> is moot since very few sites operate with a POST as their first NTLM-request
> with auth. Believe me, I know these details good enough to know exactly what
> curl does and how the communication works and doesn't work.
>
> NTLM is a bad^H^H^Hfunny authentication method as it authenticates a
> CONNECTION and not a request (against all sense and HTTP guidelines) so once
> a connection has been authenticated it doesn't need to do anything more but
> just continue to use the same TCP stream.
>
> NTLM is a multi-step authentication so curl needs to send multiple requests
> in order to negotiate access and thus when it sends its first request
> knowing that NTLM is the auth method to use it KNOWS that the request will
> not be accepted by the server. The server will instead through away all
> incoming data, send back a challange to the client and the client sends back
> a second request to the server that might be received fine if the
> credentials are ok. curl only sends its POST data in that second request
> since if you'd send a 100MB POST, sending the data in the first request as
> well would imply that curl would send the data twice even though it knows
> the first round is pointless. For other auth methods of multi-phase kinds,
> curl can close the connection to shortcut the initial request but since NTLM
> is for connections it cannot do that.
>
> The problem I believe you for example is experiencing is that some proxies
> and servers STUPIDLY insists that the first request cannot send a
> zero-length POST.
>
> So, we're back to my previous suggestion --proxy-anyauth. When curl doesn't
> know the single specific auth method to use, it will do it more
> browser-style and instead try without auth first and then pick the auth the
> server says it should use and re-send the request using one of them. This
> usually works around that content-length: 0 issue since this makes curl use
> another sequence of requests.
>
>
So now we are at a point where I have a workaround, and I am happy. It is up
to you to decide if this is a bug in curl. I still believe it is, but you
know curl, http, and NTLM better than I do.

The reason I believe it is a bug is the following reasons:
1) It seems wrong and confusing that curl works with --proxy-anyauth but not
with --proxy-ntlm when they both use the same authentication system (NTLM)
in the end. Surely --proxy-ntlm could do the 2 request trick like anyauth
does.
2) It seems wrong that when you use --proxy-ntlm a zero length request gets
received by the website at the other end. The user never requested a content
length zero request, so the code is doing something he didn't ask for.
3) Your argument about the 100MB file seems sensible, but with the -d
(rather than -F) option the POST is url-encoded so it is unlikely to be this
long. Futhermore, even if it has to send a 100MB file twice, surely that is
better than what it does at the moment, which is not send it at all (in this
particular set of circumstances).

Your reply says very sensibly that curl sends the data only in the second
request. It looks to me like curl is not doing this (even if that was the
intention) which is a possible bug. I might try and get the source code to
confirm this - at least it would help me understand it better.

Anyway I have a workaround, and I am very grateful for all your help. I'll
leave it up to you to decide if it is a bug, unless you'd like me to do any
testing for you. Thanks again both for the workaround and the detailed
response.

Nick

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-01-09