On Thu, 13 Mar 2008, Raf Nulens wrote:

>> This is a silly server/site that does this, it isn't libcurl's fault. It
>> can but adapt to the circumstances.
>
> But in my traces i've never seen that server return a 100-continue...

If that's the case, then your application has turned off the "Expect:
100-continue" header libcurl normally uses. But of course, as we've found out,
that header is pointless on this site anyway.

> The first header we send (using CURLOPT_POSTFIELDS) looks like this:
>
> POST /ws/1/track/ HTTP/1.1
> Host: musicbrainz.org
> Accept: */*
> Content-Length: 0
> Content-Type: application/x-www-form-urlencoded
>
> Server responds with 401
>
> Second header:
>
> POST /ws/1/track/ HTTP/1.1
> Authorization: Digest username="doesntmatter_at_example.com",
> realm="musicbrainz.org", nonce="fa87522a9a8ca783ebd634ec209375e41205399036",
> uri="/ws/1/track/", response="ecf90cae4325454acf213a7cf9d10c28"
> Host: musicbrainz.org
> Accept: */*
> Content-Length: 11
> Content-Type: application/x-www-form-urlencoded
>
> Server responds with 401
>
> this is the way it should work.

Again you're speaking in "should" terms and while I could possibly agree to
this statement, this is not how this server works and not how most of other
deployed servers "out there" work. And in fact it isn't how HTTP works.

The server doesn't respond with a 401 until *AFTER* the entire posted data (as
specified in the Content-Length:) has been sent. There's no opportunity for
libcurl to stop sending this data in vain. There really isn't.

Just to really really make sure you hadn't found a weird state in libcurl or
in this server that I didn't spot with my previous test, I modified my code
now to also work with POSTFIELDS and even with disabling the Expect: header
just to make my test code able to match both your use cases in which you say
one of them works and one doesn't.

I claim none of the cases work the way you want them to, but instead both of
them work the way I say they work and I also claim that there's nothing
libcurl can do to avoid this extra sending of the data in case of
authentication errors.

>> Disabling 100-continue will only take away the only means the server would
>> have to deny the data to get posted when it hasn't been authorized
>> properly. So no, we don't expect removing the header to improve this
>> situation.
>
> But its not an obligation to use the 100-continue header when using digest.

There's never an obligation to use it, but it is the only real chance a client
can offer a server to deny the sending of the data that is otherwise possibly
sent in vain to the server even though the request is denied authentication.

> Libcurl should just wait with sending binary data until the 401 response is
> received from the server.

No. That's not how HTTP works in general, that's not how HTTP POSTs are made
according to the spec and that's not how this server works.

It is *OBVIOUS* and quite clear when you run my test program to this server
that it simply doesn't respond _anything_ (apart from the dummy 100 Continue)
until the *ENTIRE* post is sent.

Are you denying this? It seems you're hanging on to your stand-point without
absorbing what I say even though I've confirmed my statements with multiple
actual live tests against this server.

> And this _does_ work this way when we're using CURLOPT_POSTFIELDS.

Nope. You can easily verify that I'm right and that you're wrong by using
wireshark or CURLOPT_DEBUGFUNCTION. Or simply compiling and running my test
code.

It works the same way with CURLOPT_POSTFIELDS as with a read callback. I've
tried CURLOPT_POSTFIELDS with 10K POSTs and 150K posts and they all work
identically.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html