On 7/6/2017 9:59 AM, Daniel Stenberg wrote:
> Hey,
>
> libcurl has hueristics to include the "Expect: 100-continue" header in
> several circumstances when doing POST and PUT requests.
>
> This header was designed to allow servers to reject a request before
> any data has been sent from the client, so that there would be less
> waste. It is especially useful with various kinds of HTTP auth
> mechanisms during which the 100 can save us from having to resend the
> entire (potenitally huge) POST body.
>
> In the real world however, there seems to be many more broken servers
> than working ones when it comes to supporting 100. If curl sees no
> response within 1000 milliseconds, it continues anyway and this pause
> is a very common reason for annoyance with curl. So many servers just
> don't see or care about Expect:.
>
> Then there's still the large group of servers that send a 100 response
> back immediately on all requests with Expect: even if the auth isn't
> fine, only to deny the request later anyway. Contrary to the intention
> of the header.
>
> The net result of all this is that switching off this Expect: header
> from libcurl requests is a very common thing. In many cases it removes
> a 1000ms pause from the request. In most other cases it makes no
> difference. In a rare few cases, it causes wasted bandwidth and
> additional roundtrips.
>
> The number of requests done "out there" today that unncessarily are
> waiting 1000 ms on each request is probably substantial. Switching the
> default could automatically make a huge amount of curl requests go
> faster in the future.
>
> Is it time to remove the automatic Expect: header use and only do it
> on demand?
My gut response is:
a) leave the default behavior as it is. This is a well known behavior.
Straight "out of the box" configuration reversals "scare me" by
principal (as this one might be harmless, but hey - this is shooting
from the hip!).
b) add a message (perhaps to stderr as well as syslog) that a request
was made, but no response was made within 1000 msec, ideally in a way
that statistics could be collected (which servers never respond, e.g.)

In short, since you had to ask - my gut says this is not a guaranteed
"100% harmless" change - and I would start with a message, or as a
configure option (maybe it is that already) and packagers can decide
and/or learn what works best.

If putting it is configure is new - then you could put it in with the
default as is (e.g., true aka enabled), but add some info to README to
change the configure default to false (aka default request disabled).
And, this also gives you a moment to review the current documentation on
how to change the default in the config file (e.g., maybe what the
configure change does is change the result of a generated (default)
config file rather than actually changing code.

Hope this helps!
>
> We probably need to add a new option that turns on the current/old
> behavior if we do this. Alternatively we would just let the user set
> "Expect: 100-continue" as a custom header and detecting that, we would
> enable the internal logic for it, but that seems very error-prone and
> not very convenient.
>
> Applications that currently disable Expect: header use should of
> course not be affected by this sort of change. They would just disable
> it extra much! =)
>
> Thoughts?
>

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-07-26