On Sun, 22 Jun 2014, Daniel Stenberg wrote:

> On Tue, 17 Jun 2014, m brandenberg wrote:
>
>> I'm looking at http pipelining and trying to resolve multi_setopt()
>> documentation with what I'm seeing in the code. In create_conn() in url.c
>> (7.37.0 release) I see:
>
>> If *both* CURLOPTM_MAX_TOTAL_CONNECTIONS and CURLOPTM_MAX_HOST_CONNECTIONS
>> are set to positive values and neither limit has been reached, then
>> libcurl will only open a new connection (i.e. breadth-first). Otherwise,
>> it prefers pipelining over a new connection (i.e. depth-first) and
>> in creating new connections, a limit value of 0 is always unlimited,
>> never interpreted as '1'.
>>
>> Is that the intended behavior?
>
> Yes I think so. Is there any problems with that logic you think or is it
> contradicing the existing docs?

Not necessarily a problem in the logic but I think it isn't reflected
in the docs. If someone is writing an ambitious/aggressive app and is
using pipelining in pursuit of performance, they are likely to be
interested in connection management. Excessive connections and connection
setup are common failure points.

So I've been trying to clarify the behavior starting with the above.
I believe it's contrary to the following which shows up in several
places in curl_multi_setopt():

    However, for backwards compatibility, setting it to 0 when
    CURLMOPT_PIPELINING is 1 will not be treated as unlimited.
    Instead it will open only 1 connection and try to pipeline on it.

(This one from CURLMOPT_MAX_HOST_CONNECTIONS.) In fact, setting
the value to '0' *is* treated as unlimited. What changes is that
existing connections are pipelined in preference to creating new
connections. (Also not certain how this is backwards compatibility
as there was no pipelining before, but that's a nit...)

What may be surprising to a developer is that removing one of these
settings will silently change connection behavior. If they were using
both and, for example, removed the MAX_TOTAL_CONNECTIONS limit as
being redundant given their access pattern and MAX_HOST_CONNECTIONS
setting, they'd instead get a very different connection pattern. It
might actually be a better pattern for some usages. But I'm not
calling for an option to force pipelining-first behavior.

The other documentation points are the conditions where pipelining
is permitted. Current rules appear to be: GET and HEAD requests
only and only on connections having only GET/HEAD requests. That's
useful to know in itself. There's also an implicit idempotency
assumption about GET/HEAD requests here that allows them to be
retried without problems of server state change. That's probably
valid for most use cases but still worth documenting for those
special cases where the assumption doesn't hold.

--
Monty Brandenberg, Software Engineer                               MCB, Inc.
mcbinc_at_panix.com                                             P.O. Box 425292
mcbinc_at_pobox.com                                   Cambridge, MA  02142-0006
617.864.6907
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html