On Fri, 31 Oct 2014, Carlo Wood wrote:

> Well, it all comes down to if you want to support using the same
> multi-handle for different servers or not. My application, written when I
> didn't have to support pipelining at all, would need to be almost rewritten
> when I want to use a multi handle per server :/.

I'm certainly not sure how all people usually use libcurl with pipelining, but
I would assume that they mostly enable pipelining for the single multi handle
they use or all the handles they use.

> The only way that this would be useful therefore is when the user is able to
> set a limit on the number of connections for a *specific* host:port, where
> we are assuming that the application knows which host:port supports
> pipelining :/.

.. which I still think is the exception.

> Just to check we're on the same page. Usually when I work on a library (or
> protocol design) the rule of thumb is that if there is even ONE reasonable
> use case thinkable then it has to be supported (if possible).

First, we of course want to support all reasonable use cases that can be
supported without too much sacrifice.

I'm bringing up this "you" vs "how I've seen others use it" over and over
again since you keep saying absolute things about how things should work when
you have pipelining enabled in a general sense, when in fact they are
requirements for YOUR specific setup and YOUR specific use of pipelining and
not at all necessarily for how others use it. I only do it to emphasize that
we've made decisions in the past based on actual existing users of pipelining.

Everything we add to libcurl, every line of code, every minor option and
combinations of options, every unintended feature combination, we need to
support for a very very long time ahead and we don't break backwards
compatiblity. So everything is a future burdon and we don't pile on more stuff
to carry unless we have to. So yes, we need proper motivation to add stuff. So
yes, what's "likely" to be usuable, used or a good idea is very interesting.
What's likely to not be used or to be a good idea, is less interesting to add.

I'm not dismissing your usecase, and I really appreciate your eyes and hands
on the pipelining code as I believe it'll make it better (for everyone).

> it seems that also 8 connections is not appreciated by the server (don't ask
> me why - I have no control about that at all; I know nothing of that server
> except what I can empirically determine.

Both desktop and mobile browsers these days will open 6 to 8 (or more)
connections to every host name (if it serves enough objects), so servers not
handling such limits might have more problems.

Also, shouldn't you then set the MAX_CONNECTIONS limit to 2 or 4 or something?
Or do you really want it to be 8 or 16 for other hosts?

> One possible way to achieve this is by using the callback that you
> mentioned. I really like that idea.

> If every time a connection is made to (to libcurl) unknown service (ie, a
> new bundle is created) a callback to the user application happens where they
> can specify things like maximum number of connections and whether or not the
> server must be assumed to support pipelining, be blacklisted, or stay
> undetermined, then ALL of this would be solved wonderfully!

I'm glad you like it and it seems to work out for you. I looked over your
patch quickly and it looked good. There may be some minor nit but the general
approach seems totally cool.

Does it satisfy all your needs for setting maximum number of connections etc
or is there something that it doesn't cover?

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html