----------------------------------------
> Date: Fri, 6 May 2011 14:31:49 +0200
> From: daniel_at_haxx.se
> To: curl-library_at_cool.haxx.se
> Subject: Re: Controlling the number of open connections with pipelining
>
> On Thu, 5 May 2011, Vladimir Grishchenko wrote:
>
> > The specifics of our app is that we need to fetch content from a limited
> > number of servers, but there may be many requests generated from a single
> > client to a particular server that need to be serviced at the same time,
> > which sounds like a perfect case for using HTTP pipelining.
>
> Possibly. Pipelining does not make them get serviced *at the same time*, just
> queued up on the same physical connection and thus reach the server with less
> time in between the requests compared to doing them serially on the same
> connection without pipelining.
>

Well, perhaps I'm expecting too much, but my understanding was that while requests
and responses are serialized it is possible that that actual processing of the
requests, e.g. database queries, can be done in parallel provided the servers
have enough resources for that. Of course this may or may not
happen in each particular case but nonetheless I think it is worth experimenting
with pipelining, in our case we also own the servers so additional server-side
optimizations for pipelining support are not out of the question.

> Doing N parallell connections to the same server and doing requests on all of
> them will undoubtably be much more like *at the same time*.
>

Possibly :) depending on how they are handled by the server.

> > At the same time we want to cap the number of connections from a client to a
> > particular server to avoid overloading as the number of clients can be
> > large.
>
> Well, a middle-ground might be to cap the amount of simultaneous connections
> to the same host to N. Pretty much the way browsers work.
>

Meaning cap the number of same-host easy handles added to a multi to N? That might
work although HTTP RFC mandates N=2, so the benefit may be too limited.

> > I was able to gather pipelining support is still work in progress
>
> I wouldn't exactly call it work in progress. It is a feature that we have and
> that some people use. It isn't widely used but then pipelining isn't even
> widely used in browsers, so there are probably more quirks to find and iron
> out before it gets completely solid...
>
> --

Point taken.

Thanks,
Vladimir
                                               
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-05-06