On 3 May 2016, at 16:31, Alexandre BOUIN <abouin_at_telitem.com> wrote:

> We performed a first test, using python. Once half the request (20kB only) has been downloaded, we start a new request.
> This is a bit like if we were using 2 pipelines.
> This way all the bandwidth is consumed.

It's not like using 2 pipelined requests. It's like using two connections, each with their own TCP buffers etc. With 2 pipelined requests, you only get one connection, and your two requests execute serially.

I'd suggest you EITHER use two (or more) *connections* (not pipelining), or BETTER find out why one connection is not using all your bandwidth but two are.

I would guess you might have a bandwidth delay product issue:
  https://en.wikipedia.org/wiki/Bandwidth-delay_product

Specifically you may not be able to open your TCP window sufficiently large to perform the download. Imagine (as thought experiment) you have a 1 gigabit per second connection to the moon, but it takes 2.5 seconds for any information to get there (and another 2.5 seconds to get back). Assume your tcp connection has a maximum window size of 64 kilobytes, and (with selective acknowledgement) a maximum of two data frames of 64k can be in flight at once. Therefore, you move 128 kilobytes every 5 seconds, or 205 kbits/s on your 1 gigabit/s link. Adding another tcp session will also run at 205kbit/s (maximum). There's no way of improving that without using a larger window size, or a different protocol. This is why RFC1323 (TCP window scale) option was invented, to allow larger window sizes). https://en.wikipedia.org/wiki/TCP_window_scale_option

If that sounds like your problem, then I'd suggest you investigate why you aren't getting sensible TCP window sizes.

Of course it could be another issue, such as the server-side rate-limiting of each TCP connection.

But whatever the issue is, I'd be pretty sure you won't solve it with pipelining.

-- 
Alex Bligh
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:  https://curl.haxx.se/mail/etiquette.html