cURL / Mailing Lists / curl-library / Single Mail

curl-library

curl multi pipelining and CURLOPT_CONNECTTIMEOUT

From: Török Edwin <edwin+ml-curl_at_etorok.net>
Date: Thu, 24 Jan 2013 17:46:17 +0200

Hi,

I am using the curl multi interface and I see a lot of timeouts on slowish links (all fine on fast links of course).
I have concluded that CURLOPT_CONNECTTIMEOUT sets not only the timeout for connect(), but also the timeout
for starting to receive a reply for a pipelined request.

Due to how pipelining works this means that all N-1 pipelined requests must finish during the CONNECTTIMEOUT
of the Nth request, otherwise all N (and later) requests are considered as timed out.
Is this intended behaviour?

I would've expected CURLOPT_CONNECTTIMEOUT to set only the timeout for connect(). For pipelined connections
as long as _any_ easy handle is receiving data on that connection I don't want the 'connect' timeout
of other pipelined requests to trigger. Is there a way to do that without modifying curl?

Alternatively I could queue up all requests that would go to the same host and only add the easy handle
to a multi handle when previous request is finished, but it'd be nice if curl could do this itself.

Note: I'm actually interested in persistent connections only, not necesarely pipelining (that is just a nice addition),
but in the multi interface I don't see a way to request that only: it is either both (MOPT_PIPELINING on), or neith
(MOPT_PIPELINING off, separate connections).

Here is how I encountered this issue:
rc = curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 60);
...
rc2 = curl_multi_setopt(x->multi, CURLMOPT_PIPELINING, 1);
....

If I add lots of easy handles to the multi handle (lets say 10 requests to same host each with a 1MB reply)
then the last requests tends to timeout with an error on a slowish link:
Timeout was reached, Operation timed out after 60009 milliseconds with 0 out of 0 bytes received

At first I thought there is some other timeout option that I'm not setting, but setting this didn't help
(its the default value anyway):
rc = curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 0);

Changing CURLOPT_CONNECCTIMEOUT to 10s will cause all the multi pipelined transfers to time out after ~10000 ms, so that
supports my original idea that CONNECTTIMEOUT influences timeout for pipelined requests.

Best regards,
--Edwin
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-01-24