Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: curl_multi_timeout and the multi_socket API
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Henrik Holst via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 5 Apr 2021 20:26:57 +0200
Den mån 5 apr. 2021 kl 17:33 skrev Daniel Stenberg <daniel_at_haxx.se>:
> On Mon, 5 Apr 2021, Henrik Holst wrote:
>
> > On a side note (and perhaps I should have written this in a separate
> mail) I
> > did notice that the timer function callback returns far too low values
> for
> > long polling http servers if the user haven't set CURLOPT_TIMEOUT.
> Caveat
> > here is that I have only tested this with 7.58 and 7.68 as of yet and
> not
> > with HEAD.
>
> As I mentioned before, we've removed the need for polling since then so a
> lot
> of the really low-value timeouts are gone for a normal build. Now libcurl
> waits on a socket even during a threaded name resolve.
>
> > Here is one such run with CURLOPT_VERBOSE set, unfortunately the server
> is
> > not public so replication might be hard:
>
> If this problem was widespread/common, then you wouldn't need to use a
> private
> server to reproduce surely?
>
I don't know of any public http server doing long poll and this particular
server that does employ long polling was what I was working with at the
moment and unfortunately it's private service (not under my control since
it's an external service) but the server is most likely not the issue here,
just that it employs long poll so that there is no read event quickly after
connect.
>
> Simply based on the fact that libcurl is fairly well used and nobody
> (else) is
> reporting "100% cpu use", I don't think this is a common problem. But I'm
> also
> sure there are bugs still to fix and I'm also sure that we've fixed bugs
> since
> the versions you run. Some of those might change these things.
>
I can agree with that, but as you could see from my output, curl did give
me a 1ms timeout as the last timeout to use. So every 1ms I did call
curl_multi_socket_action() with CURL_SOCKET_TIMEOUT, isn't that what I'm
supposed to do when the callback tells me that the timeout is 1ms?
And this particular run with 1ms did not lead to 100% cpu, it was when curl
gave me 0ms as timeout as the last timeout which lead me to call
curl_multi_socket_action() every 0ms, which turned it into a busy wait loop
and hence the 100% cpu. Shouldn't curl return -1 as infinte timeout instead
there since it's waiting for read on a connected socket (after having sent
the request in full) with no defined CURLOPT_TIMEOUT?
The 100% cpu was just a side effect of the 0 timeout in my particular code,
my main question is why curl told me that the timeout should be 1/0 and not
-1, unless I set CURLOPT_TIMEOUT and I get a "proper value" instead?
/HH
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-04-05
Date: Mon, 5 Apr 2021 20:26:57 +0200
Den mån 5 apr. 2021 kl 17:33 skrev Daniel Stenberg <daniel_at_haxx.se>:
> On Mon, 5 Apr 2021, Henrik Holst wrote:
>
> > On a side note (and perhaps I should have written this in a separate
> mail) I
> > did notice that the timer function callback returns far too low values
> for
> > long polling http servers if the user haven't set CURLOPT_TIMEOUT.
> Caveat
> > here is that I have only tested this with 7.58 and 7.68 as of yet and
> not
> > with HEAD.
>
> As I mentioned before, we've removed the need for polling since then so a
> lot
> of the really low-value timeouts are gone for a normal build. Now libcurl
> waits on a socket even during a threaded name resolve.
>
> > Here is one such run with CURLOPT_VERBOSE set, unfortunately the server
> is
> > not public so replication might be hard:
>
> If this problem was widespread/common, then you wouldn't need to use a
> private
> server to reproduce surely?
>
I don't know of any public http server doing long poll and this particular
server that does employ long polling was what I was working with at the
moment and unfortunately it's private service (not under my control since
it's an external service) but the server is most likely not the issue here,
just that it employs long poll so that there is no read event quickly after
connect.
>
> Simply based on the fact that libcurl is fairly well used and nobody
> (else) is
> reporting "100% cpu use", I don't think this is a common problem. But I'm
> also
> sure there are bugs still to fix and I'm also sure that we've fixed bugs
> since
> the versions you run. Some of those might change these things.
>
I can agree with that, but as you could see from my output, curl did give
me a 1ms timeout as the last timeout to use. So every 1ms I did call
curl_multi_socket_action() with CURL_SOCKET_TIMEOUT, isn't that what I'm
supposed to do when the callback tells me that the timeout is 1ms?
And this particular run with 1ms did not lead to 100% cpu, it was when curl
gave me 0ms as timeout as the last timeout which lead me to call
curl_multi_socket_action() every 0ms, which turned it into a busy wait loop
and hence the 100% cpu. Shouldn't curl return -1 as infinte timeout instead
there since it's waiting for read on a connected socket (after having sent
the request in full) with no defined CURLOPT_TIMEOUT?
The 100% cpu was just a side effect of the 0 timeout in my particular code,
my main question is why curl told me that the timeout should be 1/0 and not
-1, unless I set CURLOPT_TIMEOUT and I get a "proper value" instead?
/HH
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-04-05