curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Using libcurl sending much more HTTP request in parallel

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Fri, 11 Aug 2017 11:12:55 +0200 (CEST)

On Thu, 10 Aug 2017, Jack wrote:

>> Unconditionally returning every 10 milliseconds like this will make you
>> spend a lot of CPU in vein. Don't do that.

> [Fan] Maybe this is not root cause, because when no http request send, CPU
> almost 0.1%.

Sure, but doing like that unconditionally is usually a sign of not having
understood the libcurl event API correctly.

>> I think you should put the curl_multi_info_read() call outside of the loop
>> checking for activities

> [Fan] So, when do I call curl_multi_info_read?

First, curl_multi_info_read() is super fast to call so its not like this is
going to cause you a lot of wasted CPU. I just think it felt wrong to do it
that often. You can of course decide yourself when to call it.

You call curl_multi_info_read() to figure out what transfers that have
completed, right?

I would first handle all the events that need taking care of, *then* I would
loop over curl_multi_info_read() to see which transfers that ended. Comparing
with the hyperfifo example isn't really fair since that's using libevent which
delivers each event one by one to a callback so it can't easily wait with the
call to curl_multi_info_read().

> CURLMOPT_TIMERFUNCTION not about epoll fd, is it right?
> CURLMOPT_TIMERFUNCTION only indicating whether time out or not.

CURLMOPT_TIMERFUNCTION gives you the longest time from that moment until you
should call socket_action() with the TIMEOUT argument. If another event
triggers before that timer has lapsed, chances are you'll get an updated
timeout so that many timeouts never trigger but are simply moved further away
in time.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2017-08-11