By the way I use hiperfifo.c to send HTTPS 40 requests at the same time in
https://curl.haxx.se/libcurl/c/hiperfifo.html

CPU sometimes burst to 69% or 80%, quickly down to 25% then to 0.0%

Code below:
static void fifo_cb(int fd, short event, void *arg)
{
  char s[1024];
  long int rv=0;
  int n=0;
  GlobalInfo *g = (GlobalInfo *)arg;
  (void)fd; /* unused */
  (void)event; /* unused */

  int i=0;
  do {
    s[0]='\0';
    rv=fscanf(g->input, "%1023s%n", s, &n);
    s[n]='\0';
    if(n && s[0]) {
        for(i=0; i<40; i++) {
        //printf("new_conn start\n");
        new_conn("https://test.example.com", arg); //This code I add
      }
    }
    else
      break;
    } while(rv != EOF);
}

Complete code see attachment.

On Thu, Aug 10, 2017 at 9:23 AM, Jack <sipgurume_at_gmail.com> wrote:

>
>
> On Thu, Aug 10, 2017 at 6:33 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>
>> On Wed, 9 Aug 2017, fan fan via curl-library wrote:
>>
>> In my application, I using libcurl to send 40 HTTP requests in every
>>> one second. As I test, this consume much CPU, my application such as:
>>>
>>> main thread for sending request and using epoll listen fds whether read
>>> or write.
>>>
>>> while(running) {
>>> int ret = epoll_wait(epollFd, events, maxEventNum, 10); //10ms
>>>
>>
>> Unconditionally returning every 10 milliseconds like this will make you
>> spend a lot of CPU in vein. Don't do that. libcurl tells you how long it
>> wants you to wait at most. No need to spin around a lap when there's
>> nothing for libcurl to do.
>>
>> Do you also call libcurl to tell there's a timeout if the 10 milliseconds
>> lapse or when do you do that?
>
>
> [Fan] Maybe this is not root cause, because when no http request send,
> CPU almost 0.1%. of course I need to expand waiting time.
> this epoll_wait for listening every easy handle fd, I
> set CURLMOPT_SOCKETFUNCTION, when callback, I add this fd to epoll events.
> of course I set CURLMOPT_TIMERFUNCTION, when timeout, I call curl_multi_socket_action
> again.
>
>>
>>
>> for (i = 0; i < ret; i++) {
>>> if (events[i].events & EPOLLIN) {
>>> curl_multi_socket_action // set CURL_CSELECT_IN
>>> curl_multi_info_read // for reading
>>>
>>
>> I think you should put the curl_multi_info_read() call outside of the
>> loop checking for activities, since it might not have anything to return
>> and then its a bit of a waste to call it up to 40 times.
>>
>> But sure, curl_multi_socket_action() may of course consume CPU since
>> that's the function that will handle the traffic on the socket you give it
>> (and possibly on a few others if necessary).
>
>
> [Fan] So, when do I call curl_multi_info_read?
> I refer to:
> https://curl.haxx.se/libcurl/c/hiperfifo.html
> when socket have something to read, call curl_multi_info_read,
> is it correct?
>
>>
>>
>> As I test, send 100 or 200 or 300 or 400 or 500 in parallel maybe
>>> consume
>>> cpu 100%.
>>>
>>
>> On modern hardware that shouldn't happen.
>
> [Fan] Agree
>
>>
>>
>> I found there are too much EPOLLIN event, this consume much cpu.
>>>
>>
>> What do you mean with too much EPOLLIN event? If libcurl tells you to
>> wait for that, shouldn't you then be glad you get it so that you can
>> download the data?
>
>
> [Fan] I add socket fd to epoll event when CURLMOPT_SOCKETFUNCTION
> callback.
> If there has a lot of easy handle in processing, trigger
> frequently.
> CURLMOPT_TIMERFUNCTION not about epoll fd, is it right?
> CURLMOPT_TIMERFUNCTION only indicating whether time out or
> not.
>
>>
>>
>> 1. Does my use libcurl wrong?
>>>
>>
>> Ineffective at least. But you also didn't show the whole program so its
>> not possible to tell the whole story.
>
> [Fan] Agree
>
>>
>>
>> 2. Have any performance measurement for libcurl process in parallel?
>>>
>>
>> It should easily handle thousands of parallel connections, CPU wise. Very
>> likely tens of thousands too. Of course they may have to struggle with each
>> other for other resources (like bandwidth).
>
> [Fan]Awesome
>
>>
>>
>> --
>>
>> / daniel.haxx.se
>>
>
>

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html