Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

100% CPU consumption with in multi interface by using CURL_WRITEFUNC_PAUSE #6356

Closed
piru opened this issue Dec 22, 2020 · 1 comment
Closed
Assignees
Labels

Comments

@piru
Copy link

piru commented Dec 22, 2020

I did this

Return CURL_WRITEFUNC_PAUSE from CURLOPT_WRITEFUNCTION to pause a transfer. Specifically:

/*
 * Demonstrate the application busyloop issue with multi interface when
 * CURLOPT_WRITEFUNCTION returns CURL_WRITEFUNC_PAUSE.
 *
 * While the application does request pausing of the transfer by itself
 * it should still not lead to 100% CPU consumption. For example Webkit
 * cURL backend does use this functionality to pause transfers, and does
 * trigger this issue.
 *
 * Written by Harry Sintonen <sintonen@iki.fi>. Heavily based on multi.c
 * example by  Daniel Stenberg, Copyright (C) 1998 - 2020, Daniel Stenberg,
 * <daniel@haxx.se>, et al.
 */

#include <stdio.h>
#include <string.h>

/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>

/* curl stuff */
#include <curl/curl.h>

static size_t writefunc(void *data, size_t size, size_t nmemb, void *userp)
{
  printf("data %p size %lu nmemb %lu userp %p\n", data, size, nmemb, userp);
  return CURL_WRITEFUNC_PAUSE;
}

int main(void)
{
  CURL *handle;
  CURLM *multi_handle;

  int still_running = 0; /* keep number of running handles */

  CURLMsg *msg; /* for picking up messages with the transfer status */
  int msgs_left; /* how many messages are left */

  /* Allocate one CURL handle */
  handle = curl_easy_init();

  /* set the options */
  curl_easy_setopt(handle, CURLOPT_URL, "https://curl.se/");
  curl_easy_setopt(handle, CURLOPT_VERBOSE, 1L);
  curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, writefunc);

  /* init a multi stack */
  multi_handle = curl_multi_init();

  /* add the transfer */
  curl_multi_add_handle(multi_handle, handle);

  /* we start some action by calling perform right away */
  curl_multi_perform(multi_handle, &still_running);

  while(still_running) {
    struct timeval timeout;
    int rc; /* select() return code */
    CURLMcode mc; /* curl_multi_fdset() return code */

    fd_set fdread;
    fd_set fdwrite;
    fd_set fdexcep;
    int maxfd = -1;

    long curl_timeo = -1;

    FD_ZERO(&fdread);
    FD_ZERO(&fdwrite);
    FD_ZERO(&fdexcep);

    /* set a suitable timeout to play around with */
    timeout.tv_sec = 1;
    timeout.tv_usec = 0;

    curl_multi_timeout(multi_handle, &curl_timeo);
    if(curl_timeo >= 0) {
      timeout.tv_sec = curl_timeo / 1000;
      if(timeout.tv_sec > 1)
        timeout.tv_sec = 1;
      else
        timeout.tv_usec = (curl_timeo % 1000) * 1000;
    }

    /* get file descriptors from the transfers */
    mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);

    if(mc != CURLM_OK) {
      fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);
      break;
    }

    /* On success the value of maxfd is guaranteed to be >= -1. We call
       select(maxfd + 1, ...); specially in case of (maxfd == -1) there are
       no fds ready yet so we call select(0, ...) --or Sleep() on Windows--
       to sleep 100ms, which is the minimum suggested value in the
       curl_multi_fdset() doc. */

    if(maxfd == -1) {
#ifdef _WIN32
      Sleep(100);
      rc = 0;
#else
      /* Portable sleep for platforms other than Windows. */
      struct timeval wait = { 0, 100 * 1000 }; /* 100ms */
      rc = select(0, NULL, NULL, NULL, &wait);
#endif
    }
    else {
      /* Note that on some platforms 'timeout' may be modified by select().
         If you need access to the original value save a copy beforehand. */
      rc = select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout);
    }

    switch(rc) {
    case -1:
      /* select error */
      break;
    case 0: /* timeout */
    default: /* action */
      curl_multi_perform(multi_handle, &still_running);
      break;
    }
  }

  /* See how the transfers went */
  while((msg = curl_multi_info_read(multi_handle, &msgs_left))) {
    if(msg->msg == CURLMSG_DONE) {
      /* Find out which handle this message is about */
      if(msg->easy_handle == handle) {
        printf("HTTP transfer completed with status %d\n", msg->data.result);
      }
    }
  }

  curl_multi_cleanup(multi_handle);

  /* Free the CURL handle */
  curl_easy_cleanup(handle);

  return 0;
}

I expected the following

Transfer being paused without the process bysylooping.

curl/libcurl version

curl 7.72.0 (x86_64-pc-linux-gnu) libcurl/7.72.0 OpenSSL/1.1.1i zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.8.0 nghttp2/1.42.0 librtmp/2.3
Release-Date: 2020-08-19
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

operating system

Linux hostname 5.9.0-5-amd64 #1 SMP Debian 5.9.15-1 (2020-12-17) x86_64 GNU/Linux

@bagder
Copy link
Member

bagder commented Dec 22, 2020

Reproduces for me.

@bagder bagder self-assigned this Dec 22, 2020
bagder added a commit that referenced this issue Dec 22, 2020
... as the socket might be readable all the time when paused and thus
causing a busy-loop.

Reported-by: Harry Sintonen
Fixes #6356
@bagder bagder added the HTTP/2 label Dec 22, 2020
@bagder bagder closed this as completed in c7f95fa Dec 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging a pull request may close this issue.

2 participants