Hi,

I'm facing a problem in using libcurl in a multi-threaded application.

The application has 8 data processing threads and 8 libevent threads to run libcurl. Data processing work is pretty heavy which contributes to 80% of CPU that this application consumes in total. But non of these threads takes 100%, and none of 8 CPUs of the machine is completely busy.

This application is to POST data, at least 500K bytes content, to an HTTP server.

Normally I send 20 concurrent requests to this application, and requests are distributed to data processing thread in a round robin manner and processed, then passed to libcurl to send out in a round robin manner too.

Now I'm facing a problem that libcurl has many failures under capacity.

<08:36:44.800 **ERR** HCM 32020:32245 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[1]: failed due to Couldn't connect to server
<08:36:44.800 **ERR** HCM 32020:32202 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[2]: failed due to Couldn't connect to server
<08:36:44.821 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[3]: failed due to Failed sending data to the peer
<08:36:44.836 **ERR** HCM 32020:32243 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[7]: failed due to Couldn't connect to server
<08:36:44.836 **ERR** HCM 32020:32240 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[4]: failed due to Failure when receiving data from the peer
<08:36:44.856 **ERR** HCM 32020:32244 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[8]: failed due to Failure when receiving data from the peer
<08:36:44.856 **ERR** HCM 32020:32202 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[9]: failed due to Couldn't connect to server
<08:36:44.868 **ERR** HCM 32020:32240 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[12]: failed due to Couldn't connect to server
<08:36:44.868 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[10]: failed due to Failure when receiving data from the peer
<08:36:44.884 *WRN* HCM 32020:32243 0:0>[onTransferDone(HttpClientWorker.cpp:362)] transfer[20]: response code 501
<08:36:44.887 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[14]: failed due to Failed sending data to the peer
<08:36:44.887 **ERR** HCM 32020:32244 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[22]: failed due to Couldn't connect to server
<08:36:44.946 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[25]: failed due to Failure when receiving data from the peer
<08:36:45.056 **ERR** HCM 32020:32245 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[23]: failed due to Server returned nothing (no headers, no data)
<08:36:45.056 *WRN* HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:362)] transfer[32]: response code 414
<08:36:45.133 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[46]: failed due to Failed sending data to the peer
<08:36:45.337 *WRN* HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:362)] transfer[54]: response code 414
<08:36:45.880 *WRN* HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:362)] transfer[104]: response code 414
<08:36:50.048 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[39]: failed due to Couldn't connect to server
<08:36:50.248 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[61]: failed due to Couldn't connect to server
<08:36:50.328 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[68]: failed due to Couldn't connect to server
<08:36:50.880 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[111]: failed due to Timeout was reached
<08:36:51.009 *WRN* HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:362)] transfer[471]: response code 414
<08:36:56.008 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[478]: failed due to Timeout was reached
<08:37:01.047 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[330]: failed due to Timeout was reached
<08:37:01.153 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[338]: failed due to Couldn't connect to server
<08:37:01.348 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[345]: failed due to Timeout was reached
<08:37:02.122 **ERR** HCM 32020:32219 0:0>[onTransferDone(HttpClientWorker.cpp:370)] transfer[409]: failed due to Server returned nothing (no headers, no data)

But when I split this application to two parts, one part processing data and one part doing curl, all requests are finished successfully.

So what might causes the curl failures in single application mode?

Some codes of this application which are related to libcurl are as followings.

// disable some signals
    signal(SIGPIPE, SIG_IGN);
    signal(SIGALRM, SIG_IGN);
    signal(SIGCHLD, SIG_IGN);

// start multi handle
        if ((m_curlm = curl_multi_init()) == NULL)
        {
            hcmError("curl_multi_init error\n");
            return false;
        }

        curl_multi_setopt(m_curlm, CURLMOPT_MAXCONNECTS, getMaxConnections()); // 20 by default
        curl_multi_setopt(m_curlm, CURLMOPT_PIPELINING, isPipelining()); // false by default

        curl_multi_setopt(m_curlm, CURLMOPT_TIMERFUNCTION, doStartTimer);
        curl_multi_setopt(m_curlm, CURLMOPT_TIMERDATA, this);
        curl_multi_setopt(m_curlm, CURLMOPT_SOCKETFUNCTION, doPollSocket);
        curl_multi_setopt(m_curlm, CURLMOPT_SOCKETDATA, this);

// start easy handle
    if ((m_curl = curl_easy_init()) == NULL)
    {
        hcmError("curl_easy_init error\n");
        return false;
    }
    curl_easy_setopt(m_curl, CURLOPT_PRIVATE, this);

    curl_easy_setopt(m_curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

    curl_easy_setopt(m_curl, CURLOPT_HTTPPOST, 1);
    curl_easy_setopt(m_curl, CURLOPT_URL, getRequestUrl().c_str()); // URL always in a format of http://<ip-address>:<port>/service,

    m_headers = curl_slist_append(m_headers, "Content-Type: application/xml");
    m_headers = curl_slist_append(m_headers, "Expect:");
    curl_easy_setopt(m_curl, CURLOPT_HTTPHEADER, m_headers);

    curl_easy_setopt(m_curl, CURLOPT_POSTFIELDSIZE_LARGE, req.getPayloadLen());
    curl_easy_setopt(m_curl, CURLOPT_POSTFIELDS, req.getPayload());

    curl_easy_setopt(m_curl, CURLOPT_HEADERFUNCTION, doWriteHeader);
    curl_easy_setopt(m_curl, CURLOPT_WRITEHEADER, this);
    curl_easy_setopt(m_curl, CURLOPT_WRITEFUNCTION, doWritePayload);
    curl_easy_setopt(m_curl, CURLOPT_WRITEDATA, this);

    curl_easy_setopt(m_curl, CURLOPT_TIMEOUT_MS, m_worker->getTimeout()); // 60 seconds by default
    curl_easy_setopt(m_curl, CURLOPT_NOSIGNAL, 1);

    if (curl_multi_add_handle(m_worker->m_curlm, m_curl) != CURLM_OK)
    {
        hcmError("curl_multi_add_handle error\n");
    }

thanks,
yamin

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-08-31