curl-library
Using libcurl sending much more HTTP request in parallel
Date: Wed, 9 Aug 2017 16:44:43 +0800
Hi all,
This is my fist post to this list, so I want to use this opportunity to
thanks all of you (mainly Daniel) for the development, maintenance and
support efforts you invest on this amazing tool "cUrl".
In my application, I using libcurl to send 40 HTTP requests in every
one second.
As I test, this consume much CPU, my application such as:
main thread for sending request and using epoll listen fds whether read
or write.
while(running) {
int ret = epoll_wait(epollFd, events, maxEventNum, 10); //10ms
for (i = 0; i < ret; i++) {
if (events[i].events & EPOLLIN) {
curl_multi_socket_action // set CURL_CSELECT_IN
curl_multi_info_read // for reading
//consume much cpu
}
if (events[i].events & EPOLLOUT) {
curl_multi_socket_action // set CURL_CSELECT_OUT
}
}
//Sending 40 http request in every one second in parallel
}
main thread add every easy handle fd to epoll.
As I test, send 100 or 200 or 300 or 400 or 500 in parallel maybe consume
cpu 100%.
I found there are too much EPOLLIN event, this consume much cpu.
By the way, I use HTTPS.
I have questions:
1. Does my use libcurl wrong?
2. Have any performance measurement for libcurl process in parallel?
3. Any suggestion?
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-08-09