cURL / Mailing Lists / curl-library / Single Mail

curl-library

[PROF RFC 2/2] profiling of libcurl by curl-loader, starting stage

From: Robert Iakobashvili <coroberti_at_gmail.com>
Date: Wed, 18 Apr 2007 16:54:12 +0300

The affect of incrementing number of loading clients
by 20 clients each second. Starting stage.

CURL VERSION: 7.16.2

HOST MACHINE and OPERATING SYSTEM:
 linux, debian with vanilla kernel 2.6.20.7

COMPILER:
gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

CURL CONFIGURATION:
configure --prefix $(CURL_BUILD) \
        --enable-thread \
        --enable-ipv6 \
        --with-random=/dev/urandom \
        --with-ssl=/usr/include/openssl \
        --enable-shared=no \
        CFLAGS="-pg -g -O3 -ffast-math -finline-functions -funroll-all-loops \
        -finline-limit=1000 -mmmx -msse -foptimize-sibling-calls
-mtune=pentium4 \
        -mcpu=pentium4"

COMPILATION OPTIONS:
are the same for the application, libcurl and libevent:

-pg -g -O3 -ffast-math -finline-functions -funroll-all-loops \
-finline-limit=1000 -mmmx -msse -foptimize-sibling-calls -mtune=pentium4 \
-mcpu=pentium4

    CURL_EASY_SETOPT:

The curl_easy_setopt used are:
CURLOPT_IPRESOLVE
CURLOPT_INTERFACE
CURLOPT_NOSIGNAL
CURLOPT_URL with http://localhost/ACE-INSTALL.html (104K file)
CURLOPT_CONNECTTIMEOUT 5
CURLOPT_FRESHCONNECT 0
CURLOPT_DNS_CACHE_TIMEOUT with -1
CURLOPT_VERBOSE with 1
CURLOPT_DEBUGFUNCTION with client_tracing_function
CURLOPT_DEBUGDATA, cctx;
CURLOPT_WRITEFUNCTION with do_nothing_write_function
CURLOPT_SSL_VERIFYPEER 0 /* not used, http-url */
CURLOPT_SSL_VERIFYHOST 0 /* not used, http-url */

CURLOPT_PRIVATE, cctx
CURLOPT_WRITEDATA, cctx - for hyper
CURLOPT_ERRORBUFFER, bctx->error_buffer

CURLOPT_FOLLOWLOCATION, 1 /* not used, no 3xx */
CURLOPT_UNRESTRICTED_AUTH, 1 /* not used, no auth */
CURLOPT_MAXREDIRS, -1 /* not used, no 3xx */
CURLOPT_USERAGENT, bctx->user_agent /*MSIE-6 like string */
CURLOPT_COOKIEFILE, "" /* not used, no cookies*/

SERVER:
lighttpd-1.4.13 (ssl) from debian,
the stock configuration file with the more strings
added:
server.event-handler = "linux-sysepoll"
server.max-fds = 32000

FILE FETCHED:
104K static file

CLIENT APPLICATION:
curl-loader

CLIENT LOADING MODES:
Hyper mode - based on hipev.c example and using epoll ()
based demultiplexing with further curl_multi_socket_action ()

CLIENT LOAD:

Each load has started with 0 clients and added 20 clients each
second. Initial number of clients -0, average - 1000, final -2000.

Two loads have been run each for the time 100 seconds.
- with 0 msec between the fetched file and the new request;
- with 1000 msec between the fetched file and the new request

PROFILING METHODS:
gprof - provides application level information - collected
to files suffixed *.prof, where *.lprof files are including detailed
mapping of profiling to sources

oprofile- whole system information, including glibc
and kernel functions, filtered for the relevant curl-loader
application to files suffixed *.op;

RESULTING FILES:

hyper-inc20.profiling.tar.bz2:

hyper-inc20-0.prof - gprof res of hyper mode, 0 msec delay between requests;
hyper-inc20-0.lprof - same as above, but with mapping to sources
hyper-inc20-0.op - oprofile res of hyper mode, 0 msec delay between requests;

hyper-inc20-1000.prof - gprof res of hyper mode, 1000 msec delay b requests;
hyper-inc20-1000.lprof - same as above, but with mapping to sources;
hyper-inc20-1000.op - oprofile res of hyper mode, 1000 msec delay b requests;

The recommendations and analyses of curl gurus would be very much
appreciated. Your feedback would be highly valued.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...................................................................
Navigare necesse est, vivere non est necesse
...................................................................

Received on 2007-04-18