Hi,

Firstly, I'm a cURL beginner.

I'm writing a small web server benchmarking app that attempts to send many simultaneous HTTP requests to a single file on a web server.

I'm using Linux pthreads for creating the multiple threads (actually based on the multithread.c sample code included in the curl-devel package).
CPU is x86_64 arch.

All is well for up to about 50 threads, but even then I notice that the time required for the threads to complete increases exponentially.

Threads: 1 Average time, 0.003281 seconds
Threads: 5 Average time, 0.007052 seconds
Threads: 10 Average time, 0.012198 seconds
Threads: 50 Average time, 0.052749 seconds
Threads: 100 Average time, 0.103197 seconds
Threads: 500 Average time, 2.994846 seconds
Threads:1000 Average time, 7.310746 seconds

You can see that between 100 and 500 threads, the time required goes up by 30 when only 5 times the number of requests are made.

The CPU (Quad core) sits unused during these tests (maybe 2% utilisation), the CPU on the web server (PII 400) also sits idle. The Net connection peaks at around 3Mbps.

I ran "strace" over the program and found that it pauses for long periods (many seconds) on a calls to "futex" with FUTEX_WAIT. Maybe the library is experiencing some kind of deadlock?

Is this normal behaviour for cURL under large threading loads?
Any suggestions for debugging this? Happy to provide source code.

Kind Regards,
Nick George
_________________________________________________________________
Never miss another e-mail with Hotmail on your mobile.
http://www.livelife.ninemsn.com.au/article.aspx?id=343869
Received on 2008-06-19