On Thu, 19 Jun 2008, Nick George wrote:

> I'm writing a small web server benchmarking app that attempts to send many
> simultaneous HTTP requests to a single file on a web server.

Using c-ares? What libcurl version? Which Linux kernel?

> All is well for up to about 50 threads, but even then I notice that the time required for the threads to complete increases exponentially.
>
> Threads: 1 Average time, 0.003281 seconds
> Threads: 5 Average time, 0.007052 seconds
> Threads: 10 Average time, 0.012198 seconds
> Threads: 50 Average time, 0.052749 seconds
> Threads: 100 Average time, 0.103197 seconds
> Threads: 500 Average time, 2.994846 seconds
> Threads:1000 Average time, 7.310746 seconds
>
> You can see that between 100 and 500 threads, the time required goes up by
> 30 when only 5 times the number of requests are made.

This is quite surprising results. I've not seen any clear measurements on this
in a long time, but when we've used the multi interface and done operations
such as this using a single thread we've managed far better results than that.

I don't even know what libcurl _could_ do to cause such delays!

> I ran "strace" over the program and found that it pauses for long periods
> (many seconds) on a calls to "futex" with FUTEX_WAIT. Maybe the library is
> experiencing some kind of deadlock?

The library is written single-threaded and uses no mutexes nor locks (unless
you've introduced them) so I don't think it does.

Those futex things are rather used by system functions that perhaps fight for
a shared resource or something?

-- 
  / daniel.haxx.se