cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: 10-at-a-time.c libcurl example - limitations?

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Tue, 13 Dec 2011 09:50:17 +0100 (CET)

On Thu, 8 Dec 2011, Matthew Ford wrote:

> Actually, I think I spoke to soon. I'm now getting results, but the
> performance is much slower once we reach 1024 hostnames. I have ulimit -n =
> 65535, and if I grep FDSize /proc/<PID>/status I get 2048 returned after we
> reach 1024 hostnames queried. Are there other limits I might be running in
> to?

If you're still using select(), you need to do compile-time tweaks so that
fd_set() and friends still work and don't access data out of boundary.

Also, at that amount of descriptors I suspect that you may start seeing some
speed penalty from using select().

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2011-12-13