cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: curl-library Digest, Vol 53, Issue 26

From: Nick Gerner <nick_at_seomoz.org>
Date: Fri, 15 Jan 2010 14:03:27 -0800

>
> So that might then be because the hash didn't get pruned porperly. It'll be
> interesting to see if you see better results with current CVS.
>
> Are you really doing it with a loop like that?
>
>
Here's the number of passes through my loop in 15 second intervals on curl
7.19.3:
http://www.nickgerner.com/loops_curl.7.19.3.png

Here's the same test but on 7.20.0-20100114
http://www.nickgerner.com/loops_curl.7.20.0-20100114.png

oprofile shows that those hostcache and hash pick calls have dropped well
below other calls when using the latest cvs build.

I've been terrified of migrating off of 7.19. But 7.20 looks great, no
stability problems, and our results look more or less the same in aggregate
as when we ran on 7.19, except perf is about 17% higher. So nice work; I
can't wait for the production release. If I enable the dns cache and get
good results I'll post back.

> It's a bit unusual, since the multi_socket supports and really encourages
> an
> app to rather use something event-based. You will also get significantly
> better performance by using an event-based layer like libev or libevent and
> let that call libcurl for each socket event rather than have poll() poll
> them
> all all the time. At least if the amount of sockets is fairly high, like a
> hundred or more.
>
>
Yes, we should move to eventing. That's a bit of a change for us, but our
newest results show most of our time is spent in our event loop, so that
looks like the next low-hanging fruit.

--Nick

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-01-15