On Fri, Nov 18, 2011 at 12:16 AM, Kopparapu, Sangeeta (GE Healthcare,
consultant) <Sangeeta.Kopparapu_at_ge.com> wrote:

> >>> That seems a bit weird to me. What resolver backend are you using
> for
> >>> example?
>
> > >In the case of libCURL multi interface, its epoll. In the case of
> > >libCURL easy interface with 10 threads, its poll method.
>
> >I meant which resolver backend your libcurl build is using.
>
> >libcurl can be built to use several different resolver backends, and on
> *nix the default resolver is using synchronous name resolving and in
> such environments a threaded application using libcurl will of course be
> faster than a multi interface using one
>
> Yes, it's using default name resolving scheme as I did not make any
> changes to libCURL (development configuration) settings meaning, I am
> using library as it is with default settings.
> I modified my multi interface sample code to launch 10 threads. Still
> easy interface threaded application is having better performance than
> multi interface threaded application.
> Now, if this is the case (threaded EASY interface application with
> default name resolving scheme is faster than multi interface
> application), is it good to use my existing easy interface threaded
> application instead of moving to multi interface? Or is there something
> that can be done to multi interface to make it faster. Basically, my
> goal is to improve data transfer rate.
>
> --
>
> / daniel.haxx.se
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html
>

 Hello, lemme share my experiences with libcurl as I've also needed maximum
performance while downloading multiple urls a while ago.

First, regarding your question, as Daniel said, if you can't build curl to
use asynchronous resolver, running a threaded application will of course
result in better performance as it can run same amount of resolves in
parallel as your thread count.

However, if you're able to run with asynchronous resolver like c-ares,
please note that while c-ares does perform async resolve which should
greatly improve speed when using single thread and multi inferface, it is
(Daniel isn't gonna like this comment :)) a fairly slow library. In fact
it's so slow that it'd most likely take longer than 1.75 secs to download
1500 urls if you create an easy handle for each url and add it to multi
interface. For my application (whose requirements are to do about 400 http
reqs per sec 24/7) I'm using 8 threads (got 8 cpu cores), each one has its
own multi interface and I'm creating and adding easy handles to multi
interface on round robin basis becase when using libcurl/c-ares combo with
multi interface in only one thread app wasn't able to keep up.

My experiences are based on libcurl/7.21.4 and c-ares/1.7.4 and I haven't
been really following release notes of both libraries as current setup
satisfied my needs (don't try to fix things that work :)) so I'm sorry for
bad mouthing the libs if these things were fixed, but unless libcurl went
under considerable changes of how it uses c-ares (based on 5 second look of
Curl_open() it appears it didn't) there shouldn't be any performance
differences between 7.21.4 and 7.23.1).

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-11-18