cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Q. about optimizing use of Curl Handles

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Fri, 10 Feb 2012 22:42:10 +0100

On Fri, Feb 10, 2012 at 01:14:56PM -0800, Roger Tremblay wrote:
> I was told that establishing a network connection is costly. So how can we use LibCurl to minimize this cost?
>
> I guess when it comes to connection cost, the URL (used to configure the Curl handle) is the only parameter to consider here.

If you discount things like proxies and options that explicitly specify
to reconnect every time, then pretty well you're right here.

> So how does the connection creation/destruction scheme works with libCurl?
>
> Say a handle is created:
> CURL* handle = curl_easy_init();
> obviously the connection is not established yet because no URL has been specified yet
>
> Next the Url is specified:
> curl_easy_setopt( handle, CURLOPT_URL, "<protocol>://<IP>[:<port>/<path</file>>" );
>
> Is the connection established a this moment? Or is it created when executing
> curl_easy_perform( handle );
> ?

There are lots of other options that can affect the connection, so it
really can't be made until they are all specified which is signaled by
the call to curl_easy_perform.

> And is the connection released automatically right after the call to curl_easy_perform?

That depends on the various curl_easy_setopt options that controls this.
libcurl tries hard to avoid tearing down the connection when it can.

> Say another call
> curl_easy_setopt( handle, CURLOPT_URL, "<protocol>://<IP>[:<port>/<path</file>>" );
> is executed. Is the 1st connection released and a 2nd one created?

Not until the CURLOPT_MAXCONNECTS limit is reached (assuming the
protocol involved is able to be kept open for further requests).

> It is strongly assumed that if 2nd IP is different, then yes the 1st connection will be closed and another one created. Right? What if only the port is different on the 2nd

> URL? What if the protocol is different?

Even if the protocol is different, the previous connections will be kept
around.

> In one word!
>
> Given a sequential list of urls with different IPs, ports and or protocols..., which urls can be "performed" on the same Curl handle without creating new connections (reusing
>
> the old one)? What criteria to use to dispatch the urls to the available handles? Does the server's IP matter? If it does, can all urls with same IP can be peformed optimally
>
> on the same handle, no matter what port they use?

I'm not positive, but I belive the match is done on the IP address, not
the URL. So, a multi-homed server may have several connections depending
on the DNS timeouts.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-02-10