cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multi interface efficiency woes

From: Michael Fischer <michael_at_dynamine.net>
Date: Sat, 19 Feb 2011 15:02:07 -0800

On Sat, Feb 19, 2011 at 2:15 PM, Ben Noordhuis <info_at_bnoordhuis.nl> wrote:
> On Sat, Feb 19, 2011 at 20:24, Michael Fischer <michael_at_dynamine.net> wrote:
>> And until the response is returned, the underlying descriptor will
>> *always* be writable after it has connected.  But since we have no
>> knowledge of what's going on with the underlying HTTP transaction, we
>> ignorantly keep calling curl_multi_perform() anyway.  This is very
>> expensive, but we can't stop doing it until we get our response.  The
>> CPU expense is proportional to the response latency.
>>
>> What can be done to make this more efficient?
>
> You should register CURLMOPT_TIMERFUNCTION and CURLMOPT_SOCKETFUNCTION
> callbacks. See [1] for an example.

This seems a bit baroque. Also, why is this more efficient? Doesn't
it merely move what I'm already doing into libcurl itself? After all,
my code is already using libev to associate a callback with underlying
socket readiness.

What do you think about the idea of changing libcurl's behavior so
that it removes the fd from the write_fd_set list returned by
curl_multi_fdset() after the HTTP request has been issued and moves it
only to read_fd_set? That way my callback will fire only when some
response data has arrived (or when my timer callback fires, whichever
comes first).

--Michael

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-02-20