cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Changes to connection timeout policy when multiple DNS records are present

From: Ryan Braud <rbraud_at_gmail.com>
Date: Tue, 11 Nov 2014 10:49:47 -0800

> After doing some testing with libcurl lately I have noticed curl's connect
>> behavior on timeouts has changed sometime between 7.22 and 7.39.
>>
>
> You're taking on three years of development there. More than 4200 commits.
>

Yes, and 7.22 is still the version available in Ubuntu 12, so it doesn't
seem unreasonable at all to compare that version.

 * Rebuilt URL to: www.google.com:45/
>> * Hostname was NOT found in DNS cache
>> * Trying 74.125.239.51...
>> * After 4998ms connect time, move on!
>>
>
> Now curl seems to allocate half its time to the first connection, half of
>> that time to the second, and so on.
>>
>
> Yes, that's the new algorithm. It is meant to favour the first addresses
> more and not split the total time into just a fraction.
>

Was this change documented anywhere? Was the connect behavior ever
documented? I never even knew it would try more than one IP address until
I started looking into this.

> CURLINFO_PRIMARY_IP now returns the empty string after these fetches.
>>
>
> Yes, but is that really wrong? What do you think the primary IP is on a
> failed connect attempt to N different IP addresses? I see that it used to
> report the last tried IP in the past, but that's not exactly how it is
> documented.
>
> I can see how we can fix this back to the former data, but then we should
> also update the docs accordingly.
>

It seems wrong to me. The documentation already says:

"Pass a pointer to a char pointer to receive the pointer to a
zero-terminated string holding the IP address of the most recent connection
done with this curl handle."

I guess it all hinges on your definition of "done", but a connection
attempt was made. In addition, if there is only one IP address returned
from the hostname resolution, it still does not appear in
CURLINFO_PRIMARY_IP. It seems fairly useless to have an error like
"Connection timed out after 1000 milliseconds" but not be able to tell to
which IP the connection was attempted. In any case, this is just another
case of undocumented behavior change in the library.

 7.39:
>>
>
> * After 196ms connect time, move on!
>> * connect to 74.125.239.51 port 45 failed: Connection timed out
>> * Trying 74.125.239.50...
>> * Connection timed out after 10000 milliseconds
>> * Closing connection 0
>>
>> The strategy here seems mostly the same as in 7.36, except the values
>> don't
>> make as much sense. If you add up the times it spent on each individual
>> connection, you end up well short of 10000 ms, even though the wallclock
>> time of the program is very close to 10 seconds. CURLINFO_PRIMARY_IP is
>> also missing here.
>>
>
> The times allowed seem to be roughly the same ones as used in 7.36. It
> splits the maximum time for each IP tried. So 5 seconds for the first, 2.5
> for the next and so on which gives the fifth IP a mere 312 milliseconds
> (adjusted somewhat since time is wasted every here and there so the last
> one actually only got 196 ms).
>

I find it a bit funny that given two scenarios where one looks right
(7.36), and another scenario where it looks wrong (7.39), you are fine with
saying "eh, it's close enough." I have wanted to keep our production
environment at work fairly up-to-date with new libcurl versions, but I am
having a harder and harder time convincing myself that random things that
we depend on won't change between versions, unless we start adding a bunch
of tests ourselves specifically to test libcurl.

Ryan

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-11-11