cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: DNS issue in libcurl 7.19.3 on Mac OS 10.5

From: Heston Liebowitz <gansos_at_mac.com>
Date: Fri, 17 Jul 2009 14:12:39 -0700

On Jul 17, 2009, at 12:04 PM, Heston Liebowitz wrote:

> I'm running into a strange issue using libcurl with ruby bindings.
> I've tested with both patron(i) and curb(ii) to the same effect,
> leading me to believe this is an issue with libcurl and not the ruby
> gems.
>
> I have an array of URLs which I want to loop over and GET in turn.
> About halfway through the array, one of the URLs results in a
> malformed packet being sent back from the server (according to
> Wireshark). After that point, all subsequent GETs have their DNS
> lookups perturbed in some way. For example, here is a subset of the
> URLs with the proper DNS resolutions:
>
> 205.178.145.65 << http://povertyenvironment.net/misc/content/welcome/opera-music.html
> 195.167.170.96 << http://bioportfolio.com/bookshop/ASIN/B000IOEVAS
> 206.16.217.114 << http://vitaminshoppe.com/search/en/query.jsp?
> q="Natural+Dentist"
>
> The first URL in the list returns a malformed response ("TCP
> Previous Segment Lost" according to Wireshark). After this, GETing
> the following two URLs raises "HostResolutionError" exceptions. If I
> omit the first URL though, the next two resolve and fetch correctly.
>
> I'm using curl 7.19.3 on Mac OS 10.5.7 and ruby 1.8.6. I haven't
> tested with the latest releases of cURL.
>
> Hopefully this is enough info to reproduce the problem. Thanks for
> any insight.
>
> Best,
> Heston
>
>
> i. http://fiatdev.com/2009/07/03/announcing-patron-a-ruby-http-client-library
> ii. http://curl.haxx.se/libcurl/ruby/
>
>

Update: as an experiment I tried setting CURLOPT_DNS_CACHE_TIMEOUT to
0 to disable the DNS cache. Lo and behold the problem disappeared. It
seems to me that the response for the URL "http://povertyenvironment.net/misc/content/welcome/opera-music.html
" is corrupting the DNS cache in some way. That's about as far as I
can take it though, not being all that familiar with cURL internals.

Best,
Heston
Received on 2009-07-17