On Tuesday, August 12, 2014 10:05:37 Alastair Scobie wrote:
> Whilst porting an in-house application, which uses libcurl, from RHEL6
> to RHEL7 we noticed that the time taken by libcurl to fetch a page had
> substantially increased. Even an attempt at fetching a non-existent page
> has increased from 10-20ms to around 150ms. This with just plain HTTP.

What exactly do you mean by a non-existent page?

> Our first thought was that the API has changed in some subtle way and
> the fault must be in our code, but then we thought to try measuring the
> performance of the curl tool itself. To our surprise, we found the same
> performance disparity.
>
> Interestingly...
>
> * Adding the DNS address of the HTTP server to /etc/hosts (and
> specifying files,dns in /etc/nsswitch.conf) does not improve things

Did you try to measure the time consumed by 'getent ahosts'?

> * but, performance when using an IP address in the URL is fine (~
> 10-20ms) for both RHEL6 and RHEL7

I was not able to reproduce this behavior. Is it reproducible locally?

Kamil

> ... which kind of suggests that the delay is in libcurl's DNS code
> rather than in the upstream resolver.
>
> We did notice that when using DNS names in the URL, curl forks off a
> process to do the name lookup and then poll()s. There's no such fork
> when using an IP address in the URL. The poll() timesout after a period
> of time - we suspect this is the cause of the delay.
>
> Any help would be very gratefully received!
>
> (FYI - RHEL6 curl is 7.19, RHEL7 curl is 7.29)
>
> Thanks, Alastair Scobie
>
> School of Informatics
> University of Edinburgh
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-08-12