cURL / Mailing Lists / curl-library / Single Mail


Performance fix for dns lookup timeouts

From: Daniel Speyer <>
Date: Sun, 26 Apr 2015 00:11:47 -0400

Hello curl folks,

I was using curl in the test cases of the new profiling tool I was
developing and I stumbled on a small bug. When curl spins off a
separate thread for dns lookups, it waits using poll. However, the
child thread doesn't write to any file descriptor, so the parent waits
for a timeout. This costs several milliseconds (either when nscd has
the ip in cache, but also if the lookup time modulo the timeout is
small). If the actual remote host is over a local network, this
timeout could be a large fraction of the total run time, and if the
curl command was inside a loop in a shell script, it could add up.

I've written a fix (attached) that has the child thread write to a
pipe that gets included in the poll command. I realize pipes aren't
the most popular form of interthread communication, but they're simple
and they play nicely with poll.

I tested it on some simple downloads and it gets the expected
performance improvement. I haven't tested it on anything really
complicated (I'm not really sure what a complicated usage of curl
would be).

Hope you like it,
--Daniel Speyer

List admin:

Received on 2015-04-26