On Sun, 26 Apr 2015, Daniel Speyer wrote:

Thanks a lot for your contribution!

> I was using curl in the test cases of the new profiling tool I was
> developing and I stumbled on a small bug. When curl spins off a separate
> thread for dns lookups, it waits using poll. However, the child thread
> doesn't write to any file descriptor, so the parent waits for a timeout.
> This costs several milliseconds (either when nscd has the ip in cache, but
> also if the lookup time modulo the timeout is small).

The "punishment" can actually be much larger than that, depending on the
application. It depends on what apps do when they don't have any descriptor to
wait on.

> I've written a fix (attached) that has the child thread write to a pipe that
> gets included in the poll command. I realize pipes aren't the most popular
> form of interthread communication, but they're simple and they play nicely
> with poll.

... and therein lies one of the problems with this approach. Even if I do like
the general idea. Let me elaborate:

1.

pipe() is not that portable. Most specifically you cannot do pipes like that
on windows and wait for them with poll() since poll() and select() in windows
land only deal with sockets.

So, a solution would either use socketpair() instead (which then would exclude
windows XP I believe) or it would need the appropriate ifdef protection for
non-pipe platforms.

2.

curl_multi_wait() is the wrong place to address the waiting, as it would then
only fix that particular function and not the multi interface and all users of
that!

Instead, the proper fix would be in the multi_getsock() function that needs to
be extended to export the correct file handle and wait-bitmask for when
libcurl waits for a name resolve. That would then also lead to
curl_multi_wait() doing the right thing, but more important it would also have
the multi interface users get the info and thus be able to wait on the right
things.

> I tested it on some simple downloads and it gets the expected performance
> improvement. I haven't tested it on anything really complicated (I'm not
> really sure what a complicated usage of curl would be).

A multi-interface app that doesn't use curl_multi_wait() but extracts the info
with curl_multi_fdset() or an event based one that use
curl_multi_socket_action() would be suitable to test too.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html