curl-library
Polling on stray socket on sequential transfers.
Date: Tue, 12 Jul 2011 16:09:24 +0400
We are using curl C library in our projects. The problem arises when
making two sequential HTTP GET transfers to any URLs using one multi
handle (but different easy handles bound to it). How transfers are
performed: we are initializing new easy handle (curl_easy_init),
binding it to multi handle (curl_multi_add_handle), collecting
interesting file descriptors calling curl_multi_fdset function, then
polling on these descriptors with timeout previously reported by
callback (CURLMOPT_TIMERFUNCTION) and do curl_multi_socket_action in
appropriate event handler. The later steps are repeated until the
transfer is done (as we know from curl_multi_info_read). Then we
normally remove easy handle from multi handle
(curl_multi_remove_handle) and call curl_easy_cleanup.
First transfer is done normally, and during the second transfer
curl_multi_fdset continues reporting the file descriptor to be
available for writing, so poll returns immediately, our callback is
invoked, but curl_multi_socket_action called with the signaled socket
seems to do nothing. And these 'short-circuit' polls continue until
timeout elapses and then the transfer is finally done.
The problem is in multi.c file, line 2095, which is executed each time
in these polls during second transfers:
if(!entry)
/* Unmatched socket, we can't act on it but we ignore this fact. In
real-world tests it has been proved that libevent can in fact give
the application actions even though the socket was just previously
asked to get removed, so thus we better survive stray socket actions
and just move on. */
The comment explains a lot, but a few questions remain:
- why curl_multi_fdset call continues reporting this socket if it
already has been asked to be removed (through CURLMOPT_SOCKETFUNCTION
with CURL_POLL_REMOVE)?
- where is the actual socket for the second transfer, and why it is
not reported neither by socketfunction callback nor by
curl_multi_fdset call?
I have attached a small example application which reproduces the
problem. It uses libevent for polling (and needs it in order to
build).
To build an example:
$ cd curlfall
$ mkdir build
$ cd build
$ cmake -D CURL_PREFIX=/path/where/curl/is/installed ..
$ make
$ ./curlfall google.com | tee log.txt
Example log (as it is reproduced on my machine) is included also.
This 'stray' socket behavior is observed even during first transfer
when unresolvable url is given, ex:
$ ./curlfall aa.zzz | tee log.txt
Here we see that the socket is reported and is ready for reading even
while resolving domain name in first transfer.
Additional Information:
curl version 7.21.4
(built with cares and with --enable-debug)
cares version 1.7.4
os: Linux dev 2.6.18-238.5.1.el5 #1 SMP Fri Apr 1 18:41:58 EDT 2011
x86_64 x86_64 x86_64 GNU/Linux
glibc: 2.5
Best regards, Andrew
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
- application/x-gzip attachment: curlfall.tar.gz