cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Crashes in curl_multi_remove_handle after timeout

From: Martin Lemke <terradepaz_at_gmail.com>
Date: Mon, 26 Sep 2011 09:34:54 +0300

I think I found out a clear case for reproducing this problem (I am
working with 7.21.3 but the same crash occurs in 7.22.0 as well).
This is a combination of few factor with the "pipelining" in multi
being the most important one. Without the pipelining the crash doesn't
happen. With pipelining it happens.

The other factors are number of easy handles (I have a list of URLs
from a certain site), the connection timeout and removing the handles
from multi prior to calling multi_cleanup. In a pseudo code (actual
test code is examples/multi-app.c):

// create easy handles with a connection timeout
for (number of easy handles)
curl_easy_setopt( easy, CURLOPT_CONNECTTIMEOUT_MS, 500);

curl_multi_setopt( multi_handle, CURLMOPT_PIPELINING, 1);
// do the multi_perform thing - and time out on a connection for several handles

for (number of easy handles)
curl_multi_remove_handle( multi_handle, handles[ i ]); // Crash on
a timed out handle

Without the pipelining, in a tcpdump I see a burst of requests when
curl_multi_perform starts. A connection has a request and a response.
All requests got a response, no timeouts are reported.

With the pipelining I see a sequence of requests and responses: a
request, then a response (a redirection in my test case), then another
request, response. A connection is used for more than one
request-response. It also appears as if the requests are made not in a
burst but sequentially. As a result, many of requests simply timeout
the connection state. When removing those timed out handles a crash
occurs.

In fact I expected that pipelining would bring an increase of transfer
speed, the result is just opposite, apparently either I misunderstood
the feature, or it is not implemented properly. Anyway, due to this
consideration I disabled it anyway.

There is one more side problem that I encountered when working on this
- when building libcurl without an optimization (-O0 instead of -O2,
that is the only difference) I am getting crashes in
curl_multi_perform due to stack corruption after a call to
multi_runsingle. It is clearly seen that the stack variable "easy" is
overwritten after returning from multi_runsingle. I didn't investigate
it deeply, but the setup was the same - multi with pipelining enabled.
Just as something for a record.

Here is a list of urls that I fed to the multi-app.c code for
reproducing a problem (note that I did requests from a remote corner
of the internet with a slow connection times, you may have to play
with the list of files and connection timeout option) - these are
links from some alexa 1000 site, what is interesting is that almost
each of them requests a redirection, which may also be a factor. I
hope the mail filters wouldn't complain:

Thanks,
Martin
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-09-26

This message: [ Message body ]
Next message: Martin Lemke: "Re: Crashes in curl_multi_remove_handle after timeout"
Previous message: Eduardo Giacoman: "Re: SMTP stall issue"
In reply to: Daniel Stenberg: "Re: Crashes in curl_multi_remove_handle after timeout"
Next in thread: Martin Lemke: "Re: Crashes in curl_multi_remove_handle after timeout"
Reply: Martin Lemke: "Re: Crashes in curl_multi_remove_handle after timeout"
Reply: Daniel Stenberg: "Re: Crashes in curl_multi_remove_handle after timeout"

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]