On Thu, 23 Jul 2009, johansen_at_sun.com wrote:

(Trying to get back up to speed...)

> To paraphrase the problem: I have 10 easy handles all sharing a connection
> because they're pipelining.

Hah, previously I wasn't even aware of that you were using pipelining. I
assume you can avoid this problem completely by not doing pipelining?

> One handle, let's call it A, transitions from CURLM_STATE_PERFORM to
> CURLM_STATE_DONE in multi_runsingle(). As part of the transition, the code
> in CURLM_STATE_PERFORM removes the session handle for A from the recv
> pipeline. The function returns and we go on to process another handle. Some
> time later, handle F encounters a send error that requires it to set
> bits.closed to True. It then calls Curl_done() on the connection.
> Curl_done() calls Curl_disconnect(), which calls signalPipeClose() on the
> send, recv, and pend pipelines. Part of the PipeClose is a call to
> Curl_multi_handlePipeBreak(), which sets the easy_conn in Curl_one_easy's
> attached to the pipeline to NULL. The connection is then free'd.
>
> When handle A gets run in the CURLM_STATE_DONE state, it's easy_conn has
> been free'd by this send error, but because its SessionHandle was removed
> from the pipeline, handlePipeBreak() didn't set handle A's easy_conn to
> NULL. This is what causes us to access free'd memory and crash.

Shouldn't A perhaps immediately forget what connection it was part of when
it reaches CURLM_STATE_DONE and another SessionHandle "takes over" the
connection?

> In terms of solving the problem itself, what do you think about introducing
> a 4th list called the done_pipeline?

I'm a bit reluctant to add another list since I think we should be able to
make the existing logic work with the existing lists. I get the feeling adding
another list is just a way to escape that fixing and instead try to engineer
another way to solve the same problem.

But I'll also admit that the pipelning suffers from far too few test cases and
far too little real world usage still, and I've not yet even used it myself
for real so I'm not fluent in all the code details.

> This is where we can put requests that are done, but that haven't been
> processed in the CURLM_STATE_DONE state yet.

Uh, how would it end in such a limbo? I mean when the request is done, doesn't
it switch to CURLM_STATE_DONE at once?

-- 
  / daniel.haxx.se