On 2/20/06, Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Mon, 20 Feb 2006, Nimrod A. Abing wrote:
>
> > Now this setup seems to work fine in most cases. I am able to do several
> > uploads at once as long as *none* of those transfers take a lot of time to
> > finish. The problem is that if an upload is waiting for a response from the
> > server, other uploads/downloads seem to be blocked. I don't know much about
> > curl internals but I thought curl_multi_perform() is *not* supposed to block
> > even though one of those transfers is waiting for a response from the
> > server.
>
> Yes, that is the purpose of the multi interface. It isn't 100% non-blocking
> though, and it also varies depending on what sub-library support your libcurl
> is built to use.
>
> The blocking parts are of course meant to get fixed, and they are in fact at
> least better these days than they used to be.

I see, what would cause curl_multi_perform to block? Any list of
things to avoid?

> > I am able to reproduce the problem consistently. First I upload a file and
> > wait for it to be uploaded, it will cause the server to take some time to
> > respond. Then I try to do a second upload or even just a "server ping"
> > (which is just an HTTP request to a special URL on the server). The app will
> > then hang up seemingly waiting for the first request to finish with a reply
> > from the server.
> >
> > I have tried even using threads with this, running the queue in a separate
> > thread. Initially I tried multithreaded easy handles with curl_easy_perform.
> > Same results, which is why I swiched to the multiplexing interface. I even
> > have the following setopt calls for each new handle I create:
>
> To be honest, since you've tried multiple approaches and still get the same
> sympthoms and since I know that libcurl doesn't have any blocking in this kind
> of approach, I would blame the server end. The server is clearly not properly
> responding or reacting when you communicate with it while it is in progress
> taking care of the previous upload.

I seem to have found the culprit that's causing the problem. It seems
that it has more to do with the GUI library I am using. Since right
now I am using the multi-threaded approach running the queue processer
in a different thread. I was able to pinpoint that the processor
thread would stop suddenly when a new curl handle was added to it. So
I think you're right that curl is not causing the app to hang in this
case.

Additional info on my server setup... The server (Apache 2.0.x) takes
time to respond because it has to run a series of filters on the
uploaded file. Like I stated, it must first validate the data, then
compress it using zlib, then encrypt it using Blowfish. This is all
done in one go after the file has been uploaded to the server so while
the server is doing these things to the file, it will not be able to
respond to the upload immediately. For instance, in the worst case, it
will take the server between 5-7 minutes to respond. But I think it
should not be an issue whether the server takes a long time to
respond. HTTP(S) is connection-oriented so the client can wait on the
connection indefinitely.

Instead of waiting on the connection, what I do now is to create
"one-shot" curl handles that would abort the connection once the file
was uploaded. That is, the client does not wait for the server's
response for the upload. I don't know if this is really a good idea,
but it seems to work, all the uploads are there.

--
_nimrod_a_abing_