curl-library
Re: curl-multi: Delays with FTP downloads due to blocking select() calls / libcurl architecture
Date: Mon, 6 Dec 2004 00:07:57 +0100 (CET)
On Sun, 5 Dec 2004, Richard Atterer wrote:
> With FTP, there are a couple of blocking select() calls in ftp.c.
Yes. There's also one in Curl_SSLConnect() which affects HTTPS and FTPS.
> All in all, it would be nice if there were no select() calls at all in the
> code - everything should go via my own select() to avoid wait delays. Of
> course, turning all the code into a state machine is a lot of work. :-(
... which of course is the reason why it isn't done yet.
> I have to say such code tends to be very difficult to maintain. If there are
> only a few paths through the equivalent non-s.m. code, then the s.m. code is
> still OK (look at zlib for a nice example), but with implementations for
> protocols like FTP and HTTP, this is hopeless IMHO.
I'm not sure it is hopeless, but yes, it takes a lot of work and we risk
ending up with very hard-to-follow source code. It takes a good deal of
comments and good function names to make it as painless as possible.
> The alternative would be simply to multi-thread. AFAIK from a short look,
> this is what the Mozilla code does. Hmm, a fixed dependency on threads is
> not ideal for a library like libcurl.
I don't think libcurl itself needs to multi-thread. If it doesn't work good
enough for you, you simply need to use threads. I don't see a need for libcurl
to do it.
> Meanwhile, there's also a workaround: Allow the curl-multi user to specify
> his own function pointer and call that function instead of select(). But
> this would not solve my FTP performance problems. :-/
It would only be a half-baked work-around and not an actual solution to the
problem. I'm not sure it is worth adding.
-- Daniel Stenberg -- http://curl.haxx.se -- http://daniel.haxx.se Dedicated custom curl help for hire: http://haxx.se/curl.htmlReceived on 2004-12-06