cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: curl-multi: Delays with FTP downloads due to blocking select() calls / libcurl architecture

From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Mon, 6 Dec 2004 00:07:57 +0100 (CET)

On Sun, 5 Dec 2004, Richard Atterer wrote:

> With FTP, there are a couple of blocking select() calls in ftp.c.

Yes. There's also one in Curl_SSLConnect() which affects HTTPS and FTPS.

> All in all, it would be nice if there were no select() calls at all in the
> code - everything should go via my own select() to avoid wait delays. Of
> course, turning all the code into a state machine is a lot of work. :-(

... which of course is the reason why it isn't done yet.

> I have to say such code tends to be very difficult to maintain. If there are
> only a few paths through the equivalent non-s.m. code, then the s.m. code is
> still OK (look at zlib for a nice example), but with implementations for
> protocols like FTP and HTTP, this is hopeless IMHO.

I'm not sure it is hopeless, but yes, it takes a lot of work and we risk
ending up with very hard-to-follow source code. It takes a good deal of
comments and good function names to make it as painless as possible.

> The alternative would be simply to multi-thread. AFAIK from a short look,
> this is what the Mozilla code does. Hmm, a fixed dependency on threads is
> not ideal for a library like libcurl.

I don't think libcurl itself needs to multi-thread. If it doesn't work good
enough for you, you simply need to use threads. I don't see a need for libcurl
to do it.

> Meanwhile, there's also a workaround: Allow the curl-multi user to specify
> his own function pointer and call that function instead of select(). But
> this would not solve my FTP performance problems. :-/

It would only be a half-baked work-around and not an actual solution to the
problem. I'm not sure it is worth adding.

-- 
      Daniel Stenberg -- http://curl.haxx.se -- http://daniel.haxx.se
       Dedicated custom curl help for hire: http://haxx.se/curl.html
Received on 2004-12-06