cURL / Mailing Lists / curl-library / Single Mail

curl-library

ftp serial transfers slow ... parallel fast

From: <farr_at_metservice.com>
Date: Thu, 11 Mar 2004 11:48:55 +1300

I am using the multi interface of libcurl 7.11.0 in a visual .net 2003 project.

When I try to transfer (ftp put or ftp get) large numbers of files from a
windows box it goes really slowly. 293 files with median size of 5k and
totalling 1Mbyte takes about 60 seconds through ftp. The same files take
about 5 seconds to transfer when I copy them to the same box with smb (so
there cant be any problem with the network connection ... right?). I can
also drastically reduce the time it takes to transfer, by transfering many
of them in parallel. Transferring the files with windows ftp client also
seems to go very slowly. A very similar application on linux seems to go
much faster but havn't actually tested to see how much. I am using
pure-ftpd and ncftpd ftp daemons, and I have tried stuffing around with the
configuration to get it to go faster, but have not had any luck.

The obvious question is, how do I make it go faster without transferring
everything in parallel? If not already implicit in the answer to the
previous question, why would it seem slow from windows (perhaps I am just
imagining things ...) ?

--
By the way, if I dont add and remove the easy_curl handle from the multi 
group between file transfers it seems to crash ... how bizarre.  I am doing 
something else wrong, or is that the way it is supposed to work?
curl_multi_remove_handle(curlMultiHandle, curlEasyHandle);
curl_multi_add_handle(curlMultiHandle, curlEasyHandle);
thanks
Received on 2004-03-10