cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: ftp serial transfers slow ... parallel fast

From: <farr_at_metservice.com>
Date: Thu, 11 Mar 2004 18:47:11 +1300

At 01:47 PM 3/11/2004, you wrote:
>farr_at_metservice.com wrote:
>>I am using the multi interface of libcurl 7.11.0 in a visual .net 2003
>>project.
>>When I try to transfer (ftp put or ftp get) large numbers of files from a
>>windows box it goes really slowly. 293 files with median size of 5k and
>>totalling 1Mbyte takes about 60 seconds through ftp. The same files take
>>about 5 seconds to transfer when I copy them to the same box with smb (so
>>there cant be any problem with the network connection ... right?). I can
>>also drastically reduce the time it takes to transfer, by transfering
>>many of them in parallel. Transferring the files with windows ftp client
>>also seems to go very slowly. A very similar application on linux seems
>>to go much faster but havn't actually tested to see how much. I am using
>>pure-ftpd and ncftpd ftp daemons, and I have tried stuffing around with
>>the configuration to get it to go faster, but have not had any luck.
>>The obvious question is, how do I make it go faster without transferring
>>everything in parallel? If not already implicit in the answer to the
>>previous question, why would it seem slow from windows (perhaps I am just
>>imagining things ...) ?
>
>What are the settings for MTU, MSS and window size on all the boxes
>concerned? If one of them has too low a setting for one or all of those
>that could slow things down.
>
>Running a sniffer like tcpdump while the slow transfer is going on would
>help clarify the situation.

Perhaps you can help ... the following 3 steps happen really fast (within
3ms or so) according to ethereal
STOR filename
150 Accepted data connection
two packets of data going to server and being acked straight away, the last
packet is the last packet of data for that file.
tcp control packet sent to server
tcp control packet recieved form server
tcp control packet sent to server

now we get a long pause of any where from 120ms to 200ms then a message
from the client box port (which seems to be the same for each file in the
transfer set) to ftp on the server ... it looks like this in ethereal
3044 > ftp [ACK] Seq=327186750 Ack=2387512526 Win=63626 Len=0 (60 bytes of
control data)
Response 226-32456.2 Mbytes of free disk space
etc with everything going really fast again

So it looks like there is a packet being sent to the server that is being
ignored ... and this is causing a delay between the end of one file and the
start of the next.

>Joe
>
Received on 2004-03-11