curl-library
Re: ftp serial transfers slow ... parallel fast
Date: Thu, 11 Mar 2004 13:23:06 -0600
farr_at_metservice.com wrote:
> At 01:47 PM 3/11/2004, you wrote:
>
>> farr_at_metservice.com wrote:
>>
>>> I am using the multi interface of libcurl 7.11.0 in a visual .net
>>> 2003 project.
>>> When I try to transfer (ftp put or ftp get) large numbers of files
>>> from a windows box it goes really slowly. 293 files with median size
>>> of 5k and totalling 1Mbyte takes about 60 seconds through ftp. The
>>> same files take about 5 seconds to transfer when I copy them to the
>>> same box with smb (so there cant be any problem with the network
>>> connection ... right?). I can also drastically reduce the time it
>>> takes to transfer, by transfering many of them in parallel.
>>> Transferring the files with windows ftp client also seems to go very
>>> slowly. A very similar application on linux seems to go much faster
>>> but havn't actually tested to see how much. I am using pure-ftpd and
>>> ncftpd ftp daemons, and I have tried stuffing around with the
>>> configuration to get it to go faster, but have not had any luck.
>>> The obvious question is, how do I make it go faster without
>>> transferring everything in parallel? If not already implicit in the
>>> answer to the previous question, why would it seem slow from windows
>>> (perhaps I am just imagining things ...) ?
>>
>> What are the settings for MTU, MSS and window size on all the boxes
>> concerned? If one of them has too low a setting for one or all of
>> those that could slow things down.
>>
>> Running a sniffer like tcpdump while the slow transfer is going on
>> would help clarify the situation.
>
> Perhaps you can help ... the following 3 steps happen really fast
> (within 3ms or so) according to ethereal
> STOR filename
> 150 Accepted data connection
> two packets of data going to server and being acked straight away, the
> last packet is the last packet of data for that file.
> tcp control packet sent to server
> tcp control packet recieved form server
> tcp control packet sent to server
>
> now we get a long pause of any where from 120ms to 200ms then a message
> from the client box port (which seems to be the same for each file in
> the transfer set) to ftp on the server ... it looks like this in ethereal
> 3044 > ftp [ACK] Seq=327186750 Ack=2387512526 Win=63626 Len=0 (60 bytes
> of control data)
> Response 226-32456.2 Mbytes of free disk space
> etc with everything going really fast again
>
> So it looks like there is a packet being sent to the server that is
> being ignored ... and this is causing a delay between the end of one
> file and the start of the next.
Can you post the actual ethereal output?
Thanks
Joe
Received on 2004-03-11