curl-library
Re: ftp serial transfers slow ... parallel fast
Date: Fri, 12 Mar 2004 09:29:18 +1300
At 08:23 AM 3/12/2004, you wrote:
>farr_at_metservice.com wrote:
>>At 01:47 PM 3/11/2004, you wrote:
>>
>>>farr_at_metservice.com wrote:
>>>
>>>>I am using the multi interface of libcurl 7.11.0 in a visual .net 2003
>>>>project.
>>>>When I try to transfer (ftp put or ftp get) large numbers of files from
>>>>a windows box it goes really slowly. 293 files with median size of 5k
>>>>and totalling 1Mbyte takes about 60 seconds through ftp. The same
>>>>files take about 5 seconds to transfer when I copy them to the same box
>>>>with smb (so there cant be any problem with the network connection ...
>>>>right?). I can also drastically reduce the time it takes to transfer,
>>>>by transfering many of them in parallel.
>>>>Transferring the files with windows ftp client also seems to go very
>>>>slowly. A very similar application on linux seems to go much faster
>>>>but havn't actually tested to see how much. I am using pure-ftpd and
>>>>ncftpd ftp daemons, and I have tried stuffing around with the
>>>>configuration to get it to go faster, but have not had any luck.
>>>>The obvious question is, how do I make it go faster without
>>>>transferring everything in parallel? If not already implicit in the
>>>>answer to the previous question, why would it seem slow from windows
>>>>(perhaps I am just imagining things ...) ?
>>>
>>>What are the settings for MTU, MSS and window size on all the boxes
>>>concerned? If one of them has too low a setting for one or all of those
>>>that could slow things down.
>>>
>>>Running a sniffer like tcpdump while the slow transfer is going on would
>>>help clarify the situation.
>>Perhaps you can help ... the following 3 steps happen really fast (within
>>3ms or so) according to ethereal
>>STOR filename
>>150 Accepted data connection
>>two packets of data going to server and being acked straight away, the
>>last packet is the last packet of data for that file.
>>tcp control packet sent to server
>>tcp control packet recieved form server
>>tcp control packet sent to server
>>now we get a long pause of any where from 120ms to 200ms then a message
>>from the client box port (which seems to be the same for each file in the
>>transfer set) to ftp on the server ... it looks like this in ethereal
>>3044 > ftp [ACK] Seq=327186750 Ack=2387512526 Win=63626 Len=0 (60 bytes
>>of control data)
>>Response 226-32456.2 Mbytes of free disk space
>>etc with everything going really fast again
>>So it looks like there is a packet being sent to the server that is being
>>ignored ... and this is causing a delay between the end of one file and
>>the start of the next.
>
>Can you post the actual ethereal output?
It is attached as a libpcap file. If that is not enough I will send it
directly to you ...
Mike
>Thanks
>
>Joe
>
>
- application/octet-stream attachment: tcpdump.pcap