curl-library
Re: ftp serial transfers slow ... parallel fast
Date: Thu, 11 Mar 2004 12:25:33 +1300
At 11:58 AM 3/11/2004, you wrote:
> > The obvious question is, how do I make it go faster without transferring
> > everything in parallel? If not already implicit in the answer to the
> > previous question, why would it seem slow from windows (perhaps I am just
> > imagining things ...) ?
>The first thing, ftp is an inherently slow protocol. That's why you'll
>notice curl has special support to add additional timeouts for FTP because
>FTP is notoriously slow. The other thing, when you are comparing Linux and
>Windows, is this the same machine (i.e. dual-boot)? I mean, if you're
>comparing your Windows machine on a 56K and say a Linux shell account on a
>T1, well then obviously the Linux download is going to be faster :)
They are on the same subnet plugged into the same switch. The windows box
is a P4 3.2G and the linux box is a P3 1G, they have the same intel network
card. It is a 100mbit switched network. FTP might be slow ... but that is
just rediculous.
>Also I noticed you mentioned smb. I know someone said they were working on a
>patch for libcurl to support smb, maybe that would be a better alternative
>for you? But, I don't know if that patch was ever completed.
cool ... didn't know that. That would be really useful ... but some lamer
put the requirement for ftp in this project.
> > By the way, if I dont add and remove the easy_curl handle from the multi
> > group between file transfers it seems to crash ... how bizarre. I am
>doing
> > something else wrong, or is that the way it is supposed to work?
>I'm not totally sure what you mean, you might need to be a bit more specific
>about your code.
I have a C++ CurlSocket (which represents an curl_easy_handle). I also
have a SocketSet (which represents a set of CurlSockets, TCPSockets (for
rpc and other nasty things), UDPSockets etc ...). Another object will be
asked to do a set of downloads from a particular host, it will try to use 1
CurlSocket to do all transfers from a particular host. When a particular
download has finished it will ask the CurlSocket to do a new download like
this:
void CurlSocket::NextTransfer()
{
CURLM *curlMultiHandle = getCurlMultiHandle();
curl_multi_remove_handle(curlMultiHandle, GetCurlHandle());
curl_multi_add_handle(curlMultiHandle, GetCurlHandle());
curl_easy_setopt(_curlHandle, CURLOPT_URL, _remoteURLFile.c_str());
if (_isSending)
{
curl_easy_setopt(GetCurlHandle(), CURLOPT_INFILESIZE,
fileSize(localURL.c_str())) ;
_localStream = fopen(localURL.c_str(), "rb");
}
else
{
_localStream = fopen(localURL.c_str(), "wb");
}
}
A little while later I call processCurlMessages, and curl_multi_perform and
my write/read callbacks proceed as normal. I do not understand why I
should need the curl_multi_remove_handle and curl_multi_add_handle calls.
Received on 2004-03-11