cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: --max-filesize and streams

From: Dave <DaveM_at_mich.com>
Date: Fri, 09 Sep 2005 17:18:44 -0400

I think the existing code is pretty close to working the way I need it to.
There seems to be two problems. Sometimes it just exits without doing any
retry's, even though I specify "--retry 32" in my command line.

Sometimes it does a retry and continues appending data to the file but when
it resets the elapsed time to the beginning. So if I specify "--max-time
100" and it runs for 99 seconds and then does a retry I get 199 seconds
worth of data in my file. When it does multiple retry's the file just keeps
getting bigger and bigger.

So maybe these are bugs? It seems like it should keep doing retry's until
it has tried 32 times. It also seems like it should keep track of the total
time, even after doing a number of retry's.

Do you think it would be possible to add a "--Stream <bytes>" option to
cURL? Its purpose would be to keep appending data to the file. It would
automatically retry or reconnect whenever there is a problem. It would only
quit after the file size had reached the specified number of bytes or if
the retry parameters have been exceeded.

This option would also be useful in cases where the server allows the
client to pre-buffer data (the server sends the stream faster then
real-time). In this case the --max-time option doesn't work as well.

The script idea is interesting but I am not sure how I can get it to work.

I can use a script get it to run again but I need to have it append the new
stream data to the file that it created when it ran (and failed) the first
time. Is there an option to append to an existing file? I tried "-C -" and
it doesnt work.

I also need to have it quit when the file reaches the correct size (or the
proper amount of time has expired). Any suggestions as to how I can get
that to work? I cant figure out how to do it with a Win-ME command line script.

At 04:24 PM 9/8/2005 , Daniel Stenberg wrote:
>On Thu, 8 Sep 2005, Dave wrote:
>
>>I tried using the --max-time option and that works for me. The manual
>>says it doesn't work with Windows but I am using it on a Win-ME system
>>and it seems to be working fine.
>
>It says "doesn't work fully" - which used to mean that parts of the
>operation wasn't possible to time-out. These days that comment is not
>correct though, and I've removed it now.
>
>>I am noticing that, if the stream is interrupted, it does not resume. Is
>>there a way to get it to restart/continue after the stream has been
>>interrupted?
>
>curl currently has no such option. It should be fairly easy to check the
>return code with a script and just invoke it again if deemed that the
>transfer should be continued.
>
>--
> Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2005-09-10