Re: Corrupted download when using the resume option?! (Ray Satiro)

From: Mihail Kostira via curl-users <curl-users_at_cool.haxx.se>
Date: Mon, 15 Feb 2021 09:01:40 +0100

The server supports resume when I connect from my machine. I can't access
the corrupted file, as the IoT device has poor connection, I can only see
the stdout of curl in the log output which is uploaded to a server.

Kind regards, Mihail


On Fri, Feb 12, 2021 at 12:02 PM <curl-users-request_at_cool.haxx.se> wrote:

> Send curl-users mailing list submissions to
> curl-users_at_cool.haxx.se
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
> or, via email, send a message with subject or body 'help' to
> curl-users-request_at_cool.haxx.se
>
> You can reach the person managing the list at
> curl-users-owner_at_cool.haxx.se
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of curl-users digest..."
>
>
> Today's Topics:
>
> 1. Re: Corrupted download when using the resume option?! (Ray Satiro)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 12 Feb 2021 00:14:53 -0500
> From: Ray Satiro <raysatiro_at_yahoo.com>
> To: curl-users_at_cool.haxx.se
> Subject: Re: Corrupted download when using the resume option?!
> Message-ID: <24ed09da-a625-ad78-b690-46b3b06c12a7_at_yahoo.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 2/10/2021 9:46 AM, Mihail Kostira via curl-users wrote:
> > I'm using curl to download a 10mb file over a poor connection.
> >
> > I use the '-C -' option in a loop to resume the download if it breaks
> > (see the snippet below).
> >
> > This works when I test locally and interrupt the download by exiting
> > the shell script - after restarting the script, it will resume the
> > download.
> >
> > However, in an IoT device in the field, midway through the transfer,
> > curl errors out with "curl: (18) transfer closed with 2674439 bytes
> > remaining to read". After that, the download is retried within the
> > loop and completes - but the file is corrupt. This has happened
> > multiple times (same device).
> >
> > Am I missing something, maybe I'm using an incompatible set of
> > options? Or is it possible that the poor connection results in a
> > corrupt transfer?
> >
> > downloadFileResume() {
> > # $1 contains the URL to download
> > local fileName=$(basename "$1")
> > for (( i=0; i<=$numCurlRetries; i++ ))
> > do
> > # capture curl http code (-w %{http_code}) in stdout
> > curlHttpCode=$(curl -kSLf -u "$REPO_USERNAME:$REPO_PASSWORD"
> > -w %{http_code} -O -C - "$1")
> > local res=$?
> > [ $res -eq 0 ] && return
> > # curl gives error when attempting to resume a completely
> > downloaded file, it's a 416 error from the server
> > [ $curlHttpCode -eq 416 ] && echo "Error 416, assuming file is
> > downloaded completely" && return
> > done
> > echo "failed to download $fileName"
> > return 1
> > }
>
>
> Maybe the server does not support file resume? How is it corrupted, did
> you check the contents at the end?
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> curl-users mailing list
> curl-users_at_cool.haxx.se
> https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
>
>
> ------------------------------
>
> End of curl-users Digest, Vol 186, Issue 10
> *******************************************
>


-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-02-15