Corrupted download when using the resume option?!

From: Mihail Kostira via curl-users <curl-users_at_cool.haxx.se>
Date: Wed, 10 Feb 2021 15:46:09 +0100

Hi all,

I'm using curl to download a 10mb file over a poor connection.

I use the '-C -' option in a loop to resume the download if it breaks (see
the snippet below).

This works when I test locally and interrupt the download by exiting the
shell script - after restarting the script, it will resume the download.

However, in an IoT device in the field, midway through the transfer, curl
errors out with "curl: (18) transfer closed with 2674439 bytes remaining to
read". After that, the download is retried within the loop and completes -
but the file is corrupt. This has happened multiple times (same device).

Am I missing something, maybe I'm using an incompatible set of options? Or
is it possible that the poor connection results in a corrupt transfer?

downloadFileResume() {
    # $1 contains the URL to download
    local fileName=$(basename "$1")
    for (( i=0; i<=$numCurlRetries; i++ ))
    do
        # capture curl http code (-w %{http_code}) in stdout
        curlHttpCode=$(curl -kSLf -u "$REPO_USERNAME:$REPO_PASSWORD" -w
%{http_code} -O -C - "$1")
        local res=$?
        [ $res -eq 0 ] && return
        # curl gives error when attempting to resume a completely
downloaded file, it's a 416 error from the server
        [ $curlHttpCode -eq 416 ] && echo "Error 416, assuming file is
downloaded completely" && return
    done
    echo "failed to download $fileName"
    return 1
}

Many thanks, Mihail


-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-02-10