2015-04-29 13:59 GMT+08:00 Daniel Stenberg <daniel_at_haxx.se>:
> On Wed, 29 Apr 2015, Hongyi Zhao wrote:
>
>> In the above case, the rquested range is completely out of the range.
>> Why curl still return will 0?
>
>
> Because HTTP errors are not (necessarily) curl errors. curl got the HTTP
> (you asked for) just fine.
>
>> While for a local file, curl will return 36 if the rquested range is
>> completely out of the range, see the following:
>
>
> A bit inconsistent but on the other hand I don't think all protocols are
> alike so I'm not entirely sure this is bad.
>
> Are you suggesting we should change any of this?

The motivation is that I want to the following thing:

Pipe curl's output to dd which can seek to the relevant location
 in the output file before writing. This will allow the blocks to
 be retrieved in any order with the help of dd's arg `conv=notrunc'.

For detail, see the following sript for this purpose:

_____________

#! /bin/bash -

mirrorlist=/path/to/mirror/list

errlog=/tmp/getblock$$

flagfile=/tmp/eof$$

declare -A mirrors
while IFS= read -r line ; do
  mirrors["$line"]=1
done < "$mirrorlist"

[ ${#mirrors[@]} -lt 1 ] && exit 1

get_block()
{
  local bnum url outfile bsize
  local fbyte lbyte
  bnum=$1
  url=$2
  outfile=$3
  case $4 in
    "" | *[!0-9]* ) bsize=$(( 1024 * 1024 )) ;;
    * ) bsize=$4 ;;
  esac
  fbyte=$(( ( $bnum - 1 ) * $bsize ))
  lbyte=$(( $bnum * $bsize - 1 ))
  curl -r ${fbyte}-${lbyte} "$url" |
  dd conv=notrunc bs=$bsize of="$outfile" seek=$(( $bnum - 1 )) \
     2>&1 >/dev/null |
  head -n 1 |
  grep -q '^0'
  set -- "${PIPESTATUS[@]}"
  # this next line may be needed to catch range request errors (eof).
   [ $1 -eq 33 ] && set -- 0 0 0 0
  if [ $1 -ne 0 ]; then
    printf '%d %s\n' $bnum "$url" >> "$errlog"
  elif [ $4 -ne 1 ]; then
    printf 'EOF\n' > "$flagfile"
  fi
}

# First run through the mirror list repeatedly until either the file is
# downloaded (possibly with some missing blocks).
rm -f -- "$flagfile"
i=0
while [ ! -f "$flagfile" ]; do
  for u in "${!mirrors[@]}" ; do
    [ -f "$flagfile" ] && break
    i=$(( $i + 1 ))
    get_block $i "$u" local_file &
  done
  wait
done

# Re-try the failed blocks.
while [ -s "$errlog" ]; do
  a=()
  while read b u ; do
    unset mirrors["$u"]
    a+=( $b )
  done < "$errlog"
  [ ${#mirrors[@]} -gt 0 ] || break
  : > "$errlog"
  set -- "${a[@]}"
  rm -f -- "$flagfile"
  while [ ! -f "$flagfile" ]; do
    for u in "${!mirrors[@]}" ; do
      [ $# -lt 1 ] && printf 'EOL\n' > "$flagfile"
      [ -f "$flagfile" ] && break
      get_block $1 "$u" local_file &
      shift
    done
    wait
  done
done
______________

The above script is given by Dave Sines on the comp.unix.shell
newsgroup, for detail, you can see it from the following talking:

_____________________
From: Dave Sines <dave.gma+news002_at_googlemail.com.invalid>
Newsgroups: comp.unix.shell
Subject: Re: Is it possible to download different parts of a file and put
 them together simultaneously?
Date: Mon, 27 Apr 2015 05:41:28 +0100
Organization: A noiseless patient Spider
Lines: 138
Message-ID: <o8r01cxu27.ln2_at_perseus.wenlock-data.co.uk>
_________________________

The above script based on the curl's exit code 33 to catch the http
range request errors (eof), i.e.,
the following code line:

  # this next line may be needed to catch range request errors (eof).
   [ $1 -eq 33 ] && set -- 0 0 0 0

But just as you have told, in the http case, even the rquested range
is completely out of the range.
curl still return will 0 and not treat is as a error.

So, I've two issues:

[1] When I use curl to retrieve a remote file via http(s)/ftp(s)/sftp
protocals, how can I catch the
rquested range error based on exit code when the range is completely
out of the files size range?

[2] In the curl manpage, I can find the following information at the end part:

  33 HTTP range error. The range "command" didn't work.

But as you have told me, even the http rquested range is completely
out of the range,
curl still return will 0 and not treat is as a error. If so, why you
setting the built-in exit code 33?

Or, in other words, which case is refered by the `HTTP range error'
depicted in the exit code 33?

Regards

>
>
> --
>
> / daniel.haxx.se
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-users
> FAQ: http://curl.haxx.se/docs/faq.html
> Etiquette: http://curl.haxx.se/mail/etiquette.html

-- 
Hongyi Zhao <hongyi.zhao_at_gmail.com>
Xinjiang Technical Institute of Physics and Chemistry
Chinese Academy of Sciences
GnuPG DSA: 0xD108493
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html