cURL / Mailing Lists / curl-library / Single Mail

curl-library

Bug: cURL should check Content-Length before assuming the server does not support resume

From: Robert Xiao <brx_at_cs.cmu.edu>
Date: Fri, 22 Aug 2014 13:42:52 -0400

cURL may erroneously produce this error:

> curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.

when asked to continue downloading a file that is already completed. This
happens with servers that do not send Content-Range headers when replying
with the HTTP 416 error ("Requested Range Not Satisfiable"), even if the
server ordinarily supports Content-Range. In particular, when attempting to
continue a completed download, cURL will request a byte range past the end
of the file, which triggers a 416 error.

Test URL which triggers this bug:
http://www.ngdc.noaa.gov/mgg/global/relief/ETOPO1/data/ice_surface/grid_registered/georeferenced_tiff/ETOPO1_Ice_g_geotiff.zip

Note that the omission of Content-Range in the 416 response is permitted by
RFC 2616 (which notes only that that the response SHOULD include a
Content-Range header).

aria2c, another download utility, handles this by performing an ordinary
GET request and checking the Content-Length header, instead of relying on
the server to send Content-Range. I think a similar approach (maybe using
HEAD instead of GET to avoid overhead) would work for cURL.

Bug reported on StackOverflow: http://stackoverflow.com/q/23586214/1204143

My OS: OS X 10.9.4
cURL -V:

curl 7.37.0 (x86_64-apple-darwin13.2.0) libcurl/7.37.0 OpenSSL/1.0.1h
zlib/1.2.8 libidn/1.26
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp
smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

Thanks,
Robert

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-08-22