On 5/3/2019 11:23 PM, Christopher Head via curl-library wrote:
> On Fri, 3 May 2019 17:38:37 -0400
> Ray Satiro via curl-library <curl-library_at_cool.haxx.se> wrote:
>
>> I think this will be unclear to the reporter since he basically wants
>> to upload arbitrary parts of a file [1]. curl supports resuming an
>> upload and sets Content-Range in that case (CURLOPT_RESUME_FROM and
>> CURLOPT_INFILESIZE) but does not support CURLOPT_RANGE for arbitrary
>> parts, would you agree?
> I think the situation is pretty clear to me: I’m doing something
> outside what Curl supports, so I should just use CURLOPT_HTTPHEADER to
> set Content-Range. I’m fine with this.
>
> I think I’m now just more confused about why Curl even supports
> CURLOPT_RANGE for PUTs at all. I’m honestly not seeing how it’s
> possible to use it for *any* purpose, not even for resuming an upload
> with intent to complete the whole file: if you set INFILESIZE to the
> size of the file then Content-Length is too big, and if you set
> INFILESIZE to the size of the residue then the number after the slash
> in Content-Range is too small. So RESUME_FROM for PUTs seems to be OK
> in the sense that it’s not really standards compliant but does
> something vaguely sensible which is useful with some servers, but
> CURLOPT_RANGE for PUTs seems to never do anything useful, and yet,
> despite that, some effort was put into implementing it.
>
> So, for my own purposes I have no problem: I’ll just use HTTPHEADER,
> given that it seems my desired behaviour wasn’t ever meant to work. But
> I’m left wondering about the rest.

I hear you it's confusing to me too which is why I opened the issue to
clarify it. I see now that one type of ranged PUT is supported to allow
resuming uploading a resource but CURLOPT_RANGE is not used for that. To
resume on upload use CURLOPT_INFILESIZE(_LARGE) and
CURLOPT_RESUME_FROM(_LARGE), and Content-Range will be derived from both
of those, and *not* CURLOPT_RANGE. Maybe that is what Daniel meant by
ranged PUTs, or maybe not. As you noted when you tried that the input
file will be read until the end. For example:

printf "abc" > input
curld -v -T input -C 1 http://localhost:8000/destination

CURLOPT_UPLOAD is set to 1L for a PUT upload.
CURLOPT_READDATA is set to internal struct that contains fd of opened
file input.
CURLOPT_SEEKDATA is set to same.
CURLOPT_READFUNCTION is set to read it.
CURLOPT_SEEKFUNCTION is set to seek it.
CURLOPT_INFILESIZE_LARGE is set to the size of file input (curl_off_t)3.
CURLOPT_RESUME_FROM_LARGE is set to the offset to resume from (curl_off_t)1.
CURLOPT_RANGE is set NULL.

The server receives:

PUT /destination HTTP/1.1
Host: localhost:8000
Content-Range: bytes 1-2/3
User-Agent: curl/7.65.0-DEV
Accept: */*
Content-Length: 2
Expect: 100-continue

bc

Also as you noted CURLOPT_RANGE does not work like CURLOPT_RESUME_FROM,
in that the former does not have any effect on Content-Length.
Content-Length is determined by CURLOPT_INFILESIZE minus
CURLOPT_RESUME_FROM. Yet if you specify CURLOPT_RESUME_FROM then
CURLOPT_RANGE is ignored, and only CURLOPT_RESUME_FROM changes the
starting offset of the file.

If you use CURLOPT_RANGE and CURLOPT_INFILESIZE without
CURLOPT_RESUME_FROM then libcurl ends up in this block [1]:

      else {
        /* Range was selected and then we just pass the incoming range and
           append total size */
        conn->allocptr.rangeline =
          aprintf("Content-Range: bytes %s/%" CURL_FORMAT_CURL_OFF_T "\r\n",
                  data->state.range, data->state.infilesize);
      }

But I think that will not work correctly since Content-Length is also
CURLOPT_INFILESIZE. IIRC it is not good to set the Content-Length header
on your own because then internally libcurl doesn't change the length to
0 when it does authentication before the PUT. Therefore I think the best
thing to do here is do not use CURLOPT_RANGE (at least until its use is
clarified) and set the range header yourself, as I suggested. I will try
experimenting with it some later.

[1]: https://github.com/curl/curl/blob/curl-7_64_1/lib/http.c#L2485-L2491

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-05-04