curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: cURL behaviour when Content-Length is unrepresentable and a max file size is set

From: Brad Spencer <bspencer_at_blackberry.com>
Date: Wed, 3 Jan 2018 10:56:23 -0400

On 2018-01-02 11:02 PM, Daniel Stenberg wrote:
> Recent? I tracked down the parsing using integer back to 2005 and I
> can't recall it using anything else before that either...

AFAIK, until recently, the file sizes were reported through the API as
doubles via CURLINFO_CONTENT_LENGTH_DOWNLOAD. My memory is fuzzy here,
but I thought I used to see very large Content-Length values come all
the way through this API as doubles even when they didn't fit into
curl_off_t. Sorry if I made things confusing here.

>> So my question is, why doesn't cURL close the connection and fail the
>> request in all of these cases?
>
> As with everything: nobody has had any reason to ponder those
> decisions or has been happy with them.

Of course! I didn't mean to imply anything with that question. :)

>> Also, BTW, it's interesting that cURL does not enforce the maximum
>> file size when there is no Content-Length.
>
> I disagree. The intended functionality for that option was always to
> prevent a "too large transfer" to even start. It was not meant to
> abort transfers after N bytes. So if curl doesn't know the size of the
> transfer to come, it won't stop it due to the value of this option.

Fair enough. And I see the option is clearly documented like this.

Personally, I can't think of a situation where an application would only
want the limit enforced when the size is advertised, but I have no
complaint here. I just count the bytes myself. :)

I've been running with a small local change that implements the "abort
on unrepresentable size when a limit is set" behaviour and it's been
doing the trick for me. It sounds like there are a few collateral
changes that you've identified here; they make sense to me too, FWIW.

-- 
Brad Spencer
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2018-01-03