cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Reducing CURL_MAX_WRITE_SIZE

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Mon, 24 Oct 2011 09:50:58 -0700

On Mon, Oct 24, 2011, Daniel Stenberg wrote:
> Also, the CURL_MAX_WRITE_SIZE define is a bit overused internally and we
> should try to not have as many data buffers depend on that size when we more
> or less encourage users to change it, while lots of buffers internally that
> use that define (most implicitly via BUFSIZE) will require a decent minimum
> size.

I have two autobuilds running with reduced CURL_MAX_WRITE_SIZE. Until
recently, with a size of 400 bytes 10 tests had to be skipped because they
needed a larger size (mostly proxy tunnel tests that use the buffer for
scratch storage). With a size of 80 bytes, 21 tests had to be skipped.

> The two sensible alternatives are probably:
>
> A) leave the functionality as it is and make sure it is documented and test
> properly - and risk that there's a program out there which didn't interpret
> the documentation as it reads now
>
> B) make CURL_MAX_HTTP_HEADER no larger than CURL_MAX_WRITE_SIZE, which will
> make future programs work without any particular exception for header lines
> in a libcurl callback
>
> I'm quite indecisive myself, but I think I prefer option A,
> especially since we have users who modify CURL_MAX_WRITE_SIZE and
> when that is decreased significiantly things will break badly if we
> match the CURL_MAX_HTTP_HEADER accordingly...

An option C) might be to loop around larger buffers and split them into
multiple callbacks, each of which are no larger than CURL_MAX_WRITE_SIZE.
But, if a write callback truly needs such a maximum size, it can do that
loop itself to process the received buffer in multiple parts. Barring a
compelling reason otherwise, I'd vote for A as well.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-10-24