On Mon, 24 Oct 2011, Yang Tse wrote:

> Relative to CURL_MAX_WRITE_SIZE it may be interesting to remember our
> 'ADVISORY: libcurl data callback excessive length'
> http://curl.haxx.se/mail/lib-2010-02/0075.html

Yes, I certainly have that in mind when I tread into this territory and I'm
careful about what guarantees we have made and what promises there are in the
API.

The wording has not been perfect (like referring to the size of "the data" in
the write callback), but then I also believe that very few users actually set
CURLOPT_HEADER to get the header data passed into the write callback.

I also think that very few people actually receive headers that are larger
than CURL_MAX_WRITE_SIZE.

I'll of course value more feedback from people on this.

> Maybe it would be interesting to actually keep the amount of data passed to
> the write callback limited by CURL_MAX_WRITE_SIZE and for the header
> callback by the smaller of CURL_MAX_WRITE_SIZE and CURL_MAX_HTTP_HEADER.

Yes we could indeed, but then we'd kind of break the API anyway since we won't
deliver "full header lines" anymore. Then we could just as well decrease
CURL_MAX_HTTP_HEADER down to CURL_MAX_WRITE_SIZE since there would probably be
of little value to actually hold full header lines without being able to
deliver them to the app...

> Or maybe establish some option which allows using app to limit the amount of
> data the write and header callbacks may return.

Right now I've updated the documentation for how it actually works, and I'm
about to add a new test and client code adjustments to verify that headers
larger than CURL_MAX_WRITE_SIZE can be received by curl...

The two sensible alternatives are probably:

A) leave the functionality as it is and make sure it is documented and test
    properly - and risk that there's a program out there which didn't interpret
    the documentation as it reads now

B) make CURL_MAX_HTTP_HEADER no larger than CURL_MAX_WRITE_SIZE, which will
    make future programs work without any particular exception for header lines
    in a libcurl callback

I'm quite indecisive myself, but I think I prefer option A, especially since
we have users who modify CURL_MAX_WRITE_SIZE and when that is decreased
significiantly things will break badly if we match the CURL_MAX_HTTP_HEADER
accordingly...

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html