On Tue, 19 Jun 2012, Marc Hoersken wrote:

> thanks! Actually there is one more thing we can discuss. In the current
> implementation the increased buffer sizes are tried to be reduced to the
> initial buffer size after each and every receive operation. On the one hand
> this means that libcurl would free up memory after some high memory usage
> scenario which could be very important for long-term SSL/TLS connections.

I can throw in another thought too. We really should aboid unbounded
enlargements of the buffer as it can possibly lead to a rouge server to try to
exploit this by providing a stream that expands to something ridiculously
large making us allocate huge chunks of memory leading to badness. So there
should probably be a maximum. Also, we should double the buffer space for each
loop, not just expand with bufsize/2 IMHO as that is usually what comes out as
the better algorithm.

Given a ceiling as mentioned above, I think it we should leave the enlarged
buffer allocated to avoid lots of reallocs over the course of a single
transfer. That's what we do for several other buffers we expand in similar
style.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html