On Thu, 21 Jun 2007, Allen Pulsifer wrote:

> I had assumed that curl_multi_perform would call the CURLOPT_WRITEFUNCTION
> function at most once every time it is called. This ensures that my
> application never has to deal with more than CURL_MAX_WRITE_SIZE bytes until
> it is ready for more.

But why would it be limited to that? What says it can't receive more than
CURL_MAX_WRITE_SIZE from the peer and if it has received the data already,
why shouldn't it deliver it to the app?

> The problem is that I have seen curl_multi_perform call
> CURLOPT_WRITEFUNCTION more than once per invocation, delivering in total
> more than CURL_MAX_WRITE_SIZE bytes. So far in testing, it has sometimes
> called it twice, but there seems to be no guarantee. If can call
> CURLOPT_WRITEFUNCTION more than once, it could conceivably call
> CURLOPT_WRITEFUNCTION an indeterminate number of times, essentially breaking
> the "pull" aspect of the interface.

I don't see how it breaks the pull aspect as it does this only when you ask
for it and it stops when there's nothing left to pull. I understand it breaks
the way you think of the concept, but I don't think I completely agree with
that.

> So in summary, the problem is that curl_multi_perform sometimes calls
> CURLOPT_WRITEFUNCTION more than once each time it is called, which delivers
> in total more than CURL_MAX_WRITE_SIZE and overwhelms my application with
> data. The solution would be to ensure that curl_multi_perform can call
> CURLOPT_WRITEFUNCTION at most one time before returning.

In my view it is more about being able to better control how much data libcurl
should be allowed to deliver to the application.

You consider a callback with the maximum amount of data to be what you can
deal with, but I figure there might be other users who would rather prefer to
control it to deliver even smaller pieces at a time if they could...

I think it would make a lot of sense to provide such means. The docs/TODO
document has mentioned a desire for a better "flow control" for both uploads
and downloads for several years - and I think this is pretty much that.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html