curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Couple cuestions about CURLOPT_WRITEFUNCTION callback

From: David Requena <drequena_at_gmail.com>
Date: Mon, 13 Feb 2017 15:03:16 +0100

2017-02-13 14:15 GMT+01:00 Daniel Stenberg <daniel_at_haxx.se>:

> On Mon, 13 Feb 2017, David Requena wrote:
>
> CURLOPT_WRITEFUNCTION's documentation does explain very little about data
>> buffering, just stating default max data size is 16K. I'm afraid libcurl
>> might keep buffering events and only deliver them to my write callback in a
>> big batch. That of course would negate the whole SSE idea.
>>
>
> The man page also says
>
> This callback function gets called by libcurl as soon as there is data
> received that needs to be saved
>

But it also says

    The callback function will be passed as much data as possible in all
    invokes, but you must not make any assumptions. It may be one
    byte, it may be thousands.

:-)

> I think that implies the data is delivered fairly soon after getting
> received and not a lot of extra buffering.
>

I would expect but nothing indicates that's actually the case :-(

> No, there's no direct way to affect that. You can shrink the used buffer
> size (CURLOPT_BUFFERSIZE) which indirectly will make the function get
> called more often, but you still won't get it get called sooner because of
> that.
>

Got to find CURLOPT_BUFFERSIZE shortly after sending my initial inquiry and
thought "aha!". Unfortunately:

    This is just treated as a request, not an order. You cannot be
    guaranteed to actually get the given size.

> libcurl uses non-blocking sockets and will call the write callback as soon
> as it has received data into its buffer. It doesn't know nor care about
> linefeeds or text vs binary. Everything is the same to libcurl: data.
>

That much I did gather from the docs

libcurl will actually "hold" the data as short as possible before passing
> it on to the application.

There is no hint about what the triggering conditions for the callback
invocation might be (some timeout with no received data, whole chunk
received, buffer 90% full, all of the former?, some of them?)

At this point I'm at a loss. I'll be receiving some small data item every
couple seconds and cannot see how I'll be processing them in a timely
manner.
I'm of course very new to libcurl and might be missing something obvious.
I'm open to ideas or suggestions for further investigation.

Thanks again!

Saludos / Regards,
David Requena

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-02-13