> Subject: Re: how libcurl receives chunked data from http server?
> On Sat, 20 Sep 2003, Jerry G. Chiuan wrote:
>
> > My http server responds to the client a huge data that can't be carried
by
> > single TCP segment. the data size = 1160 + 1460*6 + 337 = 10,257
(bytes),
> > please refer to info. shown below
> >
> > As a result, it needs totally 8 TCP segments to transfer, the first one
> > carries http header.
>
> > - If yes, can libcurl receive all binary data at a time in libcurl's
buffer?
>
> libcurl will receive N bytes into its buffer and then call the write
callback.
> You shall not assume any size, as libcurl is not guaranteed to get or use
any
> specific size. It of course has an internal buffer of a certain size so
you
> can be sure to never get any MORE data than CURL_MAX_WRITE_SIZE bytes in
one
> single function call.

Hi Daniel,
very appreciate your explanation.
Here, what is the purpose of receiving partial response data and then call
write callback.
why is it not designed to receive the whole response at a time? in single
buffer?

Is it because libcurl expects application to copy partial data from buffer
and the buffer can be reused by libcurl for rest of response data?
( by doing this, less memory would be consumed )

> > I found that CURLOPT_WRITEFUNCTION would be called several times in
order to
> > receive the whole data.
>
> Yes of course.
>
> > For instance, it might receive first 2 packets, then receive the rest
etc...
> > how can we receive the data at a time with libcurl? can it be
configured?
>
> You can't. Unless you patch libcurl to do so. I just don't see the point
in
> adding such an option.

The reason I want to know more about how libcurl works for that is I would
like to optimize the performance and memory usage.
I hope libcurl can receive the whole response data at a time, even in a
single buffer, then my application doesn't need to copy partial response
data
from libcurl buffers to an extra buffer before dealing with it. ( I mean I c
an avoid one more copy ).
If libcurl doesn't do that,
- first, I need to copy every partial data to an extra buffer, reconstrcut
them over there to be a whole response. ( response consists of several
chunks )
- secondly, copy "one more time" for each chunk from there to the buffer I
have had in my application.

I hope I can avoid "fisrt time" copy!

>You would add significant complexity to the code
> though.

any risk if I do so? will it be really compicated?

Regds,
- Jerry

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Received on 2003-09-22