On 3/16/2017 4:18 PM, doa379 wrote:
>> On 3/16/2017 3:16 PM, doa379 wrote:
>>> I'm using the API to fetch data and am using the WRITEDATA,
>>> WRITEFUNCTION options to handle it. Unfortunately I am finding that in
>>> some instances where the data is too long it gets separated in more
>>> than one call of the WRITEFUNCTION callback function. At this point
>>> the application loses track of the data because I am using the API to
>>> fetch other URLs as well. Is there any way to control the size of the
>>> buffer so that all of the data can be handled in a single call of
>>> WRITEFUNCTION?
>>>
>>> An earlier approach that I tried was to have predefined slots for each
>>> of the URLs (tracks of data). In which case each of the slots got
>>> populated with data at leisure over multiple calls of the
>>> WRITEFUNCTION. This works but there's some drawback in efficiency and
>>> this implementation faces some other symptomatic inadequacies with the
>>> cURL API.
>>>
>>> I have now decided to use a single WRITEDATA buffer. But I wish to set
>>> or increase its size so that WRITEFUNCTION need only be called a
>>> single time for any given track of data or URL.
>>
>> No, there's no guarantee of how many times the write function will be
>> called regardless of the data size. If you are doing multiple transfers
>> in parallel using multi you'll need to account for that, in your case I
>> suspect by using separate buffers.
>>
>>
>
>
> That's a solution I got from you IIRC, to store the data into multiple
> slots. Based on the getinmemory.c prototype. I am trying to look at
> this problem again in order to tidy it up using a single buffer slot
> if possible.
>
> Yes, I appreciate there is no control over how many times the
> WRITEFUNCTION will be called. But I would like to parse the data
> obtained through any given URL using a single pass of WRITEFUNCTION.
> Well I suppose it's possible implementing a more elaborate parser by
> using intermediate storage to reconstruct the structure of the data. I
> am curious why are the data chunks sent to WRITEFUNCTION they size
> that they are and if it was possible to adjust the size of these chunks.

libcurl passes the data to your callback as soon as it gets it. It may
get it all at once or in pieces. Please review CURLOPT_WRITEFUNCTION
[1]. If you can help it change your design so you don't have to act on a
transfer's received data until after that transfer has completed.

[1]: https://curl.haxx.se/libcurl/c/CURLOPT_WRITEFUNCTION.html

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-03-16