Thanks, it may seem odd, but I don't actually care that i'm constantly
writing over my data on each call. I still haven't run into an
explanation for contention between PUTs and GETs when running
multithreaded.
Rob

On Aug 24, 2004, at 5:37 PM, Dylan Salisbury wrote:

> Rob,
>
> As far as your write callback (used by GET), it looks like it is
> keeping
> track of how much total data has been received, but each call the
> memcpy
> call is just copying the received chunk of data to the beginning of the
> buffer, rather than copying it after data that has already been
> received.
> So maybe you need to change this line:
>
> memcpy(stream, ptr, my_size);
>
> To something like this:
>
> memcpy(((char*) stream) + sizeLeft[thread_id], ptr, my_size);
>
> After you do this, you still need to guard against more than
> BIG_BUFFER_SIZE
> bytes received over multiple calls.
>
> Also, in your read callback, it looks like the first two arguments to
> memcpy
> are reversed.
>
> I hope this is helpful!
>
>
> Dylan
>
> -----Original Message-----
> From: curl-library-bounces_at_cool.haxx.se
> [mailto:curl-library-bounces_at_cool.haxx.se] On Behalf Of Rob Mason
> Sent: Tuesday, August 24, 2004 11:03 AM
> To: curl-library_at_cool.haxx.se
> Subject: Re: Multithreaded Issues
>
> The routines protect against too much data overflowing the buffersize
> and allow for multiple calls. I thought that maybe if curl changed the
> pointer coming in I could run into trouble but that doesnt appear to be
> the case.
>
> The two routines are below. Let me know what you think.
>
> Thanks,
> Rob
>
> -----
>
> Write callback:
> static size_t read_memory_callback(void *ptr, size_t size, size_t
> nmemb, void *stream)
> {
> size_t retcode;
> long my_size;
> int thread_id;
> int i;
>
> thread_id = get_our_thread_id();
>
> if ((nmemb*size) > BIG_BUFFER_SIZE)
> {
> my_size = BIG_BUFFER_SIZE;
> }
> else
> {
> my_size = nmemb * size;
> }
>
> if (my_size > sizeLeft[thread_id])
> {
> my_size = sizeLeft[thread_id];
> }
>
> if (my_size > 0)
> {
> memcpy(ptr, stream, my_size);
> sizeLeft[thread_id] -= my_size;
> }
>
> retcode = my_size;
>
> return retcode;
> }
>
> Read callback:
> static size_t write_memory_callback(void *ptr, size_t size, size_t
> nmemb, void *stream)
> {
> size_t retcode;
> int my_size;
> int thread_id;
> int i;
>
> thread_id = get_our_thread_id();
>
> if ((nmemb*size) > BIG_BUFFER_SIZE)
> {
> my_size = BIG_BUFFER_SIZE;
> }
> else
> {
> my_size = nmemb * size;
> }
>
> if (my_size > 0)
> {
> memcpy(stream, ptr, my_size);
> }
>
> sizeLeft[thread_id] += my_size;
> retcode = my_size;
> return retcode;
> }
>
>> ------------------------------
>>
>> Message: 9
>> Date: Mon, 23 Aug 2004 15:42:06 -0700
>> From: "Dylan Salisbury" <dylan_at_tellme.com>
>> Subject: RE: Multithreaded Issues
>> To: "'libcurl development'" <curl-library_at_cool.haxx.se>
>> Message-ID: <20040823224206.5F7138B3_at_mail01.corp.tellme.com>
>> Content-Type: text/plain; charset="US-ASCII"
>>
>> You might be overflowing your data buffers (local variables
>> writeBuffer and
>> readBuffer). Here are some questions about your callback functions:
>>
>> 1. What do your callback functions do if the amount of data given to
>> them is
>> more than BIG_BUFFER_SIZE?
>>
>> 2. Do they correctly deal with the response being split up between
>> multiple
>> calls?
>>
>> 3. Do they assume that the buffer will be initialized to anything in
>> particular before the first time libcurl calls them? The code you
>> posted
>> does not initialize the buffers, so they may contain data from
>> previous
>> calls.
>>
>>> From the code that you posted, I would guess that your callbacks are
>>> not
>> robust enough, and when you test with all GETs or all PUTs you do not
>> notice
>> the problem because all the actions are very similar.
>>
>> If this reply is not helpful enough, you could post your callback
>> functions
>> to the list, as well as a more specific description of the undesirable
>> behavior.
>>
>>
>> Dylan
>>
>>
>
>
>
Received on 2004-08-24