Rob,

As far as your write callback (used by GET), it looks like it is keeping
track of how much total data has been received, but each call the memcpy
call is just copying the received chunk of data to the beginning of the
buffer, rather than copying it after data that has already been received.
So maybe you need to change this line:

      memcpy(stream, ptr, my_size);

To something like this:

      memcpy(((char*) stream) + sizeLeft[thread_id], ptr, my_size);

After you do this, you still need to guard against more than BIG_BUFFER_SIZE
bytes received over multiple calls.

Also, in your read callback, it looks like the first two arguments to memcpy
are reversed.

I hope this is helpful!

Dylan

-----Original Message-----
From: curl-library-bounces_at_cool.haxx.se
[mailto:curl-library-bounces_at_cool.haxx.se] On Behalf Of Rob Mason
Sent: Tuesday, August 24, 2004 11:03 AM
To: curl-library_at_cool.haxx.se
Subject: Re: Multithreaded Issues

The routines protect against too much data overflowing the buffersize
and allow for multiple calls. I thought that maybe if curl changed the
pointer coming in I could run into trouble but that doesnt appear to be
the case.

The two routines are below. Let me know what you think.

Thanks,
Rob

-----

Write callback:
static size_t read_memory_callback(void *ptr, size_t size, size_t
nmemb, void *stream)
{
   size_t retcode;
   long my_size;
   int thread_id;
   int i;

   thread_id = get_our_thread_id();

   if ((nmemb*size) > BIG_BUFFER_SIZE)
   {
      my_size = BIG_BUFFER_SIZE;
   }
   else
   {
      my_size = nmemb * size;
   }

   if (my_size > sizeLeft[thread_id])
   {
      my_size = sizeLeft[thread_id];
   }

   if (my_size > 0)
   {
      memcpy(ptr, stream, my_size);
      sizeLeft[thread_id] -= my_size;
   }

   retcode = my_size;

   return retcode;
}

Read callback:
static size_t write_memory_callback(void *ptr, size_t size, size_t
nmemb, void *stream)
{
   size_t retcode;
   int my_size;
   int thread_id;
   int i;

   thread_id = get_our_thread_id();

   if ((nmemb*size) > BIG_BUFFER_SIZE)
   {
      my_size = BIG_BUFFER_SIZE;
   }
   else
   {
      my_size = nmemb * size;
   }

   if (my_size > 0)
   {
      memcpy(stream, ptr, my_size);
   }

   sizeLeft[thread_id] += my_size;
   retcode = my_size;
   return retcode;
}

> ------------------------------
>
> Message: 9
> Date: Mon, 23 Aug 2004 15:42:06 -0700
> From: "Dylan Salisbury" <dylan_at_tellme.com>
> Subject: RE: Multithreaded Issues
> To: "'libcurl development'" <curl-library_at_cool.haxx.se>
> Message-ID: <20040823224206.5F7138B3_at_mail01.corp.tellme.com>
> Content-Type: text/plain; charset="US-ASCII"
>
> You might be overflowing your data buffers (local variables
> writeBuffer and
> readBuffer). Here are some questions about your callback functions:
>
> 1. What do your callback functions do if the amount of data given to
> them is
> more than BIG_BUFFER_SIZE?
>
> 2. Do they correctly deal with the response being split up between
> multiple
> calls?
>
> 3. Do they assume that the buffer will be initialized to anything in
> particular before the first time libcurl calls them? The code you
> posted
> does not initialize the buffers, so they may contain data from previous
> calls.
>
>> From the code that you posted, I would guess that your callbacks are
>> not
> robust enough, and when you test with all GETs or all PUTs you do not
> notice
> the problem because all the actions are very similar.
>
> If this reply is not helpful enough, you could post your callback
> functions
> to the list, as well as a more specific description of the undesirable
> behavior.
>
>
> Dylan
>
>
Received on 2004-08-24