Digging a little deeper, I discovered what may be happening:

All objects (and their corresponding URLs) are of fixed size, 64KB.
Let's assume that the URL is http://www.example.com/object-1-2-3. The
case is that 2 threads are trying to fetch the same URL by calling the
readData function. One thread is using one writer callback function
(as set in CURLOPT_WRITEFUNCTION) that writes to a file, and the other
thread is using another callback that writes to a memory buffer.

From the documentation: "The callback function will be passed as much
data as possible in all invokes, but you cannot possibly make any
assumptions. It may be one byte, it may be thousands.". What I'm
seeing is that at each call of the callback there's approximately 6K
worth of data passed down to it. There is some sort of "race"
happening here, despite the fact that there's no shared curl handle
between threads.

Any ideas?

Thanks
Alex
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-11-04