On Thu, Jul 01, 2010 at 09:47:36PM -0600, Donald Boissonneault wrote:
> I also tried this, however I think that I need to set the htmlbody in
> the write_function due to the fact the treads could be overwriting one
> an other varablies. However I can not for the life of me figure out how
> to pass it an integer so I know which of the array's to store it in. I
> will be working on this later tonight and will keep you posted.

A couple of thoughts: if you're new to C programming, it might be better
to start with a single-threaded program and work from there. The
10-at-a-time example uses the Curl_multi interface, which is not
multi-threaded, but does allow you to perform multiple transfers at
once. In some performance tests that I ran about a month ago, it was
fastest to use one multi handle with up to 50 simultaneous requests.
After that, threading showed a benefit. This was on Solaris, so
performance may be different elsewhere.

That said, your write-function isn't going to work in a multi-threaded
environment. The pointer 'memory', is declared globally, but
manipulated by each thread while it performs the write. I'm surprised
that this works at all. If you're dead-set on storing this data in an
array, you'll need to use a mutex to synchronize access to the parts of
the routine that can be modified by multiple threads at once.
Personally, I'd store these completed objects in a linked list, but if
you have to use an array, keep a global counter that points to the next
free entry. Only allow threads to store the pointer in the array when
they're holding the lock, and have them increment the counter after
assigning the pointer. This is a simplified explanation of the
procedure, but it should be enough to get you started.

You could avoid the need for most of this by using the curl_multi
interface. It allows you to perform multiple downloads using a single
thread of control.

-j
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-07-02