I am trying to keep statistics on a page as it is downloading. The program
must be multi threaded and have multiple threads retrieving pages at the
same time, calculating the same statistics. I want to do something like the
following:

In thread[1...n]
1. Initiate a page request
2. While the page is retrieved, count the number of times a particular
string is retrieved. The time is stored when the first occurence is found.
3. After the string is returned a predetermined number of times, get the
time and return the time it took to receive all occurences (difference
between the start and end times). Discontinue retrieving the document.

I am not sure how to pass this information from a callback(
CURLOPT_WRITEFUNCTION ) to the thread that initiated the request. I am also
unsure how to stop retrieving a file once the request is made.

If I can get this to work, the final hurdle I see is really a C programming
problem. If the data sent to the callback has only half of the data I am
looking for, a string search on that data and the data in the next callback
will may not match the string, but a concatenation of both data parcels
will:

Search for string "hello"
request n data: xxxxxxxxxxxxxxxx....xxhel
request n+1 data: loxxxxxxxxxxxxxx....xxxxx

My plan is to concatenate data blocks into a buffer until the string is
found and then perform a memmove of the buffer, starting with character
after the string is found to the beginning of the buffer. If anyone knows
of a better approach or why this approach wouldn't work, I would very much
like to know.

Any help is appreciated. I am new to the library but think it is
fantastic!!

Regards,
Mike Malloy
Received on 2001-12-16