OK, I know very little (i.e. almost nothing) about pycurl, but...

Is it possible in pycurl to use the WRITEFUNCTION callback command to
write to memory, as in the "docs/examples/getinmemory.c" example
program?

What the C program does is put the data into a data structure in memory
instead of writing to disk. If pycurl can do the same, the download
speed would be bytes/sec website-to-memory, rather than bytes/sec
website-to-disk.

Once the data is in an array or string variable, curl's part of the
download is over and you have your stats. Then you can dump the data to
disk in your own time and go on to parse it.

If pycurl can't do it, you might want to consider using the
getinmemory.c program almost verbatim to handle the data fetch and get
the speed stats, then go on with pycurl to process the received file.

Ralph Mitchell

bragiba_at_simi.is wrote:

> That´s what I figured. I am little bummed out because of this. My
> plan was basically to do the following:
>
> 1. Download a page like CNN.com
>
> 2. Parse it and download all frame documents and images.
>
> 3. While doing this I wanted to save the page to disk such that I
> could periodically check that what had been downloaded was complete.
>
> During all the downloads I wanted to collect speed data that I am
> getting per download and record that into a database for benchmarking
> purposes
>
> in our GPRS system.
>
> Looks like I just have to skip the writing to disk data and just stick
> with downloading stuff and just blowing it away.
>
>
>
> Does anyone have any ideas on how to do benchmarking like the one I am
> intending.

-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T
handheld. Power & Color in a compact size!
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en
Received on 2002-11-28