You just need to change pycurl's writefunction to that of your file-like
object:

crl = pycurl.Curl()
output = cStringIO.StringIO()
crl.setopt(pycurl.WRITEFUNCTION, output.write)
...
crl.perform()

response = output.getvalue()

You could also use a custom write function if you wanted to parse the
response specially, but you aren't guaranteed to get the response in useful
pieces--i.e. it could be split up mid-line or mid-word. I just store the
whole response and read it out afterward, so that I can be sure that it's
complete.

- James

On Fri, Sep 4, 2009 at 9:44 PM, Josh Mayer <joshuaamayer_at_gmail.com> wrote:

> Hi,
>
> I am looking to use pycurl to fetch urls and store the text in a
> database. I would like to have the urls fetched straight to memory
> (as in the getinmemory.c example) but this seems impossible in pycurl.
> Being able to use StringIO from the cStringIO module instead of a
> file handle would be really nice. Any ideas on how to go about doing
> this? The use case I have is such that the time needed in reading the
> files from disk is significant and would be ideal to eliminate. Thanks
> for the help.
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2009-09-05