I'm not sure about using pycurl with tornado, may be you need to
return 0 from WRITEFUNCTION ?

On Fri, Apr 13, 2012 at 10:05 AM, Andy Pellett <anrope_at_gmail.com> wrote:
> Hi
>
> I'm working with curl via pycurl via tornado, so I might be barking up the
> wrong tree; let me know if I am.
>
> I'm having an issue where my tornado app seems to work fine for a while, but
> over time degrades until it basically isn't working at all (e.g. lots of
> timeouts on processing requests). This makes me think the issue has
> something to do with resource usage. I've watched memory, there doesn't seem
> to be a leak, so now I'm looking at the number of network connections.
>
> I've noticed that when my app starts performing poorly, the number of
> connections reported by:
>
> $ lsof -i
>
> is larger than 'normal'.
>
> I'm worried that I'm doing something wrong when it comes to handling curl
> connections. Here's where I think I might be doing something wrong:
>
> I'm using tornado's curl_httpclient, and basically just plugging in
> customized header and body callbacks.
>
> Sometimes I decide that I don't want to process an HTTP response's body by
> looking at the headers. So, in the curl header callback (set with setopt
> HEADERFUNCTION), I look for certain signs, and set a flag to ignore the body
> of the request. If the flag is set, then in the curl body callback (set with
> setopt WRITEFUNCTION) I return a number different than the number of bytes
> passed in. From reading the libcurl documentation, it seems to me that this
> should abort that transfer and (I'm hoping) free up the easy handle for
> another connection.
>
> So, is this a reasonable thing to do to abort a curl connection? Or am I
> shooting myself in the foot? Any input is appreciated.
>
> Thanks for your time.
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
>
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2012-04-13