curl-and-python

trying to understand where pycurl overhead is

From: Mark Seger <mjseger_at_gmail.com>
Date: Wed, 30 Jan 2013 12:23:30 -0500

I'm trying to use pycurl to performance some benchmarking by seeing
how many small string I can upload per/sec. I'm currently testing
with 1k strings for now, and am comparing pycurl's rates with a tool I
write using a different library and am finding pycurl slower. To be
specific, with this other tool I can do on the order of 15 uploads/sec
but with pycurl it's closer to 5. I'm just trying to understand if
there are some additional settings I might use or if the problem might
actually be on the server I'm trying to talk to, though I have no idea
how I'd be able to tell.

I've been immersing myself (and others) in tcpdumps, ssldumps and even
straces. A couple of things did jump out, though I don't know how
significant they are and was wondering if someone could comment:
- according to ssldump, the other tool says its client version is 3.1
and its ciper suite is TLS_DHE_RSA_WITH_AES_256_CBC_SHA. pycurl
shows a client version of 3.2 and its cipher is
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- there were also a couple of differences in the tcpdump output. it
looks like the actual upload the tool I wrote does consists of just a
few network exchanges and pycurl was a lot more chattier. when I then
dug into the strace it looks like there were a lot of calls to poll(),
whereas the other tool made just a couple of calls to epoll()

The simple answer might come down to this is just the way it is, and
if so that's fine, but if there are way to speed things up even a
little more that would be great to know too.

-mark
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2013-01-30