my suggestions:
* use http 1.1, i.e. reuse connections, set empty Expect not to wait for
continue
* use curl share, reuse dns
* use multi interface, no pipelining, but your requests go out in parallel,
might be out of your scope
* make sure your libcurl and openssl versions are new enough, few years
back ssl session reuse was broken

in my experience, I was able to reach 40ms latency (i.e. 25 serial requests
per second) against a full LAMP stack that had a lot of php in it. I think,
if tcp connection is reused, your request is small enough, server is fast,
your latency should be same as your ping.

test against localhost (or not) apache serving small static file with
pycurl and then ab, part of apache tools, with 1 connection at a time.
results should be pretty close.

On 30 January 2013 18:23, Mark Seger <mjseger_at_gmail.com> wrote:

> I'm trying to use pycurl to performance some benchmarking by seeing
> how many small string I can upload per/sec. I'm currently testing
> with 1k strings for now, and am comparing pycurl's rates with a tool I
> write using a different library and am finding pycurl slower. To be
> specific, with this other tool I can do on the order of 15 uploads/sec
> but with pycurl it's closer to 5. I'm just trying to understand if
> there are some additional settings I might use or if the problem might
> actually be on the server I'm trying to talk to, though I have no idea
> how I'd be able to tell.
>
> I've been immersing myself (and others) in tcpdumps, ssldumps and even
> straces. A couple of things did jump out, though I don't know how
> significant they are and was wondering if someone could comment:
> - according to ssldump, the other tool says its client version is 3.1
> and its ciper suite is TLS_DHE_RSA_WITH_AES_256_CBC_SHA. pycurl
> shows a client version of 3.2 and its cipher is
> TLS_DHE_RSA_WITH_AES_128_CBC_SHA
> - there were also a couple of differences in the tcpdump output. it
> looks like the actual upload the tool I wrote does consists of just a
> few network exchanges and pycurl was a lot more chattier. when I then
> dug into the strace it looks like there were a lot of calls to poll(),
> whereas the other tool made just a couple of calls to epoll()
>
> The simple answer might come down to this is just the way it is, and
> if so that's fine, but if there are way to speed things up even a
> little more that would be great to know too.
>
> -mark
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2013-01-30