dima - thanks for the feedback. some comments below:

On Wed, Jan 30, 2013 at 1:16 PM, Dima Tisnek <dimaqq_at_gmail.com> wrote:
> my suggestions:
> * use http 1.1, i.e. reuse connections, set empty Expect not to wait for
> continue

yes, that one took me a little while to realize, so yes I am setting
an empty Expect and it no longs stalls for a full second like it used
to ;)

> * use curl share, reuse dns

I'm not sure what this means

> * use multi interface, no pipelining, but your requests go out in parallel,
> might be out of your scope

my understanding is multi is for parallelizing requests, certainly a
good thing, but I want to send requests serially for bench-marking
purposes

> * make sure your libcurl and openssl versions are new enough, few years back
> ssl session reuse was broken
>

this is a brand new install of Ubutu Precise so I'm assuming
everything is pretty current

> in my experience, I was able to reach 40ms latency (i.e. 25 serial requests
> per second) against a full LAMP stack that had a lot of php in it. I think,
> if tcp connection is reused, your request is small enough, server is fast,
> your latency should be same as your ping.
>

that's good to know. at least now I have a target number

> test against localhost (or not) apache serving small static file with pycurl
> and then ab, part of apache tools, with 1 connection at a time. results
> should be pretty close.
>

I'm not sure I'd know how or would want to do that. The point of this
exercise is to test a specific REST interface on an existing webserver
and I wouldn't even want to contemplate trying to replicate it locally

-mark

>
>
> On 30 January 2013 18:23, Mark Seger <mjseger_at_gmail.com> wrote:
>>
>> I'm trying to use pycurl to performance some benchmarking by seeing
>> how many small string I can upload per/sec. I'm currently testing
>> with 1k strings for now, and am comparing pycurl's rates with a tool I
>> write using a different library and am finding pycurl slower. To be
>> specific, with this other tool I can do on the order of 15 uploads/sec
>> but with pycurl it's closer to 5. I'm just trying to understand if
>> there are some additional settings I might use or if the problem might
>> actually be on the server I'm trying to talk to, though I have no idea
>> how I'd be able to tell.
>>
>> I've been immersing myself (and others) in tcpdumps, ssldumps and even
>> straces. A couple of things did jump out, though I don't know how
>> significant they are and was wondering if someone could comment:
>> - according to ssldump, the other tool says its client version is 3.1
>> and its ciper suite is TLS_DHE_RSA_WITH_AES_256_CBC_SHA. pycurl
>> shows a client version of 3.2 and its cipher is
>> TLS_DHE_RSA_WITH_AES_128_CBC_SHA
>> - there were also a couple of differences in the tcpdump output. it
>> looks like the actual upload the tool I wrote does consists of just a
>> few network exchanges and pycurl was a lot more chattier. when I then
>> dug into the strace it looks like there were a lot of calls to poll(),
>> whereas the other tool made just a couple of calls to epoll()
>>
>> The simple answer might come down to this is just the way it is, and
>> if so that's fine, but if there are way to speed things up even a
>> little more that would be great to know too.
>>
>> -mark
>> _______________________________________________
>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
>
>
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
>
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2013-01-30