Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: relative performance
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Ben Greear via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 2 Sep 2021 11:36:07 -0700
On 9/2/21 11:31 AM, Rich Gray via curl-library wrote:
> Ben Greear via curl-library wrote:
>> On 8/24/21 8:20 AM, Daniel Stenberg via curl-library wrote:
>>> Hi all,
>>>
>>> For a long time I've wanted to get something done that allows us to compare curl's relative performance. Ideally something we can run every once in a while
>>> to compare that nothing major has turned sour without us being aware of it.
>>>
>>> A first step would be a tool we can run that measures "relative performance". Like doing N transfers of size X and measure how fast it can complete them.
>>> Running the same tool on the same host with the same server but built to use different libcurl versions should then not get noticably worse speeds over time.
>>> (Barring the difficulty of measuring network things when other programs are also running on the test host.)
>>>
>>> I'm not sure exactly how to do this, but I have a first shot at such a tool written and I figured we can create a new repository for this (curl/relative I'm
>>> thinking) and perhaps add more smaller tools for various tests as we advance. Then work out how to actually run them with different/current libcurls.
>>>
>>> Thoughts?
>>>
>>
>> What is your network-under-test in this case?
>>
>> And, if you want a network emulator (and don't want to mess with
>> netem), contact me off list, I'll give you free software licenses
>> for our product. Our rig can easily bundle a web server since a (virtual)
>> routed network too, so it should be a pretty complete test rig for this
>> case if you wish...
>>
>> Thanks,
>> Ben
>>
>
> Daniel, this is a generous offer you should probably accept, unless you already have such a tool. Performance issues aside, the ability to make a slow/ratty
> network will force all sorts of edge cases in the face of delays and timeouts could be of considerable value. I found all sorts of bugs in some SNMP code I was
> working on when I started inserting delays, dropping and duplicating packets, etc. While higher level protocols will hide a lot of packet level crud, I still
> would expect emulating a glitchy network could drive curl through some of curl's less frequented code.
>
> In general, I don't know if it makes sense to do performance testing against internet servers without running multiple passes to normalize any anomalies which
> are happening along the uncontrollable connection path. A local setup with the ability to control speed, congestion, dropped/duplicated packets, timeouts,
> failed connections, etc. is more real world than just make it go as fast as possible with the fastest connection available. (Tests should be repeatable.) Of
> course, how's it do flat out is useful too.
>
> Cheers!
> Rich
Hello Rich,
I'll make same offer of free software licenses to you if you want to use our software to test curl, with resulting
test results contributed back to the curl community. We can provide at least minimal support to get your started.
Thanks,
Ben
Date: Thu, 2 Sep 2021 11:36:07 -0700
On 9/2/21 11:31 AM, Rich Gray via curl-library wrote:
> Ben Greear via curl-library wrote:
>> On 8/24/21 8:20 AM, Daniel Stenberg via curl-library wrote:
>>> Hi all,
>>>
>>> For a long time I've wanted to get something done that allows us to compare curl's relative performance. Ideally something we can run every once in a while
>>> to compare that nothing major has turned sour without us being aware of it.
>>>
>>> A first step would be a tool we can run that measures "relative performance". Like doing N transfers of size X and measure how fast it can complete them.
>>> Running the same tool on the same host with the same server but built to use different libcurl versions should then not get noticably worse speeds over time.
>>> (Barring the difficulty of measuring network things when other programs are also running on the test host.)
>>>
>>> I'm not sure exactly how to do this, but I have a first shot at such a tool written and I figured we can create a new repository for this (curl/relative I'm
>>> thinking) and perhaps add more smaller tools for various tests as we advance. Then work out how to actually run them with different/current libcurls.
>>>
>>> Thoughts?
>>>
>>
>> What is your network-under-test in this case?
>>
>> And, if you want a network emulator (and don't want to mess with
>> netem), contact me off list, I'll give you free software licenses
>> for our product. Our rig can easily bundle a web server since a (virtual)
>> routed network too, so it should be a pretty complete test rig for this
>> case if you wish...
>>
>> Thanks,
>> Ben
>>
>
> Daniel, this is a generous offer you should probably accept, unless you already have such a tool. Performance issues aside, the ability to make a slow/ratty
> network will force all sorts of edge cases in the face of delays and timeouts could be of considerable value. I found all sorts of bugs in some SNMP code I was
> working on when I started inserting delays, dropping and duplicating packets, etc. While higher level protocols will hide a lot of packet level crud, I still
> would expect emulating a glitchy network could drive curl through some of curl's less frequented code.
>
> In general, I don't know if it makes sense to do performance testing against internet servers without running multiple passes to normalize any anomalies which
> are happening along the uncontrollable connection path. A local setup with the ability to control speed, congestion, dropped/duplicated packets, timeouts,
> failed connections, etc. is more real world than just make it go as fast as possible with the fastest connection available. (Tests should be repeatable.) Of
> course, how's it do flat out is useful too.
>
> Cheers!
> Rich
Hello Rich,
I'll make same offer of free software licenses to you if you want to use our software to test curl, with resulting
test results contributed back to the curl community. We can provide at least minimal support to get your started.
Thanks,
Ben
-- Ben Greear <greearb_at_candelatech.com> Candela Technologies Inc http://www.candelatech.com -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2021-09-02