Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Fwd: libcurl read-like interface
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Wed, 16 Dec 2020 13:58:46 +0100 (CET)
On Wed, 16 Dec 2020, XSLT2.0 via curl-library wrote:
> Naive "reverse" benchmark adding an extra useless memcpy:
> https://gitlab.com/BylonAkila/astreamfs/-/blob/master/benchmark/benchmark.c
Oh right, that's a fun way to try stuff.
> Summary impact of a *single memcpy* on a typical HTTP GET: 11% to 15%
> (depending mainly on CPU) on "user time".
But why measure user time? We want to see the performance inpact as a whole,
right? How much extra time does that memcpy() relay a 10GB transfer. Then we
measure wall clock time.
> You can repeat the tests with your own machine, instructions and comments
> are in the short source.
I did two things to spice up the test in my end:
1. CURLOPT_BUFFERSIZE = 102400L (for localhost transfers, this makes a
significant difference in speed)
2. I compare what time says for "real"
The result? I basically cannot detect a slowdown in the SLOW version! I don't
fully understand why, but it rather seems the SLOW version is actually
slightly *faster* than the original version. Although the non-SLOW version
uses significantly less user time - even a biffer diff than in your tests I'd
say.
On my host, I did 3 passes with each version, downloading 8GB from localhost:
org
real 0m2.346s
real 0m2.304s
real 0m2.304s
SLOW
real 0m2.087s
real 0m2.124s
real 0m2.108s
This uses a rather old core i7: Intel(R) Core(TM) i7-3770K CPU _at_ 3.50GHz
> Question: would you think this "more realistic" test has any interest as a
> "reverse" benchmark?
I'm not sure it adds much value to create another synthetic benchmark.
> So yes, the valid question would be: how many users of libcurl need
> something much more sophisticated than "writing to a file" with some
> "decoration" around, and would benefit from "less copies"?
I think that is *a* question, but for me that's not the important one.
Primarily because I know for sure that many users take care of the data
themselves in a callback and can benefit from fewer copies.
My primary question is probably more:
what's the performance gain for the user who'd want to use this, and can the
API be done in a way that makes this feature practical/attractive.
Date: Wed, 16 Dec 2020 13:58:46 +0100 (CET)
On Wed, 16 Dec 2020, XSLT2.0 via curl-library wrote:
> Naive "reverse" benchmark adding an extra useless memcpy:
> https://gitlab.com/BylonAkila/astreamfs/-/blob/master/benchmark/benchmark.c
Oh right, that's a fun way to try stuff.
> Summary impact of a *single memcpy* on a typical HTTP GET: 11% to 15%
> (depending mainly on CPU) on "user time".
But why measure user time? We want to see the performance inpact as a whole,
right? How much extra time does that memcpy() relay a 10GB transfer. Then we
measure wall clock time.
> You can repeat the tests with your own machine, instructions and comments
> are in the short source.
I did two things to spice up the test in my end:
1. CURLOPT_BUFFERSIZE = 102400L (for localhost transfers, this makes a
significant difference in speed)
2. I compare what time says for "real"
The result? I basically cannot detect a slowdown in the SLOW version! I don't
fully understand why, but it rather seems the SLOW version is actually
slightly *faster* than the original version. Although the non-SLOW version
uses significantly less user time - even a biffer diff than in your tests I'd
say.
On my host, I did 3 passes with each version, downloading 8GB from localhost:
org
real 0m2.346s
real 0m2.304s
real 0m2.304s
SLOW
real 0m2.087s
real 0m2.124s
real 0m2.108s
This uses a rather old core i7: Intel(R) Core(TM) i7-3770K CPU _at_ 3.50GHz
> Question: would you think this "more realistic" test has any interest as a
> "reverse" benchmark?
I'm not sure it adds much value to create another synthetic benchmark.
> So yes, the valid question would be: how many users of libcurl need
> something much more sophisticated than "writing to a file" with some
> "decoration" around, and would benefit from "less copies"?
I think that is *a* question, but for me that's not the important one.
Primarily because I know for sure that many users take care of the data
themselves in a callback and can benefit from fewer copies.
My primary question is probably more:
what's the performance gain for the user who'd want to use this, and can the
API be done in a way that makes this feature practical/attractive.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ ------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2020-12-16