curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Fwd: libcurl read-like interface

From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Wed, 16 Dec 2020 13:58:46 +0100 (CET)

On Wed, 16 Dec 2020, XSLT2.0 via curl-library wrote:

> Naive "reverse" benchmark adding an extra useless memcpy:
> https://gitlab.com/BylonAkila/astreamfs/-/blob/master/benchmark/benchmark.c

Oh right, that's a fun way to try stuff.

> Summary impact of a *single memcpy* on a typical HTTP GET: 11% to 15%
> (depending mainly on CPU) on "user time".

But why measure user time? We want to see the performance inpact as a whole,
right? How much extra time does that memcpy() relay a 10GB transfer. Then we
measure wall clock time.

> You can repeat the tests with your own machine, instructions and comments
> are in the short source.

I did two things to spice up the test in my end:

1. CURLOPT_BUFFERSIZE = 102400L (for localhost transfers, this makes a
    significant difference in speed)
2. I compare what time says for "real"

The result? I basically cannot detect a slowdown in the SLOW version! I don't
fully understand why, but it rather seems the SLOW version is actually
slightly *faster* than the original version. Although the non-SLOW version
uses significantly less user time - even a biffer diff than in your tests I'd
say.

On my host, I did 3 passes with each version, downloading 8GB from localhost:

org

real 0m2.346s
real 0m2.304s
real 0m2.304s

SLOW

real 0m2.087s
real 0m2.124s
real 0m2.108s

This uses a rather old core i7: Intel(R) Core(TM) i7-3770K CPU _at_ 3.50GHz

> Question: would you think this "more realistic" test has any interest as a
> "reverse" benchmark?

I'm not sure it adds much value to create another synthetic benchmark.

> So yes, the valid question would be: how many users of libcurl need
> something much more sophisticated than "writing to a file" with some
> "decoration" around, and would benefit from "less copies"?

I think that is *a* question, but for me that's not the important one.
Primarily because I know for sure that many users take care of the data
themselves in a callback and can benefit from fewer copies.

My primary question is probably more:

what's the performance gain for the user who'd want to use this, and can the
API be done in a way that makes this feature practical/attractive.

-- 
  / daniel.haxx.se
  | Commercial curl support up to 24x7 is available!
  | Private help, bug fixes, support, ports, new features
  | https://www.wolfssl.com/contact/
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html
Received on 2020-12-16