curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: a strlen update

From: Henrik Holst via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 14 Feb 2022 20:23:40 +0100

Those single byte memcpy() comes from alloc_addbyter() which in turn is
called by Curl_dyn_vprintf() and curl_mvaprintf() and I can see from traces
that they are e.g called like this:

   0.180 us [ 66160] | strlen(".curlrc") = 7;
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.110 us [ 66160] | memcpy(0x5616b0c976c5,
0x7fff3533def7, 1);
   0.290 us [ 66160] | } /* dyn_nappend */
   0.401 us [ 66160] | } /* Curl_dyn_addn */
   0.501 us [ 66160] | } /* alloc_addbyter */
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.100 us [ 66160] | memcpy(0x5616b0c976c6,
0x7fff3533def7, 1);
   0.280 us [ 66160] | } /* dyn_nappend */
   0.380 us [ 66160] | } /* Curl_dyn_addn */
   0.481 us [ 66160] | } /* alloc_addbyter */
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.100 us [ 66160] | memcpy(0x5616b0c976c7,
0x7fff3533def7, 1);
   0.280 us [ 66160] | } /* dyn_nappend */
   0.380 us [ 66160] | } /* Curl_dyn_addn */
   0.480 us [ 66160] | } /* alloc_addbyter */
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.100 us [ 66160] | memcpy(0x5616b0c976c8,
0x7fff3533def7, 1);
   0.280 us [ 66160] | } /* dyn_nappend */
   0.380 us [ 66160] | } /* Curl_dyn_addn */
   0.491 us [ 66160] | } /* alloc_addbyter */
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.100 us [ 66160] | memcpy(0x5616b0c976c9,
0x7fff3533def7, 1);
   0.280 us [ 66160] | } /* dyn_nappend */
   0.380 us [ 66160] | } /* Curl_dyn_addn */
   0.481 us [ 66160] | } /* alloc_addbyter */
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.100 us [ 66160] | memcpy(0x5616b0c976ca,
0x7fff3533def7, 1);
   0.280 us [ 66160] | } /* dyn_nappend */
   0.381 us [ 66160] | } /* Curl_dyn_addn */
   0.481 us [ 66160] | } /* alloc_addbyter */
            [ 66160] | alloc_addbyter() {
            [ 66160] | Curl_dyn_addn() {
            [ 66160] | dyn_nappend() {
   0.100 us [ 66160] | memcpy(0x5616b0c976cb,
0x7fff3533def7, 1);
   0.270 us [ 66160] | } /* dyn_nappend */
   0.381 us [ 66160] | } /* Curl_dyn_addn */
   0.481 us [ 66160] | } /* alloc_addbyter */
  17.002 us [ 66160] | } /* dprintf_formatf */

So first ".curlrc" is strlen:ed and then 7 calls to alloc_addbyter() which
is the exact length of ".curlrc" so it looks like some function goes over
strings one byte at a time and adds them to a dynamic buffer one by one.
Perhaps this is done to exclude invalid characters or something?

Many times a file path is first strlen:ed when I see these one byte memcpy
(but far from every time). alloc_addbyter() could most likely be optimized
to do a single write to the buffer instead of calling memcpy() by way of
Curl_dyn_addn(buf, out, 1) but I think some one should take a look at why
those file paths are gone over one by one (there could be a good reason)
and if there might be a better approach.

/HH

Den mån 14 feb. 2022 kl 09:33 skrev Daniel Stenberg via curl-library <
curl-library_at_lists.haxx.se>:

> Other fun observations running this same command line:
>
> The top-25 lib calls done:
>
> 2239 free
> 635 malloc
> 545 memcpy
> 300 memchr
> 245 strcmp
> 225 strlen
> 162 inet_pton
> 134 clock_gettime
> 110 OPENSSL_sk_value
> 110 ASN1_STRING_length
> 110 ASN1_STRING_get0_data
> 75 strdup
> 63 sigaction
> 56 calloc
> 50 nghttp2_session_get_stream_user_data
> 38 realloc
> 37 ERR_clear_error
> 36 strchr
> 31 poll
> 31 nghttp2_session_want_read
> 28 nghttp2_session_send
> 26 nghttp2_session_mem_recv
> 26 nghttp2_session_check_request_allowed
> 25 getenv
> 24 SSL_set_ex_data
>
> Those 545 memcpy() calls are interesting because 384 of them copy only a
> single byte!
>
> --
>
> / daniel.haxx.se
> | Commercial curl support up to 24x7 is available!
> | Private help, bug fixes, support, ports, new features
> | https://curl.se/support.html
> --
> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
> Etiquette: https://curl.haxx.se/mail/etiquette.html
>


-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2022-02-14