curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Curl segfault in curl_multi_perform

From: Richard W.M. Jones via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 31 Jul 2023 10:41:34 +0100

http://oirase.annexia.org/tmp/no-stream-verbose.txt

^^ This is the output when nbdkit / curl crashes, with CURLOPT_VERBOSE == 1.

For a bit of background, there are about 270 threads which have
submitted an easy handle to the background thread. The background
thread adds these to a single multi handle and processes it in a loop:

https://gitlab.com/rwmjones/nbdkit/-/blob/e5f11c7df8c648a6e1e11a370f99b6d41ea3b98c/plugins/curl/pool.c#L233

Looking at the core dump, it seems there are 117 easy handles on the
multi when it crashes:

(gdb) print *(struct Curl_multi *)multi
$3 = {magic = 764702, easyp = 0x7fccbc001010, easylp = 0x7fca78001010,
  num_easy = 117, num_alive = 117, msglist = {head = 0x0, tail = 0x0,
    dtor = 0x0, size = 0}, pending = {head = 0x0, tail = 0x0, dtor = 0x0,
    size = 0}, msgsent = {head = 0x0, tail = 0x0, dtor = 0x0, size = 0},
  socket_cb = 0x0, socket_userp = 0x0, push_cb = 0x0, push_userp = 0x0,
  hostcache = {table = 0x7fcd5c006c50,
    hash_func = 0x7fcd6d4e1150 <Curl_hash_str>,
    comp_func = 0x7fcd6d4e2df0 <Curl_str_key_compare>,
    dtor = 0x7fcd6d4e11d0 <freednsentry>, slots = 71, size = 1}, psl = {
    psl = 0x0, expires = 0, dynamic = false}, timetree = 0x7fca78001ec8,
  ssl_backend_data = 0x0, sockhash = {table = 0x0,
    hash_func = 0x7fcd6d4f6380 <hash_fd>,
    comp_func = 0x7fcd6d4f6370 <fd_key_compare>,
    dtor = 0x7fcd6d4f6360 <sh_freeentry>, slots = 911, size = 0},
  conn_cache = {hash = {table = 0x7fcd5c005d20,
      hash_func = 0x7fcd6d4e1150 <Curl_hash_str>,
      comp_func = 0x7fcd6d4e2df0 <Curl_str_key_compare>,
      dtor = 0x7fcd6d4c92c0 <Curl_resolver_cleanup>, slots = 97, size = 1},
    num_conn = 5, next_connection_id = 5, next_easy_id = 133, last_cleanup = {
      tv_sec = 3012568, tv_usec = 71694}, closure_handle = 0x558655710cf0},
  maxconnects = -1, max_host_connections = 0, max_total_connections = 16,
  timer_cb = 0x0, timer_userp = 0x0, timer_lastcall = {tv_sec = 0,
    tv_usec = 0}, max_concurrent_streams = 100, wakeup_pair = {3, 4},
  ipv6_up = 2 '\002', multiplexing = 1, recheckstate = 0, in_callback = 0,
  ssl_seeded = 0, dead = 0}


I can reproduce the problem very readily. The problem for
reproduction is not building nbdkit, but having a local HTTP/2 server
with a sufficiently large file (10GB in this case). The reproducer
command (run from the nbdkit build directory) is:

$ ./nbdkit -r -U - curl http://dev3.home.annexia.org/html/fedora-36.qcow2 http-version=2.0 --run 'nbdcopy -p $uri null: --connections=16 ' -fv -D curl.verbose=1

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html
Received on 2023-07-31