Hello curl-library hackers,

I am seeing an intermittent (2-3 times a week) segfault inside libcurl
code, and it is frequent enough that it is becoming a real problem.
I'm having problems finding the root cause so any help would be
appreciated.

A bit of context: This is an application that is responsible for
fetching data from multiple sources via HTTP and uploading the results
to a central server via HTTPS. It is multi-threaded, but to rule out
any threading problems a global mutex is locked around use of libcurl.
This at least reduced the number of different stack traces I am seeing
to this one:

(gdb) where
<<snip>>
#6 <signal handler called>
#7 0x080a8e12 in Curl_expire_clear (data=0xb3c1acf4) at multi.c:3022
#8 0x080be7f7 in Curl_close (data=0xb3c1acf4) at url.c:396
#9 0x080a7986 in curl_multi_cleanup (multi=0xb3c19b8c) at multi.c:2211
#10 0x080be833 in Curl_close (data=0xb3c03204) at url.c:408
#11 0x080a126a in curl_easy_cleanup (data=0xb3c03204) at easy.c:833
#12 0x080876da in HTTPClient::HTTPRequest (this=0x98ad74c,
    aURL=0xb3c13604 "http://localhost:50080/<<redacted>>",
    aPostData=0x0, aPostSize=0, aPage=0xb6dd32d8, aTimeoutMS=10000,
    aConnectTimeoutMS=10000, aContentType=0x0, aHeaderFields=0x0)
    at <<redacted>>/HTTPClient.cpp:115
<<snip>>

The crash happens in the cleanup after what appears to be an otherwise
successful HTTP GET request.
It looks like something in Curl's state has become corrupted somewhere:

(gdb) list
3021 /* flush the timeout list too */
3022 while(list->size > 0)
<--- crash is here
3023 Curl_llist_remove(list, list->tail, NULL);
(gdb) print list
$9 = (struct curl_llist *) 0x0
(gdb) print data->state
$11 = {conn_cache = 0xb3c19bf4, multi_owned_by_easy = false, keeps_speed = {
    tv_sec = 0, tv_usec = 0}, lastconnect = 0x0,
  headerbuff = 0xb3c19e4c
"�\002��@\236��@\236��20375\t66700\t2\t2\t2\t0\t0\t0\t0\t0\t0\t4\t14\t3\t0\t0\t0\t0\n1481544716015001\t1\t2\t4\t194\t9717\t20373\t59900\t4\t4\t4\t0\t0\t0\t0\t0\t0\t5\t12\t2\t0\t0\t0\t0\n1481544716015002\t1\t3\t7\t194\t9717\t20374\t57900\t4\t4\t4\t0\t0\t0\t0\t0\t0\t5\t2\t"...,
  headersize = 256, buffer = '\0' <repeats 16384 times>,
  uploadbuffer = '\0' <repeats 16384 times>, current_speed = -1,
  this_is_a_follow = false, first_host = 0x0, first_remote_port = 0,
  session = 0x0, sessionage = 0, tempwrite = 0x0, tempwritesize = 0,
  tempwritetype = 0, scratch = 0x0, errorbuf = false, os_errno = 0,
  prev_signal = 0, allow_port = false, digest = {nonce = 0x0, cnonce = 0x0,
    realm = 0x0, algo = 0, stale = false, opaque = 0x0, qop = 0x0,
    algorithm = 0x0, nc = 0}, proxydigest = {nonce = 0x0, cnonce = 0x0,
    realm = 0x0, algo = 0, stale = false, opaque = 0x0, qop = 0x0,
    algorithm = 0x0, nc = 0}, authhost = {want = 0, picked = 0, avail = 0,
    done = false, multi = false, iestyle = false}, authproxy = {want = 0,
    picked = 0, avail = 0, done = false, multi = false, iestyle = false},
  authproblem = false, resolver = 0x0, expiretime = {tv_sec = 0,
    tv_usec = -1}, timenode = {smaller = 0xffffffff, larger = 0x0, same = 0x0,
    key = {tv_sec = 0, tv_usec = 0}, payload = 0x0}, timeoutlist = 0x0,
  most_recent_ftp_entrypath = 0x0, ftp_trying_alternative = false,
  httpversion = 0, expect100header = false, pipe_broke = false,
---Type <return> to continue, or q <return> to quit---
  prev_block_had_trailing_cr = false, crlf_conversions = 0, pathbuffer = 0x0,
  path = 0x0, slash_removed = false, use_range = false,
  rangestringalloc = false, range = 0x0, resume_from = 0,
  rtsp_next_client_CSeq = 0, rtsp_next_server_CSeq = 0, rtsp_CSeq_recv = 0,
  infilesize = 0, drain = 0, done = false, fread_func = 0, in = 0x0,
  stream_depends_on = 0x0, stream_depends_e = false, stream_weight = 0}

From my reading of the code I think tv_usec and timenode.smaller are
not supposed to be 0xffffffff. The list of timeouts is allowed to be
NULL, but only if expiretime.tv_sec and expiretime.tv_usec are both 0.
That is the immediate cause of the segfault. But I can't figure out
why expiretime.tv_usec and timenode.smaller are set to invalid values.

Of course adding a NULL guard around that list dereference on line
3022 would fix the immediate crash but that doesn't fix the real
issue.

The code that calls libcurl is pretty much a copy-paste of libcurl
example code, but I can post a (slightly redacted) version if it's
needed.

This is libcurl version 7.51.0 (but it also happened in libcurl
7.50.3) running on a Debian system and it is configured as follows:

./configure --enable-threaded-resolver --without-ssl
--with-mbedtls=<<redacted>>/local --disable-dict --disable-gopher
--disable-imap --disable-ldap --disable-pop3 --disable-rtsp
--disable-smb --disable-smtp --disable-telnet
--prefix=<<redacted>>/local --disable-shared --enable-debug

Any help in tracking down this segfault would be greatly appreciated!

Kind regards,
Theo

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-12-14