cURL / Mailing Lists / curl-library / Single Mail

curl-library

Mass test failures on OS X

From: Daniel Johnson <daniel.johnson31_at_gmail.com>
Date: Fri, 24 Jul 2009 21:05:34 -0400

I just did a libcurl build that failed spectacularly. I noticed that
something was wrong when postfix refused to mail the build log because
it was too big: 60 MB! Almost every test failed with a memory leak
error. Here's test 001:
====================

test 001...[HTTP GET]

** MEMORY FAILURE
Leak detected: memory still allocated: 64 bytes
At 102e564, there's 64 bytes.
  allocated by ../../curl/lib/mprintf.c:1080
== Contents of files in the log/ dir after test 1
=== Start of file curl.log
  ../libtool --mode=execute /sw/bin/valgrind --tool=memcheck --leak-
check=yes --num-callers=16 --log-file=log/valgrind1 ../src/curl --
output log/curl1.out --include --verbose --trace-time http://127.0.0.1:8990/1
>>log/stdout1 2>>log/stderr1
=== End of file curl.log
=== Start of file curl1.out
  HTTP/1.1 200 OK
  Date: Thu, 09 Nov 2010 14:49:00 GMT
  Server: test-server/fake
  Last-Modified: Tue, 13 Jun 2000 12:10:00 GMT
  ETag: "21025-dc7-39462498"
  Accept-Ranges: bytes
  Content-Length: 6
  Connection: close
  Content-Type: text/html
  Funny-head: yesyes
  -foo-
=== End of file curl1.out
=== Start of file curlverout.log
  curl 7.19.6-CVS (i386-apple-darwin9.7.0) libcurl/7.19.6-CVS OpenSSL/
0.9.7l zlib/1.2.3 c-ares/1.6.1-CVS libidn/1.13 libssh2/1.1
  Protocols: tftp ftp telnet dict ldap http file https ftps scp sftp
  Features: AsynchDNS Debug TrackMemory GSS-Negotiate IDN IPv6
Largefile NTLM SSL libz
=== End of file curlverout.log
=== Start of file server.input
  GET /1 HTTP/1.1
  User-Agent: curl/7.19.6-CVS (i386-apple-darwin9.7.0) libcurl/7.19.6-
CVS OpenSSL/0.9.7l zlib/1.2.3 c-ares/1.6.1-CVS libidn/1.13 libssh2/1.1
  Host: 127.0.0.1:8990
  Accept: */*
=== End of file server.input
=== Start of file server.response
  HTTP/1.1 200 OK
  Content-Length: 17
  WE ROOLZ: 55604
  HTTP/1.1 200 OK
  Date: Thu, 09 Nov 2010 14:49:00 GMT
  Server: test-server/fake
  Last-Modified: Tue, 13 Jun 2000 12:10:00 GMT
  ETag: "21025-dc7-39462498"
  Accept-Ranges: bytes
  Content-Length: 6
  Connection: close
  Content-Type: text/html
  Funny-head: yesyes
  -foo-
=== End of file server.response
=== Start of file stderr1
  20:02:15.159482 * About to connect() to 127.0.0.1 port 8990 (#0)
  20:02:15.223691 * Trying 127.0.0.1... connected
  20:02:15.241542 * Connected to 127.0.0.1 (127.0.0.1) port 8990 (#0)
  20:02:15.270245 > GET /1 HTTP/1.1
  20:02:15.270245 > User-Agent: curl/7.19.6-CVS (i386-apple-
darwin9.7.0) libcurl/7.19.6-CVS OpenSSL/0.9.7l zlib/1.2.3 c-ares/1.6.1-
CVS libidn/1.13 libssh2/1.1
  20:02:15.270245 > Host: 127.0.0.1:8990
  20:02:15.270245 > Accept: */*
  20:02:15.270245 >
  20:02:15.302513 * HTTP 1.1 or later with persistent connection,
pipelining supported
  20:02:15.308143 < HTTP/1.1 200 OK
  20:02:15.313568 < Date: Thu, 09 Nov 2010 14:49:00 GMT
  20:02:15.314123 < Server: test-server/fake
  20:02:15.315266 < Last-Modified: Tue, 13 Jun 2000 12:10:00 GMT
  20:02:15.315756 < ETag: "21025-dc7-39462498"
  20:02:15.316238 < Accept-Ranges: bytes
  20:02:15.322244 < Content-Length: 6
  20:02:15.325107 < Connection: close
  20:02:15.328659 < Content-Type: text/html
  20:02:15.329150 < Funny-head: yesyes
  20:02:15.334901 <
  20:02:15.336890 { [data not shown]
    % Total % Received % Xferd Average Speed Time Time
Time Current
                                   Dload Upload Total Spent
Left Speed

    0 6 0 6 0 0 24 0 --:--:-- --:--:--
--:--:-- 24
    0 6 0 6 0 0 21 0 --:--:-- --:--:--
--:--:-- 020:02:15.389642 * Closing connection #0
=== End of file stderr1
=== Start of file sws.log
  20:02:10.929985 Wrote pid 55604 to .http.pid
  20:02:10.930586 Running IPv4 version on port 8990
  20:02:11.946439 ====> Client connect
  20:02:11.946597 Read 207 bytes
  20:02:11.946863 ProcessRequest() called
  20:02:11.948433 Got request: GET /verifiedserver HTTP/1.1
  20:02:11.949193 Are-we-friendly question received
  20:02:11.949814 Wrote request (207 bytes) input to log/server.input
  20:02:11.950038 Send response number -2 part 0
  20:02:11.950432 Identifying ourselves as friends
  20:02:11.951068 Sent off 56 bytes
  20:02:11.951496 Response sent (56 bytes) and written to log/
server.response
  20:02:11.951846 special request received, no persistency
  20:02:11.952416 ====> Client disconnect
  20:02:15.230924 ====> Client connect
  20:02:15.268961 Read 194 bytes
  20:02:15.269049 ProcessRequest() called
  20:02:15.269359 Got request: GET /1 HTTP/1.1
  20:02:15.269680 Requested test number 1 part 0
  20:02:15.270434 ProcessRequest found a complete request
  20:02:15.270938 Wrote request (194 bytes) input to log/server.input
  20:02:15.271250 Send response number 1 part 0
  20:02:15.271731 Sent off 200 bytes
  20:02:15.272038 Sent off 56 bytes
  20:02:15.272456 Response sent (256 bytes) and written to log/
server.response
  20:02:15.272722 => persistant connection request ended, awaits new
request
  20:02:15.392338 Connection closed by client
  20:02:15.392429 ====> Client disconnect
=== End of file sws.log
=== Start of file valgrind1
  ==55623== Memcheck, a memory error detector.
  ==55623== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward
et al.
  ==55623== Using LibVEX rev exported, a library for dynamic binary
translation.
  ==55623== Copyright (C) 2004-2009, and GNU GPL'd, by OpenWorks LLP.
  ==55623== Using valgrind-3.5.0.SVN, a dynamic binary instrumentation
framework.
  ==55623== Copyright (C) 2000-2009, and GNU GPL'd, by Julian Seward
et al.
  ==55623== For more details, rerun with: -v
  ==55623==
  ==55623== My PID = 55623, parent PID = 55622. Prog and args are:
  ==55623== /build-4775/src/.libs/curl
  ==55623== --output
  ==55623== log/curl1.out
  ==55623== --include
  ==55623== --verbose
  ==55623== --trace-time
  ==55623== http://127.0.0.1:8990/1
  ==55623==
  --55623-- /build-4775/src/.libs/curl:
  --55623-- dSYM directory is missing; consider using --auto-run-
dsymutil=yes
  --55623-- /sw/lib/libidn.11.dylib:
  --55623-- dSYM directory is missing; consider using --auto-run-
dsymutil=yes
  --55623-- /deps/lib/libssh2.1.0.1.dylib:
  --55623-- dSYM directory is missing; consider using --auto-run-
dsymutil=yes
  --55623-- /sw/lib/libintl.3.4.3.dylib:
  --55623-- dSYM directory is missing; consider using --auto-run-
dsymutil=yes
  --55623-- /sw/lib/libiconv.2.4.0.dylib:
  --55623-- dSYM directory is missing; consider using --auto-run-
dsymutil=yes
  ==55623==
  ==55623== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0
from 0)
  ==55623== malloc/free: in use at exit: 63,083 bytes in 211 blocks.
  ==55623== malloc/free: 2,209 allocs, 1,998 frees, 315,841 bytes
allocated.
  ==55623== For counts of detected errors, rerun with: -v
  ==55623== searching for pointers to 211 not-freed blocks.
  ==55623== checked 2,188,512 bytes.
  ==55623==
  ==55623== 76 bytes in 1 blocks are definitely lost in loss record 25
of 46
  ==55623== at 0x2E2B8: realloc (vg_replace_malloc.c:472)
  ==55623== by 0x9F981: curl_dorealloc (memdebug.c:213)
  ==55623== by 0x95072: alloc_addbyter (mprintf.c:1080)
  ==55623== by 0x948A8: dprintf_formatf (mprintf.c:885)
  ==55623== by 0x95129: curl_maprintf (mprintf.c:1109)
  ==55623== by 0x9D3D: operate (in /build-4775/src/.libs/curl)
  ==55623== by 0xB821: main (in /build-4775/src/.libs/curl)
  ==55623==
  ==55623== LEAK SUMMARY:
  ==55623== definitely lost: 76 bytes in 1 blocks.
  ==55623== indirectly lost: 0 bytes in 0 blocks.
  ==55623== possibly lost: 0 bytes in 0 blocks.
  ==55623== still reachable: 63,007 bytes in 210 blocks.
  ==55623== suppressed: 0 bytes in 0 blocks.
  ==55623== Reachable blocks (those to which a pointer was found) are
not shown.
  ==55623== To see them, rerun with: --leak-check=full --show-
reachable=yes
=== End of file valgrind1
=== Start of file verifiedserver
  WE ROOLZ: 55604
=== End of file verifiedserver
=== Start of file verifyhttp
  * About to connect() to 127.0.0.1 port 8990 (#0)
  * Trying 127.0.0.1... connected
  * Connected to 127.0.0.1 (127.0.0.1) port 8990 (#0)
> GET /verifiedserver HTTP/1.1
> User-Agent: curl/7.19.6-CVS (i386-apple-darwin9.7.0) libcurl/
7.19.6-CVS OpenSSL/0.9.7l zlib/1.2.3 c-ares/1.6.1-CVS libidn/1.13
libssh2/1.1
> Host: 127.0.0.1:8990
> Accept: */*
>
  * HTTP 1.1 or later with persistent connection, pipelining supported
  < HTTP/1.1 200 OK
  < Content-Length: 17
  <
  { [data not shown]
  * Connection #0 to host 127.0.0.1 left intact
  * Closing connection #0
=== End of file verifyhttp

====================

I think it might be because of this change:

===================================================================
RCS file: /cvsroot/curl/curl/src/main.c,v
retrieving revision 1.527
retrieving revision 1.528
diff -u -r1.527 -r1.528
--- curl/src/main.c 2009/07/17 22:34:16 1.527
+++ curl/src/main.c 2009/07/22 22:49:02 1.528
@@ -18,7 +18,7 @@
   * This software is distributed on an "AS IS" basis, WITHOUT
WARRANTY OF ANY
   * KIND, either express or implied.
   *
- * $Id: main.c,v 1.527 2009/07/17 22:34:16 gknauf Exp $
+ * $Id: main.c,v 1.528 2009/07/22 22:49:02 bagder Exp $
    
***************************************************************************/
  #include "setup.h"

@@ -4694,6 +4694,16 @@
            my_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
            my_setopt(curl, CURLOPT_SSL_VERIFYHOST, 1);
          }
+ else {
+ char *home = homedir();
+ char *file = aprintf("%s/%sssh/known_hosts", home, DOT_CHAR);
+ if(home && file) {
+ free(home);
+ my_setopt_str(curl, CURLOPT_SSH_KNOWNHOSTS, file);
+ }
+ else
+ return CURLE_OUT_OF_MEMORY;
+ }

          if(config->no_body || config->remote_time) {
            /* no body or use remote time */

I don't think 'file' ever gets freed.

Daniel

  • application/pkcs7-signature attachment: smime.p7s
Received on 2009-07-25