cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: segfault with gzip encoding support

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Mon, 22 Dec 2003 21:36:41 -0800

On Mon, Dec 22, 2003 at 04:54:43PM -0800, Vikram wrote:
> We have been having problems with using multiple threads and downloading
> websites with 'gzip' content-encoding support. A backtrace from gdb is
> pasted below. This happens on a FreeBSD machine [4.8-release, 2gb ram].
>
> I then wrote a simple python program which does nothing else except create
> threads and each thread uses pycurl which uses curl-7.10.8 and does a
> simple get with encoding set to 'gzip'. The program segfaults. (zlib
> version 1.1.4)
>
> I am not sure whether the problem is with libcurl or with pycurl. Have
> there been any other problems reported with gzip encoding support and
> multiple threads (esp on freebsd machines)? Any suggestions ? Thanks.

It's unlikely to be a problem with pycurl--the backtrace goes deep into the
bowels of libcurl.

>
> Vikram
>
> -----------------------------------------------------------------
> first backtrace:
>
> #0 0x884eb302 in zmemcpy (
> dest=0xbfaedd6c ' ' <repeats 13 times>, "<html> <head>
> <title>Welcome to AdWords</title> <link href=\"AdsLater.css\"
> rel=\"stylesheet\"> <style type=\"text/css\"><!--\n\n
> div.buttonborder {\n width: auto;\n border: 2"...,
> source=0x8503000 ' ' <repeats 13 times>, "<html> <head>
> <title>Welcome to AdWords</title> <link href=\"AdsLater.css\"
> rel=\"stylesheet\"> <style type=\"text/css\"><!--\n\n
> div.buttonborder {\n width: auto;\n border: 2"..., len=5074)
> at zutil.c:68
> #1 0x884ee5cd in inflate_flush (s=0x8471900, z=0x83c93bc, r=0) at
> infutil.c:50
> #2 0x884ee46a in inflate_codes (s=0x8471900, z=0x83c93bc, r=0) at
> infcodes.c:237
> #3 0x884ecd55 in inflate_blocks (s=0x8471900, z=0x83c93bc, r=0) at
> infblock.c:340
> #4 0x884eb908 in inflate (z=0x83c93bc, f=2) at inflate.c:221
> #5 0x883beb2b in Curl_unencode_gzip_write (data=0x8356000, k=0x83c9358,
> nread=1572) at content_encoding.c:326
> #6 0x883bb300 in Curl_readwrite (conn=0x83c9000, done=0xbfafdf23 "") at
> transfer.c:990
> #7 0x883bbe04 in Transfer (conn=0x83c9000) at transfer.c:1401
> #8 0x883bc563 in Curl_perform (data=0x8356000) at transfer.c:1875
> #9 0x883bc8d9 in curl_easy_perform (curl=0x8356000) at easy.c:261
> #10 0x88397953 in do_curl_perform (self=0x833c40c, args=0x815f02c) at
> pycurl.c:651
> #11 0x80f7344 in PyCFunction_Call ()
> #12 0x80b2cb2 in call_function ()
> #13 0x80b017c in eval_frame ()
> #14 0x80b16f1 in PyEval_EvalCodeEx ()

This first backtrace is the interesting one, although it doesn't completely
make sense to me at first glance. It appears that the zmemcpy is overwriting
an internal libcurl buffer which is allocated on the stack. My first guess
would be that your thread's stack isn't large enough; that buffer is 64 KiB
long, so your thread's stack has to be considerably larger than that.

That buffer started out at 8 KiB in an earlier version if libcurl, which
is a reasonable size to allocate on the stack, but had to be increased
to 64 KiB when 8 proved to be inadequate. It should probably be changed
to be allocated on the heap, but it's difficult to avoid a memory leak
when an error occurs that way.

I also noticed that a few zlib fields aren't being explicitly initialized
within libcurl. They're probably zeroed out elsewhere, but it's safer
to do it anyway; the attached patch does so. There's a slight but
non-zero chance this might be related to your problem.

> -----------------------------------------------------------------------------
> second backtrace:
>
>
> #0 0x88383915 in Curl_unencode_gzip_write (data=0x82d8000, k=0x8284b58,
> nread=1294) at content_encoding.c:210

This line doesn't have any associated code. It may be an anomaly due
to a corrupted stack.

> #1 0x88380300 in Curl_readwrite (conn=0x8284800, done=0xbfafe843 "") at
> transfer.c:990
> #2 0x88380e04 in Transfer (conn=0x8284800) at transfer.c:1401
> #3 0x88381563 in Curl_perform (data=0x82d8000) at transfer.c:1875
> #4 0x883818d9 in curl_easy_perform (curl=0x82d8000) at easy.c:261
> #5 0x8835c953 in do_curl_perform (self=0x82cfc0c, args=0x815f02c) at
> pycurl.c:651
> #6 0x80f7344 in PyCFunction_Call ()
> #7 0x80b2cb2 in call_function ()
> #8 0x80b017c in eval_frame ()
> #9 0x80b16f1 in PyEval_EvalCodeEx ()
> #10 0x80b2fcd in fast_function ()
> #11 0x80b2d85 in call_function ()
> #12 0x80b017c in eval_frame ()
> #13 0x80b16f1 in PyEval_EvalCodeEx ()
> #14 0x80b2fcd in fast_function ()
> #15 0x80b2d85 in call_function ()
> #16 0x80b017c in eval_frame ()
> #17 0x80b16f1 in PyEval_EvalCodeEx ()
> #18 0x80f6c5e in function_call ()
> #19 0x805ace7 in PyObject_Call ()
> #20 0x80b2950 in PyEval_CallObjectWithKeywords ()
> #21 0x80daf37 in t_bootstrap ()
> #22 0x8816a0a8 in _thread_start () from /usr/lib/libc_r.so.4

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved

-------------------------------------------------------
This SF.net email is sponsored by: IBM Linux Tutorials.
Become an expert in LINUX or just sharpen your skills. Sign up for IBM's
Free Linux Tutorials. Learn everything from the bash shell to sys admin.
Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click

Received on 2003-12-23