Hi all!

I write web page crawler on PHP with multicurl, and strange bug
appears: my crawler works normally for 3-4 hours and then starts eat
100% of CPU and doing nothing.

I compiled libcurl and php with -g flag, run crawler and, when it
stops (starts eat 100% CPU), attached to it with GDB. The problem is
in function Curl_removeHandleFromPipeline (lib/url.c): pipeline size
== 0, but head and tail links != NULL, list has about 26 elements, and
they are connected as a ring. So curr->next never becames NULL and
while(curr) loop never ends.

Environment:
aaa_at_aaa:~$ uname -a
Linux aaa 2.6.26-2-amd64 #1 SMP Thu Nov 25 04:30:55 UTC 2010 x86_64 GNU/Linux
aaa_at_aaa:~$ curl --version
curl 7.18.2 (x86_64-pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.8g
zlib/1.2.3.3 libidn/1.8 libssh2/0.18
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

I compiled newest version of libcurl (7.22) and will try to catch this
bug in it some days later, but may be this is known bug and have been
fixed in some newer version?
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-10-08