cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: blocking on dns resolving

From: Pedro Larroy <pedro.larroy.lists_at_gmail.com>
Date: Sun, 18 Dec 2011 20:06:05 +0100

Sorry the formatting got broken, here's the attached backtrace in text.

On Sun, Dec 18, 2011 at 8:04 PM, Pedro Larroy
<pedro.larroy.lists_at_gmail.com> wrote:
> This is what i mean:
>
> (gdb) bt#0  0x00007ffff5aad6d0 in poll () from
> /lib/x86_64-linux-gnu/libc.so.6#1  0x0000000000497505 in Curl_poll
> (ufds=0x7fffffffd1c0, nfds=1, timeout_ms=0) at select.c:371#2
> 0x000000000049a57f in waitperform (conn=0xa05df8, timeout_ms=0) at
> asyn-ares.c:287#3  0x000000000049a67f in Curl_resolver_is_resolved
> (conn=0xa05df8, dns=0x7fffffffd328) at asyn-ares.c:324#4
> 0x000000000049b80f in Curl_resolv (conn=0xa05df8, hostname=0xa06418
> "romavictrix.blogspot.com", port=80, entry=0x7fffffffd3e0) at
> hostip.c:463#5  0x000000000049b915 in Curl_resolv_timeout
> (conn=0xa05df8, hostname=0xa06418 "romavictrix.blogspot.com", port=80,
> entry=0x7fffffffd3e0, timeoutms=60000) at hostip.c:628#6
> 0x00000000004b1bed in resolve_server (data=0x97b8e8, conn=0xa05df8,
> async=0x7fffffffd726) at url.c:4583#7  0x00000000004b2d37 in
> create_conn (data=0x97b8e8, in_connect=0xa05510, async=0x7fffffffd726)
> at url.c:5066#8  0x00000000004b2feb in Curl_connect (data=0x97b8e8,
> in_connect=0xa05510, asyncp=0x7fffffffd726,
> protocol_done=0x7fffffffd725) at url.c:5197#9  0x0000000000492db1 in
> multi_runsingle (multi=0x74def8, now=..., easy=0xa054f8) at
> multi.c:1045#10 0x0000000000495023 in multi_socket (multi=0x74def8,
> checkall=false, s=-1, ev_bitmask=0, running_handles=0x7fffffffdc24) at
> multi.c:2194#11 0x0000000000495480 in curl_multi_socket_action
> (multi_handle=0x74def8, s=-1, ev_bitmask=0,
> running_handles=0x7fffffffdc24) at multi.c:2274#12 0x0000000000470cfb
> in timer_cb (fd=-1, kind=1, userp=0x7fffffffd9d0) at
> src/crawler.cc:767#13 0x00007ffff704f97c in event_base_loop () from
> /usr/lib/libevent-2.0.so.5#14 0x00000000004737da in main (argc=1,
> argv=0x7fffffffdec8) at src/crawler.cc:1133
>
> Curl was built with:
>
>        '--disable-ldap',
>        '--without-libssh2',
>        '--without-librtmp',
>        '--enable-debug',
>        '--enable-ares={0}'.format(os.path.join(old_dir,
> '3rd_party','c-ares_install')),
>        '--prefix={0}'.format(os.path.join(old_dir, '3rd_party',
> 'curl_install')),
>
>
> On Sun, Dec 18, 2011 at 7:34 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>> On Sun, 18 Dec 2011, Pedro Larroy wrote:
>>
>>> After some flustration with blocking on dns requests while using the multi
>>> interface with libevent (https://github.com/larroy/mycelium) I investigated
>>> how curl uses ares and I came to the conclusion that the design of
>>> asyn-ares.c is very limiting.
>>
>>
>> Can you elaborate on when exactly it is "very limiting" ?
>>
>>
>>> What's the point of having all the goodies of the multi interface to use
>>> epoll etc, when waitperform function in asyn-ares.c just polls waiting for
>>> DNS queries to finish?
>>
>>
>> It doesn't. The multi interface tells your app about the socket c-ares uses
>> and your app waits for actions on that socket before it tells libcurl to
>> work on it.
>>
>>
>>> I think the proper way would be to expose the resolving sockets up to the
>>> multi interface so the whole system is really asynchronous. Is there any
>>> reason why it wasn't done this way?
>>
>>
>> It _is_ done like that. There are just a few exceptions to that rule. They
>> are documented and will hopefully all be fixed one by one until gone one day
>> in the future.
>>
>> --
>>
>>  / daniel.haxx.se
>>
>> -------------------------------------------------------------------
>> List admin: http://cool.haxx.se/list/listinfo/curl-library
>> Etiquette:  http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html

  • text/plain attachment: bt.txt
Received on 2011-12-18