curl-library
Re: Memory consumption of DNS cache when network is down?
Date: Mon, 4 Feb 2019 16:37:51 +0100
On 04.02.2019 15:41, Ricardo Mota via curl-library wrote:
> Hello Daniel, thank you for your reply.
> 
> This is the build configuration I've used for an ARMv6 linux embedded 
> target:
> 
>         /Host setup:       arm-none-linux-gnueabi/
> 
>         /  Compiler:         arm-none-linux-gnueabi-gcc/
> 
>         /   CFLAGS:          -Werror-implicit-function-declaration -O2
>         -Wno-system-headers -pthread/
> 
>         /   LIBS:            -lssl -lcrypto -lz -lrt/
> 
>         /
>         /
> 
>         /  curl version:     7.64.0-DEV/
> 
>         /  SSL support:      enabled (OpenSSL)/
> 
>         /  SSH support:      no      (--with-libssh2)/
> 
>         /  zlib support:     enabled/
> 
>         /  brotli support:   no      (--with-brotli)/
> 
>         /  GSS-API support:  no      (--with-gssapi)/
> 
>         /  TLS-SRP support:  enabled/
> 
>         /  resolver:         POSIX threaded/
> 
>         /  IPv6 support:     enabled/
> 
>         /  Unix sockets support: enabled/
> 
>         /  IDN support:      no      (--with-{libidn2,winidn})/
> 
>         /  Build libcurl:    Shared=no, Static=yes/
> 
>         /  Built-in manual:  enabled/
> 
>         /  --libcurl option: enabled (--disable-libcurl-option)/
> 
>         /  Verbose errors:   enabled (--disable-verbose)/
> 
>         /  SSPI support:     no      (--enable-sspi)/
> 
>         /  ca cert bundle:   no/
> 
>         /  ca cert path:     no/
> 
>         /  ca fallback:      no/
> 
>         /  LDAP support:     no      (--enable-ldap / --with-ldap-lib /
>         --with-lber-lib)/
> 
>         /  LDAPS support:    no      (--enable-ldaps)/
> 
>         /  RTSP support:     enabled/
> 
>         /  RTMP support:     no      (--with-librtmp)/
> 
>         /  metalink support: no      (--with-libmetalink)/
> 
>         /  PSL support:      no      (libpsl not found)/
> 
>         /  HTTP2 support:    disabled (--with-nghttp2)/
> 
>         /  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP
>         IMAPS POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP/
> 
> 
> I've used the following /"wrapper" 
> (https://github.com/mrtazz/restclient-cpp)/ for /libcurl/ just doing the 
> initialization of an object /connection /and querying a simple REST service.
> 
> I've already tried compiling without /zlib/ or /SSL /support, thinking 
> it could be a problem related with ssl library version I was using, but 
> the problem persisted.
> 
> I can't get you an exact value for the memory consumption because I 
> don't have many debug tools for the target system but I can see the 
> memory increasing 4KB ~ 10 sec. The only thing I noticed is that if I 
> use direct IP for the connection (without the name resolution), the 
> problem goes away, and the memory only seems to increase when the 
> network is down and the connection fails to resolve the name:
> 
>         /Could not resolve host: abc.com <http://abc.com/>/
> 
>         /* Closing connection XXXX/
> 
> 
> I've already tried with different /7.xx /versions of /libcurl /but the 
> behavior is the same in all.
> 
> Any other information I can provide to help debug the issue?
> 
> 
> 
> On Mon, Feb 4, 2019 at 1:25 PM Daniel Stenberg <daniel_at_haxx.se 
> <mailto:daniel_at_haxx.se>> wrote:
> 
>     On Mon, 4 Feb 2019, Ricardo Mota via curl-library wrote:
> 
>      > I'm experiencing a problem using *libcurl *due to a memory
>     consumption
>      > increase in my application which queries a simple REST API in a loop
>      > reusing the same handle. Doing some tests, I only notice this memory
>      > increase when I'm using name resolution and specifically when the
>     network
>      > is down. I've already tried to disable DNS cache by setting
>      > *CURLOPT_DNS_CACHE_TIMEOUT
>      > *value to 0 but the problem seems to persist. Can somebody please
>     help me
>      > debugging this problem?
> 
>     What level of memory consumption increase are we talking about?
> 
>     What libcurl version on what platform?
> 
>     I don't think what you're describing is a particularly strong
>     indication that
>     the increase is indeed the DNS cache, and in fact since the cache is
>     used to
>     store name to address data and you say you use it in a loop so
>     surely you just
>     store the same name over and over (or even just reuse the cached
>     content)?
> 
>     so... how can we reproduce this issue?
> 
>     -- 
> 
>        / daniel.haxx.se <http://daniel.haxx.se>
> 
> 
I had a simular issue using libcurl on an embedded system, leaking when 
DNS could not be reached. It turned out to be a memory leak in the 
version of glibc, which I could not change.
Ended up changing to c-ares, and that stopped the leaking.
-- mvh Chris Carlmar ------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2019-02-04