curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Memory consumption of DNS cache when network is down?

From: Chris Carlmar via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 4 Feb 2019 16:37:51 +0100

On 04.02.2019 15:41, Ricardo Mota via curl-library wrote:
> Hello Daniel, thank you for your reply.
>
> This is the build configuration I've used for an ARMv6 linux embedded
> target:
>
> /Host setup:       arm-none-linux-gnueabi/
>
> /  Compiler:         arm-none-linux-gnueabi-gcc/
>
> /   CFLAGS:          -Werror-implicit-function-declaration -O2
> -Wno-system-headers -pthread/
>
> /   LIBS:            -lssl -lcrypto -lz -lrt/
>
> /
> /
>
> /  curl version:     7.64.0-DEV/
>
> /  SSL support:      enabled (OpenSSL)/
>
> /  SSH support:      no      (--with-libssh2)/
>
> /  zlib support:     enabled/
>
> /  brotli support:   no      (--with-brotli)/
>
> /  GSS-API support:  no      (--with-gssapi)/
>
> /  TLS-SRP support:  enabled/
>
> /  resolver:         POSIX threaded/
>
> /  IPv6 support:     enabled/
>
> /  Unix sockets support: enabled/
>
> /  IDN support:      no      (--with-{libidn2,winidn})/
>
> /  Build libcurl:    Shared=no, Static=yes/
>
> /  Built-in manual:  enabled/
>
> /  --libcurl option: enabled (--disable-libcurl-option)/
>
> /  Verbose errors:   enabled (--disable-verbose)/
>
> /  SSPI support:     no      (--enable-sspi)/
>
> /  ca cert bundle:   no/
>
> /  ca cert path:     no/
>
> /  ca fallback:      no/
>
> /  LDAP support:     no      (--enable-ldap / --with-ldap-lib /
> --with-lber-lib)/
>
> /  LDAPS support:    no      (--enable-ldaps)/
>
> /  RTSP support:     enabled/
>
> /  RTMP support:     no      (--with-librtmp)/
>
> /  metalink support: no      (--with-libmetalink)/
>
> /  PSL support:      no      (libpsl not found)/
>
> /  HTTP2 support:    disabled (--with-nghttp2)/
>
> /  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP
> IMAPS POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP/
>
>
> I've used the following /"wrapper"
> (https://github.com/mrtazz/restclient-cpp)/ for /libcurl/ just doing the
> initialization of an object /connection /and querying a simple REST service.
>
> I've already tried compiling without /zlib/ or /SSL /support, thinking
> it could be a problem related with ssl library version I was using, but
> the problem persisted.
>
> I can't get you an exact value for the memory consumption because I
> don't have many debug tools for the target system but I can see the
> memory increasing 4KB ~ 10 sec. The only thing I noticed is that if I
> use direct IP for the connection (without the name resolution), the
> problem goes away, and the memory only seems to increase when the
> network is down and the connection fails to resolve the name:
>
> /Could not resolve host: abc.com <http://abc.com/>/
>
> /* Closing connection XXXX/
>
>
> I've already tried with different /7.xx /versions of /libcurl /but the
> behavior is the same in all.
>
> Any other information I can provide to help debug the issue?
>
>
>
> On Mon, Feb 4, 2019 at 1:25 PM Daniel Stenberg <daniel_at_haxx.se
> <mailto:daniel_at_haxx.se>> wrote:
>
> On Mon, 4 Feb 2019, Ricardo Mota via curl-library wrote:
>
> > I'm experiencing a problem using *libcurl *due to a memory
> consumption
> > increase in my application which queries a simple REST API in a loop
> > reusing the same handle. Doing some tests, I only notice this memory
> > increase when I'm using name resolution and specifically when the
> network
> > is down. I've already tried to disable DNS cache by setting
> > *CURLOPT_DNS_CACHE_TIMEOUT
> > *value to 0 but the problem seems to persist. Can somebody please
> help me
> > debugging this problem?
>
> What level of memory consumption increase are we talking about?
>
> What libcurl version on what platform?
>
> I don't think what you're describing is a particularly strong
> indication that
> the increase is indeed the DNS cache, and in fact since the cache is
> used to
> store name to address data and you say you use it in a loop so
> surely you just
> store the same name over and over (or even just reuse the cached
> content)?
>
> so... how can we reproduce this issue?
>
> --
>
>   / daniel.haxx.se <http://daniel.haxx.se>
>
>

I had a simular issue using libcurl on an embedded system, leaking when
DNS could not be reached. It turned out to be a memory leak in the
version of glibc, which I could not change.

Ended up changing to c-ares, and that stopped the leaking.

--
mvh
Chris Carlmar
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2019-02-04