curl-library
Re: Memory consumption of DNS cache when network is down?
Date: Mon, 4 Feb 2019 16:37:51 +0100
On 04.02.2019 15:41, Ricardo Mota via curl-library wrote:
> Hello Daniel, thank you for your reply.
>
> This is the build configuration I've used for an ARMv6 linux embedded
> target:
>
> /Host setup: arm-none-linux-gnueabi/
>
> / Compiler: arm-none-linux-gnueabi-gcc/
>
> / CFLAGS: -Werror-implicit-function-declaration -O2
> -Wno-system-headers -pthread/
>
> / LIBS: -lssl -lcrypto -lz -lrt/
>
> /
> /
>
> / curl version: 7.64.0-DEV/
>
> / SSL support: enabled (OpenSSL)/
>
> / SSH support: no (--with-libssh2)/
>
> / zlib support: enabled/
>
> / brotli support: no (--with-brotli)/
>
> / GSS-API support: no (--with-gssapi)/
>
> / TLS-SRP support: enabled/
>
> / resolver: POSIX threaded/
>
> / IPv6 support: enabled/
>
> / Unix sockets support: enabled/
>
> / IDN support: no (--with-{libidn2,winidn})/
>
> / Build libcurl: Shared=no, Static=yes/
>
> / Built-in manual: enabled/
>
> / --libcurl option: enabled (--disable-libcurl-option)/
>
> / Verbose errors: enabled (--disable-verbose)/
>
> / SSPI support: no (--enable-sspi)/
>
> / ca cert bundle: no/
>
> / ca cert path: no/
>
> / ca fallback: no/
>
> / LDAP support: no (--enable-ldap / --with-ldap-lib /
> --with-lber-lib)/
>
> / LDAPS support: no (--enable-ldaps)/
>
> / RTSP support: enabled/
>
> / RTMP support: no (--with-librtmp)/
>
> / metalink support: no (--with-libmetalink)/
>
> / PSL support: no (libpsl not found)/
>
> / HTTP2 support: disabled (--with-nghttp2)/
>
> / Protocols: DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP
> IMAPS POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP/
>
>
> I've used the following /"wrapper"
> (https://github.com/mrtazz/restclient-cpp)/ for /libcurl/ just doing the
> initialization of an object /connection /and querying a simple REST service.
>
> I've already tried compiling without /zlib/ or /SSL /support, thinking
> it could be a problem related with ssl library version I was using, but
> the problem persisted.
>
> I can't get you an exact value for the memory consumption because I
> don't have many debug tools for the target system but I can see the
> memory increasing 4KB ~ 10 sec. The only thing I noticed is that if I
> use direct IP for the connection (without the name resolution), the
> problem goes away, and the memory only seems to increase when the
> network is down and the connection fails to resolve the name:
>
> /Could not resolve host: abc.com <http://abc.com/>/
>
> /* Closing connection XXXX/
>
>
> I've already tried with different /7.xx /versions of /libcurl /but the
> behavior is the same in all.
>
> Any other information I can provide to help debug the issue?
>
>
>
> On Mon, Feb 4, 2019 at 1:25 PM Daniel Stenberg <daniel_at_haxx.se
> <mailto:daniel_at_haxx.se>> wrote:
>
> On Mon, 4 Feb 2019, Ricardo Mota via curl-library wrote:
>
> > I'm experiencing a problem using *libcurl *due to a memory
> consumption
> > increase in my application which queries a simple REST API in a loop
> > reusing the same handle. Doing some tests, I only notice this memory
> > increase when I'm using name resolution and specifically when the
> network
> > is down. I've already tried to disable DNS cache by setting
> > *CURLOPT_DNS_CACHE_TIMEOUT
> > *value to 0 but the problem seems to persist. Can somebody please
> help me
> > debugging this problem?
>
> What level of memory consumption increase are we talking about?
>
> What libcurl version on what platform?
>
> I don't think what you're describing is a particularly strong
> indication that
> the increase is indeed the DNS cache, and in fact since the cache is
> used to
> store name to address data and you say you use it in a loop so
> surely you just
> store the same name over and over (or even just reuse the cached
> content)?
>
> so... how can we reproduce this issue?
>
> --
>
> / daniel.haxx.se <http://daniel.haxx.se>
>
>
I had a simular issue using libcurl on an embedded system, leaking when
DNS could not be reached. It turned out to be a memory leak in the
version of glibc, which I could not change.
Ended up changing to c-ares, and that stopped the leaking.
-- mvh Chris Carlmar ------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2019-02-04