curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Memory consumption of DNS cache when network is down?

From: Ricardo Mota via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 4 Feb 2019 14:41:02 +0000

Hello Daniel, thank you for your reply.

This is the build configuration I've used for an ARMv6 linux embedded
target:

*Host setup: arm-none-linux-gnueabi*
>
> * Compiler: arm-none-linux-gnueabi-gcc*
>
> * CFLAGS: -Werror-implicit-function-declaration -O2
>> -Wno-system-headers -pthread*
>
> * LIBS: -lssl -lcrypto -lz -lrt*
>
>
>> * curl version: 7.64.0-DEV*
>
> * SSL support: enabled (OpenSSL)*
>
> * SSH support: no (--with-libssh2)*
>
> * zlib support: enabled*
>
> * brotli support: no (--with-brotli)*
>
> * GSS-API support: no (--with-gssapi)*
>
> * TLS-SRP support: enabled*
>
> * resolver: POSIX threaded*
>
> * IPv6 support: enabled*
>
> * Unix sockets support: enabled*
>
> * IDN support: no (--with-{libidn2,winidn})*
>
> * Build libcurl: Shared=no, Static=yes*
>
> * Built-in manual: enabled*
>
> * --libcurl option: enabled (--disable-libcurl-option)*
>
> * Verbose errors: enabled (--disable-verbose)*
>
> * SSPI support: no (--enable-sspi)*
>
> * ca cert bundle: no*
>
> * ca cert path: no*
>
> * ca fallback: no*
>
> * LDAP support: no (--enable-ldap / --with-ldap-lib /
>> --with-lber-lib)*
>
> * LDAPS support: no (--enable-ldaps)*
>
> * RTSP support: enabled*
>
> * RTMP support: no (--with-librtmp)*
>
> * metalink support: no (--with-libmetalink)*
>
> * PSL support: no (libpsl not found)*
>
> * HTTP2 support: disabled (--with-nghttp2)*
>
> * Protocols: DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3
>> POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP*
>
>
I've used the following *"wrapper"
(https://github.com/mrtazz/restclient-cpp
<https://github.com/mrtazz/restclient-cpp>)* for *libcurl* just doing the
initialization of an object *connection *and querying a simple REST service.

I've already tried compiling without *zlib* or *SSL *support, thinking it
could be a problem related with ssl library version I was using, but the
problem persisted.

I can't get you an exact value for the memory consumption because I don't
have many debug tools for the target system but I can see the memory
increasing 4KB ~ 10 sec. The only thing I noticed is that if I use direct
IP for the connection (without the name resolution), the problem goes away,
and the memory only seems to increase when the network is down and the
connection fails to resolve the name:

*Could not resolve host: abc.com <http://abc.com/>*
>
> ** Closing connection XXXX*
>
>
I've already tried with different *7.xx *versions of *libcurl *but the
behavior is the same in all.

Any other information I can provide to help debug the issue?

On Mon, Feb 4, 2019 at 1:25 PM Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Mon, 4 Feb 2019, Ricardo Mota via curl-library wrote:
>
> > I'm experiencing a problem using *libcurl *due to a memory consumption
> > increase in my application which queries a simple REST API in a loop
> > reusing the same handle. Doing some tests, I only notice this memory
> > increase when I'm using name resolution and specifically when the network
> > is down. I've already tried to disable DNS cache by setting
> > *CURLOPT_DNS_CACHE_TIMEOUT
> > *value to 0 but the problem seems to persist. Can somebody please help me
> > debugging this problem?
>
> What level of memory consumption increase are we talking about?
>
> What libcurl version on what platform?
>
> I don't think what you're describing is a particularly strong indication
> that
> the increase is indeed the DNS cache, and in fact since the cache is used
> to
> store name to address data and you say you use it in a loop so surely you
> just
> store the same name over and over (or even just reuse the cached content)?
>
> so... how can we reproduce this issue?
>
> --
>
> / daniel.haxx.se
>

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-02-04