On Tue, 23 May 2017, Prasanna Venkateswaran via curl-library wrote:

> I have written an application to post a json payload to a given REST API
> endpoint. The application uses libevent2 and multi curl to achieve
> asynchronous behavior. While testing I noticed that the heap memory usage of
> the process keeps increasing when multiple parallel POST requests are in
> progress and the memory usage never comes down even after the requests are
> complete.

Heap memory is often allocated from the system piece by piece but is not
returned again until the application exits - by design. (Unless the malloc()
is very large, for which some memory systems have an mmap exception - like
glibc does.) When the application needs more memory and malloc() can't return
enough memory from the heap it already has, it will request more from the
system. It'll eventually reach a "plateau" unless there's a leak.

> to narrow down the root cause of memory usage, I compiled and ran the
> hiperfifo example. I noticed that the heap memory usage increases even when
> using the sample code.

The explanation is simply that your program required a peak usage of 9MB
memory to perform all those parallel transfers. If you managed to get close to
200 parallel transfers, that's only 45KB per transfer.

If you reduce the amount of parallelism, you'll most likely also reduce the
maximum memory requirement.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html