cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Curl HTTPS memory usage

From: Weng Xuetian <wengxt_at_gmail.com>
Date: Sun, 6 May 2012 20:21:53 +0800

On Sun, May 6, 2012 at 7:59 PM, Andrei Cipu <acipu_at_ixiacom.com> wrote:
> Hi Weng,
>
> Even without knowing anything about your app, it does make sense for HTTPS to use more memory than HTTP, since you have the TLS connection on top of what HTTP usually does. The TLS connection will need memory for encrypting/decrypting packets, remembering the server certificates etc.
>
>> curlm is set to use most 10 connection, seems more connection actually used, memory will be higher, and
>> finally it takes me about 26MB memory when memory usage is finally stable on a 64-bit Linux.
>
> Now, about the actual values you see, I'm guessing 26MB is the memory consumption of your whole application. It does seem a bit high for just 10 connections. I've been able to run ~750 concurrent connection with only 600 MB taken by the application (which does much more that HTTPS connections), but the overhead might decrease with the increase in the number of connections.
>
> Perhaps you want to test the memory usage with a minimal application that does nothing else except the actual HTTPS requests?
>
> Regards,
>   Andrei
Thank you for your reply!

Actually the service I request also provides an http interface, if I
simply replace https -> http, it will use nearly no more than 1MB.

So it seems all extra usage comes from https.

I seems finally found out what cause this.

I have change the malloc option.
    int pagesize = sysconf(_SC_PAGESIZE);
    mallopt(M_TRIM_THRESHOLD, 5 * pagesize);

After remove it now it looks like only 8M :D

Thank you!

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-05-06