cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Memory consumption during large uploads..

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Wed, 8 Feb 2012 14:36:50 +0100

On Wed, Feb 08, 2012 at 06:32:46PM +0530, sukanya c wrote:
> I work for activesync application that use libcurl for client HTTP operations.
>  
> I noticed that during 10MB data upload, the peak memory consumption reaches
> more than 10MB.

Have you tracked the source of this memory usage? Are you sure it's not
caused by the application?

> Is it possible to send data to network in the form of chunks and discard the
> chunks that are already posted to network. This can reduce the peak memory
> consumption on the client during uploads.

That's already how libcurl works. Once it sends data, it's not kept
around any longer. If libcurl needs the data again, it will call the
CURLOPT_SEEKFUNCTION function rather than keeping it around.

> Please suggest any other better methods that I can use to reduce the memory
> consumption for clients during large data upload.

Use valgrind or another memory usage tracker to find out exactly where
that memory is going. In the unlikely event that it turns out to be in libcurl,
then we here would certainly be interested in knowing.

>>> Dan

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-02-08