On Wed, 22 May 2013 21:14:55 +0100 Ben Thomas wrote:
>On Wed, May 22, 2013 at 6:49 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>> A) use really larger buffers, like several hundred KBs at least
>>
>> and
>>
>> B) use a sliding buffer technique instead of letting the existing buffer
>> drain completely before it gets refilled
>>
> So is it possible for me to do A and B using cURL?

You can do A) by compiling a custom libcurl configured with something like
CPPFLAGS=-DCURL_MAX_WRITE_SIZE=1048576 That will change the default 16 KiB
buffer size to 1 MiB, which will kill your memory footprint but should speed up
your transfers. libcurl isn't expecting such a large buffer so it isn't
particularly careful about using it efficiently, but I've tried setting it as
high as 32 MiB and it still passes the test suite (and takes 132 MiB of RAM
to download a 200 byte file over HTTP!)

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-05-24