curl-library
Re: Memory when threading curl
Date: Mon, 29 Jun 2009 11:17:19 +0200 (CEST)
On Mon, 29 Jun 2009, John D wrote:
> The issue is how much ram curl it is requiring. I literally must leave the
> default stack size within debian at 8mb. So with 16gb of ram I am getting
> 1800-2000 threads. Any more and I will encounter a seg fault in relation to
> thread resources. A lower stack and it is curl's chance to seg fault.
I know people have used libcurl in more threads with less RAM with no
problems. Each easy handle allocates perhaps 100K on average. (And if we
wanted to we could work on shrinking that.)
> I just am trying to decide if I need to do something like take the
> Curl_BufferSize into my own hands? I know it does not leak memory but I do
> not for the life of me understand how a 100kb transaction needs an 8mb stack
> size?
I have no answer to that either. I know lots of embedded uses of libcurl and
they certainly have no 8MB around for stack. I know the stack usage surpasses
several K (as in kilobytes) at times, but I would be surprised if it even
reaches 100K and if it would I would start working on decreasing it.
libcurl serves a lot of embedded and tiny-memory-amount devices so I think
it's worth putting an effort at keeping the stack usage at a minimum level.
But if you get a crash, can't you just force a small stack size and see where
libcurl crashes and then backtrack from that?
-- / daniel.haxx.seReceived on 2009-06-29