cURL / Mailing Lists / curl-library / Single Mail

curl-library

Memory when threading curl

From: John D <cononet_at_gmail.com>
Date: Mon, 29 Jun 2009 12:01:40 +0300

Hi

This is more of a general question to Daniel if he has a moment for this.

I have an application that uses a large amount of threads. Say 1000-2000
Each thread is responsible for calling a function that conducts an
easy_setopt on about 4-5 calls.
These calls equate to about 100kb total in data back and forth with it
mostly inbound.
This is a mix of get's/post on standard http url.

The issue is how much ram curl it is requiring. I literally must leave the
default stack size within debian at 8mb.
So with 16gb of ram I am getting 1800-2000 threads. Any more and I will
encounter a seg fault in relation to thread resources.
A lower stack and it is curl's chance to seg fault.

I just am trying to decide if I need to do something like take the
Curl_BufferSize into my own hands?
I know it does not leak memory but I do not for the life of me understand
how a 100kb transaction needs an 8mb stack size?

With regards,

John
Received on 2009-06-29