cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Cacheing In

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sat, 16 Feb 2002 18:51:33 +0100 (MET)

On Mon, 11 Feb 2002 CHRIS.CLARK_at_FRIENDSPROVIDENT.CO.UK wrote:

> Hi!

Hey. Sorry it took a while to respond, no one else seems to have grabbed this
and I was on vacation. Anyway, here's my shot...

> When each thread starts up, it creates its own curl handle, and then sets
> up filenames on that handle for a client certificate, a client private key
> and password, and a trusted server certificate, using curl_easy_setopt
> calls.
>
> The values of these filenames, passwords, etc, given by each thread are
> identical (each thread is using a single client certificate and key, to
> represent itself - a single client application - to the Web server, and a
> single trusted server certificate to represent the one and only Web server
> it talks to).
>
> At a high rate of knots after that, my threads are called (in no
> particular order, but in rapid succession) and a called thread will fire
> off a single HTTPS URL to the (one and only) Web server. When the thread
> gets the response, and processes it, it goes back to sleep for a while,
> until it's prodded into life again.
>
> My question is this - for each thread, how many times do the various files
> (client certificate, client key, and server certificate) get read in
> physically from disk? Is it only once each per thread (for the first time
> the thread gets called to issue a URL)? That would be really neat, as it
> would indicate a fantastic caching deal going on somewhere, either in
> libcurl, or OpenSSL, or possibly both.
>
> Or would it be what I fear, which is three disk reads (one per file), per
> thread, per requested URL? Yes I know that even then I'd probably get a
> good deal from a goodly-sized disk cache, but I'd like this thing to really
> fly if I can make it.
>
> Any takers? Thanks in advance for any illumination you can shed.

I won't deny that some of your fears as stated above most likely are true.
Not for every possible file etc, but still for a fair share of them.

I'll willingly admit that a very small percentage of my development time on
curl is spent on increasing performance and I need help and input to improve
it. Far too much of the curl development tend to get done by me alone. This
said, I'm sure we can improve several of these things to get cached or
read-once instead of read-every-time. I would love a detailed study on what
curl does today and some suggestions on how we can improve this. Like the
talk about certificates and how we can improve performance on that handling,
provided by Cris Bailiff a few weeks ago.

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/
Received on 2002-02-16