curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Cookie storage customization?

From: Isaac Jurado via curl-library <>
Date: Sat, 18 Sep 2021 02:46:37 +0200

On Thu, Sep 16, 2021 at 11:32 PM Daniel Stenberg <> wrote:

> I would love to learn what problems replacing the storage will solve for
> you.

There are two main goals in my case:

1. Minimize the amount of "active" sessions to against a domain
2. Minimize the amount of authentication "dances"

I have domain IP (Identity Provider) and then I have domains A, B and C.
They all need a session cookie to operate, which they generate through IP
(with something based on OAuth). On the client side I have N process that
may operate against A, B and C in an unpredictable order.

Each time a new session cookie needs to be generated, there's an
authentication dance:

- The client contacts A, B or C
- Server redirects to IP with a token
- Client contacts IP (verifying identity via a special "super cookie")
- IP responds with a redirection back to A, B or C, with the new session
identifier in the query string
- Client contacts A, B or C with the path given by IP
- Server redirects to the original location and emits a Set-Cookie header,
with the new session ID

That's 4 round trips over two different connections. When a session is
valid, only one round trip is necessary.

With SQLite based approach, cookie modifications are almost immediately
visible to the rest of the processes. With a plain text file approach,
authentication cycles are more likely to happen due to cookie updates only
happening at the process lifetime boundary (cookie jar loaded when a
process starts and flushed when a process terminates).

In essence, it's nothing other than a protocol optimization under a very
peculiar concurrency environment.

> We're usually open to discuss improvements; be it in functionality, in
> performance, in API or in accessibility. But I also think that every
> suggested
> change needs to be properly motivated and there needs to be a clear and
> real
> use case for every proposal.

The use case is real: it is a toolkit intended to enable automation on
applications constrained by their web user interface. There's a mixture of
web scrapping and back end service interaction, all over HTTP and using the
same authentication mechanism. In a way, it tries to behave like a web
browser: you can have the same website open in various tabs, but they'll
all share the cookies.

But, just as it's real, I'm aware that it's a statistical outlier.

Received on 2021-09-18