cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: mutex stuff

From: Noel Byron <NoelByron_at_gmx.net>
Date: Thu, 21 Mar 2002 16:03:59 +0100 (MET)

> On Thu, 21 Mar 2002, Noel Byron wrote:
>
> (No longer posted to the curl-users list, only curl-library now.)
>
> > > Did you check out the previous posts on this subject?
> >
> > A huge thread.
>
> ... and here comes another huge lump of text! ;-)
>
> > I'm already through with my first 'proposal code'. That means I read
> your
> > email to late. But I think we all have very similar thoughts.
>
> That's good news. ;-)
>
> > I have supplied a README in the attached archive that touches this
> question
> > in the point: Design.
>
> For the sake of the discussion on the list, I pasted your motivation here:
>
> - The user has to supply a create, a delete, a lock and a unlock
> callback.
> I added the first two because this gives libcURL the freedom to
> use as many different mutexes as necessary. I think this is very
> important since we can not know what additions will be made to libcURL
> in the Future.
> - I introduced a opaque curl_mutex_t. This is a struct in which the
> user can store everything that is needed for the mutexes.
> - The four functions are gathered in one struct.
>
> I'll start with describing why I think the 'create' and 'delete' functions
> are only adding complexity to libcurl:
>
> * As I visiualize this, the callback coming from libcurl would include a
> "mutex identifier" (a simple enumerator, using 1, 2, 3 etc for the
> amount
> of mutexes we can come up with - new ones might be added in the future).
> That id would identify what mutex this particular lock call would
> concern.
> The application can implement one mutex for all identifiers, or one for
> each. Or none at all in fact, it can just pretend it has mutexes.
>
> * This has the benefit from the library's point of view that it never has
> to
> create or delete any mutexes. We just have to add code that
> lock_mutex(id)
> first and then unlock_mutex(id). When we add a new mutex, we just add a
> new
> identifier and use that within the code.
>
> * I do not want a solution that forces the application to actually make
> one
> native mutex for each mutex id that libcurl might use. That would impose
> much too strong restriction on the application for my taste. Imagine a
> system which has a limited number of mutexes.

Ah, I have not though about that... But since the the user code implements
the create function it can easily supply the same mutex (handle) to all
calls
to Curl_mutex_new(). In addition we could pass a CURL_MUTEX_USAGE to
Curl_mutex_new(int) so the user can decide wich mutexes he wants to use
for the different tasks in libcURL.

If I'm right, the only real difference I see is that libcURL does not have
to call
Curl_mutex_new() (and hopefully Cur_mutex_free() :o). I agree, that
could improve stability.

But if we want to reuse mutexes at all we should consider the following
situation: Thread A locks mutex m1 and while it is locked, Thread A
locks another mutex m2. This is no problem if m1 and m2 are different
mutexes, but if m2 and m1 are the same mutex we would have a
deadlock situation. Most mutex implementations are not recursive.
We should keep that in mind.
 
> * As most applications will *not* use mutexes, I believe in letting the
> complex things about mutexes get handled outside libcurl, by the app
> itself.

I agree. If you do not supply a mutex implementation in my example program,
the curl_mutex_t will be an int and the Curl_mutex* functions will do
nothing.
That is quite similar to the 'id aproach'. If I got it right...

> struct curl_mutex1 {
> int version; /* set to a fixed define to be able to track changes in
> the future */
> void *clientp; /* this pointer is passed to the callbacks */
> unlock(CURL *handle, curl_mutex_id id, void *clientp);
> lock(CURL *handle, curl_mutex_id id, void *clientp);
> };
>
> In your particular program, you probably create all mutexes first and when
> libcurl calls you, you just map the incoming id to whatever mutex you want
> to
> use. If libcurl would call your function with an id you don't have a
> special
> mutex for, you can have one single general-purpose mutex for "unknown
> ids".
>
> You then enable mutex magic in libcurl similar to this:
>
> struct curl_mutex1 moootexx=
> {CURL_MUTEX_ONE, NULL, my_unlock, my_lock};
>
> curl_easy_setopt(easy, CURLOPT_MUTEX, &moootexx);

Ah, now it becomes clearer. I had a totally diffent view. Ok, that looks
quite
elegant. But what I don't really understand is that you tie mutexex to CURL
handles. For example the global DNS cache. You would add the mutex lock
and unlock code for it as soon as a CURL handle (with a activated mutex)
would access it? That would mean some CURL handles could operate on
the cache synchronized and some not depending on CURLOPT_MUTEX
for the handle?

> Again, these are my opinions and some of what I had envisioned. I am open
> for
> discussions on different solutions to how this could/should be done. I am
> not
> a dictator here. (I just wanted it said, I hope I haven't given that
> impression.)

The same is true for me. It is no problem to change my code but I have
a important question:

A use case... I intended to use the global DNS cache. In addition I wanted
to write a function that lets me fill this cache outside the a CURL
operation.
Eg curl_hostname_resolve_and_add_to_cache("www.somwhere.xy/foo");

My next idea was to increase the cache timeout to several hours since
DNS names don't change often. And now my most twisted idea... I would
start a thread that does nothing else then hostname resolving an list of
URLs which have to be downloaded later. Remember, I want to write a
GUI app that must download a massive amount of URLs.

This is my answer to ansynchronous DNS resolving. ;o)

It could be interesting for the multi API, too. Since resoving a DNS name
could take several seconds and longer.

So my question: would this be possible with the 'id' approach?

Kind regards,
Noel

-- 
GMX - Die Kommunikationsplattform im Internet.
http://www.gmx.net
Received on 2002-03-21