cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: mutex stuff

From: Noel Byron <NoelByron_at_gmx.net>
Date: Thu, 21 Mar 2002 17:18:03 +0100 (MET)

> On Thu, 21 Mar 2002, Noel Byron wrote:
>
> > Ah, I have not though about that... But since the the user code
> implements
> > the create function it can easily supply the same mutex (handle) to all
> > calls to Curl_mutex_new(). In addition we could pass a CURL_MUTEX_USAGE
> to
> > Curl_mutex_new(int) so the user can decide wich mutexes he wants to use
> > for the different tasks in libcURL.
> >
> > If I'm right, the only real difference I see is that libcURL does not
> have
> > to call Curl_mutex_new() (and hopefully Cur_mutex_free() :o). I agree,
> that
> > could improve stability.
> >
> > But if we want to reuse mutexes at all we should consider the following
> > situation: Thread A locks mutex m1 and while it is locked, Thread A
> locks
> > another mutex m2. This is no problem if m1 and m2 are different mutexes,
> > but if m2 and m1 are the same mutex we would have a deadlock situation.
> > Most mutex implementations are not recursive. We should keep that in
> mind.
>
> That's a very good point, and I haven't really put much thought on that
> yet.
>
> It only makes a dead-lock if the libcurl code attempts to lock two mutexes
> at
> the same time, as then the application might use the same native mutex for
> both of them and thus it'll hang forever.
>
> I say we should avoid (at all costs) locking two mutexes at the same time
> within libcurl (I mean within one single thread running libcurl).
>
> Right? Wouldn't that prevent this dead-lock risk?

Yes, it should be quite easy to avoid.

> > > You then enable mutex magic in libcurl similar to this:
> > >
> > > struct curl_mutex1 moootexx=
> > > {CURL_MUTEX_ONE, NULL, my_unlock, my_lock};
> > >
> > > curl_easy_setopt(easy, CURLOPT_MUTEX, &moootexx);
> >
> > Ah, now it becomes clearer. I had a totally diffent view.
>
> I figured that. That's also why I like to talk around actual code examples
> and prototypes. It makes it less abstract.
>
> > Ok, that looks quite elegant. But what I don't really understand is that
> > you tie mutexex to CURL handles. For example the global DNS cache. You
> > would add the mutex lock and unlock code for it as soon as a CURL handle
> > (with a activated mutex) would access it? That would mean some CURL
> handles
> > could operate on the cache synchronized and some not depending on
> > CURLOPT_MUTEX for the handle?
>
> You focus too much on the DNS cache! ;-) I'm trying to set a generic
> approach
> for all kinds of sharing between handles.
>
> The "global" DNS cache is a kind of violation of the whole libcurl
> principle
> (global variables don't belong in library code), and as such it must not
> be
> used as a (good) example when we discuss data sharing.

Ok, now I think, I got it. If I want to 'simulate' global data such as the
DNS cache
I set the same cache for each CURL handle. And if somebody wants to use
3 CURL handles with cookie-jar A and another 4 CURL handles with jar B
it remains possible.

But I think this will be quite hard to implement with the CURLOPT_SHARE
method. For example, somebody wants to to share the DNS cache (sorry)
between all CURL handles. And he uses the one thread and CURL handle
per download method. Every time he starts a thread (and builds a CURL
handle)
he has to know about al the other handles to share the ressource with them.
This is not possible IMHO.

If we want to do so, we must pass references (handles) of the resources
out of libcURL. A client could for example obtain a handle to the DNS
cache save it globally and pass it to every CURL handle. And such a
ressource should be associated with a mutex.

What do you think?

> > I intended to use the global DNS cache. In addition I wanted to write a
> > function that lets me fill this cache outside the a CURL operation. Eg
> > curl_hostname_resolve_and_add_to_cache("www.somwhere.xy/foo");
>
> There's no API for "filling" the cache from outside libcurl and I honestly
> can't see the point of doing it either. Can you elaborate on what good
> this
> would do for your program?

I think, that is another discussion. :o)

> > And now my most twisted idea... I would start a thread that does nothing
> > else then hostname resolving an list of URLs which have to be downloaded
> > later. Remember, I want to write a GUI app that must download a massive
> > amount of URLs.
> >
> > This is my answer to ansynchronous DNS resolving. ;o)
>
> Yes, but the total running time will remain the same. You won't gain
> anything
> by resolving the names prematurely, on the contrary, you only risk that
> the
> info is bad when you eventually need/use it.

May be I'm wrong but I think that the DNS cache stores hostname/ip
pairs. And this mapping remains valid for hours. If you don't consider
new economy companies. ;o)

The total time will remain the same, but I can make sure that my download
threads does not stick in the DNS resolval. That is a real time question.
;o)
Ok, that may be overkill, I admit. But again the multi interface will block
in
this situation, too.

BTW if somebody knows how to do asynchronous DNS for all the other
plattforms, I could contribute the windows stuff? :o)

Kind regards,
Noel

-- 
GMX - Die Kommunikationsplattform im Internet.
http://www.gmx.net
Received on 2002-03-21