On Wed, Sep 17, 2008 at 6:34 PM, Dan Fandrich <dan_at_coneharvesters.com>wrote:

> On Wed, Sep 17, 2008 at 03:37:17PM +0200, Aurélien wrote:
> > I'm using libcurl in a multi-threaded monitoring application that is
> using cURL
> > to make reports via HTTP. I read the documentation about threading and
> libcurl,
> > and deduced the following things:
> >
> > - cURL should be initialized and released via curl_global_*() functions,
> when
> > there's only the main thread running.
> > - A cURL handle should not be used by more than one thread at a time.
> > - If using HTTPs, proper locking functions should be installed in the
> openssl
> > library.
> >
> > Maybe I already missed or misunderstood something here.
>
> Sounds like you're on the right track here. Don't forget about disabling
> signals, though.
>

It was done (one thread manages all the signals, via sigwait()), and the
proper curl option was set to not use signals. I forgot to mention it.
Sorry.

>
> > Anyway, after libcurl is initialized as advised via curl_global_*()
> function -
> > I'm using normal HTTP without authentication, so no need for openssl
> stuff, my
> > application's threads are using a single function that deals with
> libcurl. This
> > function makes a HTTP request and immediately returns the result (content
> +
> > return code) to the caller. It can be summarized in pseudo-code as:
> >
> > string fetchURL(string URL, string postdata)
> > {
> > lock() // I'm using a pthread_mutex specific to this function.
> > curl_easy_setopt(CURLOPT_WRITEFUNCTION...);
> > curl_easy_setopt(CURLOPT_WRITEDATA...);
> > curl_easy_setopt(CURLOPT_POSTFIELDS...);
> > curl_easy_setopt(CURLOPT_HTTPHEADER...);
> > curl_easy_setopt(CURLOPT_URL...);
> >
> > curl_easy_perform(...);
> > ...
> > unlock()
> > }
> >
> > Each running thread can call this function at any time, hence the
> locking. If
> > you need any functional details about my application, feel free to ask.
>
> As long as you provide for mutual exclusion like this, you should be fine.
> That's assuming that the other threads aren't using any of libcurl's
> dependent libraries in a non-thread-safe manner, of course.
>

No, I don't think. There's one other place where I use openssl to verify a
signature, but it's before starting other threads.

>
> > My problem is that, if the connection to the server is lost (modem
> problem,
> > network cable unplugged, etc), sometimes the thread currently detaining
> the
> > lock never gets out of curl_easy_perform. I have compiled libcurl with
> > debugging information and got the following backtraces:
>
> "never" gets out? How long did you wait? TCP is designed to cope with
> network outages like this by retrying, potentially for a long time. If
> you don't like that, you can set a timeout option like
> CURLOPT_LOW_SPEED_LIMIT
> to make libcurl abort early.

I waited some days (really), so I don't think it's entirely a TCP-related
issue. But, the options you mentioned seems to be effective in my test
environment, so I will bench them over the next few days in production.
Anyway, I imagine that after some reasonable amount of time, the apache
server at the other end closes the connection, and at this time, curl should
stop waiting... which it doesn't do in my application.

The problem is quite hard and long (12-24hrs) to reproduce in production, so
i can only test it in a VM. I'm not exactly sure of the exact conditions in
which it occurs.

What period of time is "a long time" for you ? Hours ? days ? longer ?

Anyways, thanks for your quick answer :)

-- 
Aurélien Guillaume
"I love deadlines. I like the whooshing sound they make as they fly by."