Hello,

>> To cut the long story short, one use case is to be able to find out
>> the max keepalive requests setting on a webserver by fetching a URL
>> with little content using HTTP/1.1 requests. The simplest approach
>> seemed to be watching the the difference between CURLINFO_NUM_CONNECTS
>> and CURLINFO_REDIRECT_COUNT. Once this difference would exceed 0, it
>> means we had to initiate a new connection. Of course the initial
>> connection should be skipped in this loop.
>
> I don't understand why you need to involve the redirect count. Can't you
> just read the CURLINFO_NUM_CONNECTS and as long as it returns zero you
> know that it re-used the previous connection?

The explanation regarding these two parameters confused me profoundly:

CURLINFO_NUM_CONNECTS:
Pass a pointer to a long to receive how many new connections libcurl had
to create to achieve the previous transfer (only the successful connects
are counted). Combined with CURLINFO_REDIRECT_COUNT you are able to know
how many times libcurl successfully reused existing connection(s) or
not. See the Connection Options of curl_easy_setopt(3) to see how
libcurl tries to make persistent connections to save time. (Added in 7.12.3)

So my question is as follows: How do I get the amount of successful
connects before libcurl was not able anymore to reuse (being forced by
the webserver through the max'd out MaxKeepAliveRequests number) the
handle and had to re-initiate a connection?

Or in other words, if on a webserver we have following settings on an
apache 2.0.x server:

MaxKeepAliveRequests 13
KeepAlive On
KeepAliveTimeout 15

How do I get the MaxKeepAliveRequests by probing using my approach?

> I don't see how it is off by two. I set MAX_LOOP to 5, and run it on a
> URL of mine and then it says it was 3 connections while I could clearly
> see how all requests re-used the same one. Thus it was off by 4.

?? off by 2, I'd say. since you did 5 connections and the program said
it did make 3.

> Increase MAX_LOOP makes it even more off.

Well, try setting MAX_LOOP to MaxKeepAliveRequests + 1 and re-run your
tests.

> I don't see how your program can do what you describe you want it to do.

Neither do I, since I misunderstand the CURLINFO_REDIRECT_COUNT
semantics in conjunction with CURLINFO_NUM_CONNECTS.

>> And would it be intelligent to have more information on the socket
>> state reported back through curl_easy_getinfo(), such as an n-tuple of
>> socket addr/peer addr/port/peer port/state/... ?
>
> Yes, adding ability to extract further information would indeed be
> usable and I wouldn't mind adding such features, should someone write a
> patch for it.

Hmm, maybe in a spare minute I could dive into the libcurl sources but
so far I was extremely happy that the implementation and documentation
didn't require me to do so.

>> Another problem I've encountered is that if you start a couple of
>> hundreds of parallel test and you get close to resource starvation
>> libcurl kind of seems to have issues with signal handling.
>
> What kind of resources would that starve out?

Maybe signal ordering or the non-reentrancy of sleep() is the culprit. I
often ran out of file descriptors because for the tests I had to use all
kinds of systems and some of them quickly ran out of fd's (especially
those with 2.0.x kernels and systems based on virtual servers). I'll
definitely post an in-depth analysis once I get time to reproduce this.
In 99.999% of the cases all works well and I have a work-around for this
problem.

> I don't recognize it. And as you say, problems that requires a large
> amount of threads and transfers are really hard to repeat and even
> harder to fix!

Thanks for your comments,
Roberto Nibali, ratz

-- 
-------------------------------------------------------------
addr://Kasinostrasse 30, CH-5001 Aarau tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------