Hello all!

We introduced the event-based API to libcurl many years ago (late 2006) and
ever since that time we've suffered from not having a test suite that verifies
the API with all the variety and different use cases you can do with libcurl.
We used to have the similar problem with the general libcurl multi API up
until we converted everything to using multi internally, and by doing so we
fixed a whole slew of bugs.

The event based API is also quite likely the least used out of libcurl's APIs
and therefore there is likely a whole bunch of cases where it still isn't
perfected.

So what's a good way to test the socket_action() API?

The API has the downside (in testing terms at least) that it depends on an
event library which libcurl itself doesn't depend on so we need to add a
dependeny for this part of the test suite. It is unfortunate, but I don't see
a way around it. I figure we need to make sure it can be individually used for
the event-based testing only.

There are several event library alternatives these days and I'm not up-to-date
with which one we should go with. If it at all matters... libev, libevent and
libuv seem to be the candidates. There's also the possibility of adding
support for more than one of them, at least over time.

For implementing tests to verify the API with, I have an idea of how to
quickly get hundreds of test cases to run:

We write a separate implementation of curl_easy_perform() that instead of
using the normal curl_multi_perform() approach like today, it would use a
event-based curl_multi_socket_action() one instead. Let's call it
curl_easy_perform_ev() for now. If implemented correctly, and libcurl actually
works correctly, "it should just work" exactly the same! The separate
implementation would only exist in these special debug builds and we already
do some other funny magic in debug builds so it wouldn't even be the first
debug-special-thing.

We'd have to introduce some new mode/switch something to runtests.pl so that
we can have it run the easy_perform_ev functon instead of the ordinary
easy_perform when running tests. Or perhaps better: tests run first with the
normal non-event version and then _with_ event-curl (if supported by the
build).

PROS

It will provide an excellent test coverage of that API without anyone having
to write hundreds of new tests.

CONS

It will make running all test cases take almost twice the time of today and
thus quite annoying. On my current primary dev machine, which is a reasonably
fast 3.5GHz core-i7 thing, running 800+ tests _without_ valgrind takes about
400 seconds already (and some 1100 seconds _with_ valgrind!) and some of the
autobuild machines apparently spend over 2000 seconds on a run-through.

It will also unfortunately not be made by all people since some (most?) won't
have the event library required and even more people won't build with that
option enabled anyway, so in the end it'll be a set of tests that will show
occasional problems to the few who run them. Like me.

I can't think of a better way though...

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html