On Fri, 23 Jun 2017, Max Dymond via curl-library wrote:

> Now that there's coverage stats, I'm eager to add some unit tests for parts
> of the project that don't currently have any coverage.

Lovely, thanks for considering this! I too would like to see the coverage
increased.

> From https://github.com/curl/curl/blob/master/tests/README, one of the
> requirements is that you have Python. Is it therefore allowed to write test
> servers in Python? My Perl is pretty non-existent at this point!

I think its fine to use Python, yes. I imagine that most people who'd run
tests already have Python installed or can install it fairly easily. And we
should also make it so that systems without python won't try to run these
particular tests.

> Also, what's the policy on using third party executables to provide test
> functionality - is that something that should be encouraged or should it be
> a last resort? My gut feeling would be that requiring more dependencies is
> bad for testing, so trying to keep tests "in-house" is better.

If you find good code to use for test servers, I think importing that code
into our tree should be considered. Given my experience, we usually end up
modifying the servers for our purposes sooner or later anyway so it makes
sense for that reason as well.

This said: it is not necessarily a good idea to actually use a full fledged
working server for tests. We want to control exactly what the server sends and
make it capable of sending back broken or partial responses etc, so in many
cases it makes more sense to have a stupid TCP-server that we control
completely from the test cases.

> (The specific case I'm thinking of is for the DICT protocol, which currently
> has 0% coverage - there are a number of open source servers available, but a
> simple server could be created in Python to provide a very simplified DICT
> server)

I did some failed attempts previously to use our "sws" http server to run DICT
tests against, but I couldn't come up with a good way for the server to figure
out which test case that runs - which it would need to know so that it knows
how to respond/act for that specific test.

I still think it can be done using sws. Maybe we could make it work like this:

1. make sws detect DICT somehow on the incoming traffic
2. if it finds DICT, check a file ("log/DICT.testno" perhaps?) if that file
    exists, it should contain the test number so that sws knows what to send
    back in a response (if the file doesn't exist, it won't treat it as DICT
    and instead continue as usual).
3. runtests.pl would have to detect with DICT is being tested and create the
    log/DICT.testno file before it invokes the tool in DICT tests

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html