Plan: switch test servers over to "random" ports

Hi,

Test server port use overhaul coming up!

# Background

Over the years we've been discussing how to make tests run faster and we've
discussed how to run tests in parallel on the same machine.

At the same time, the test suite's use of fixed port numbers has caused us
problems over the years: on some machines those ports are used for other
services, then we can ask curl to run on other ports for that machine, but it
isn't used often enough so then we find a set of tests that can't work on
changed ports!

# TODO

A while ago I added a TODO entry with a plan on how to fix some of these
problems: https://curl.haxx.se/docs/todo.html#Use_random_ports_for_the_test

# MQTT is first

When I created and added a new test server to the mix for my ongoign MQTT
support, I decided to try out that concept with "any available port" already
from the start. Turned out to work really well (so far) and the mqttd server
now listens to any port and runtests.pl picks up the used port number and goes
with that. For creators of test cases, it shouldn't be visible. It will
effectlively make us avoid writing test cases to use fixed port numbers,
because such tests won't even work twice on the same machine!

# Switching over more test servers

Once the initial mqtt work has landed (should happen soon) and the
infrastructure for figuring out the port numbers from each server is in
master, moving over more servers to this system should be straight forward.

The problematic ones are the servers we didn't write:

  - stunnel can be told to accept traffic on port 0 and then it'll go random,
    but I can't find a "good" way to figure out the port it picked. I suspect
    we will have to parse its debug log in order to figure out which port is
    actually listens to... and that feels fragile.

  - sshd, used for SFTP and SCP tests doesn't seem to allow port 0 or "any".
    It puzzles me, so maybe I just haven't figured out yet how... I'm thinking
    maybe we can work around that limitation manually, by using a list of ports
    to try in a random order.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html