tests: run (some) servers on "any port" #5247
Conversation
bagder
force-pushed
the
bagder/tests-on-dynport
branch
from
fda3f03
to
ce5ccbc
Compare
|
what is the interaction with absolute test numbers eg. take for example test2054 ... do we want to randomise that port ? |
|
I don't think so. That test verifies that the logic works with host + (fixed) port number. I don't think that's a problem. It won't connect to those ports anyway. The port number it actually connects to will be "random" though... |
|
apologies for being slow on the uptake - do these changes mean there is now no way to absolutely specify a port ? |
|
Sure, just specify a number and that number will be used. But the port numbers for test servers should never be specified like that, not now and not before either. Up until now however, such mistakes were easily let through since it works for most test runs, while this PR will make them never work (well, except for one in many thousands). |
Makes the test servers for HTTP and Gopher pop up on a currently unused port and runtests adapts to that! Closes #5247
The runtests script now always performs variable replacement on the entire test source file before the test gets executed, and saves the updated version in a temporary file (log/test[num]) so that all test case readers/servers can use that version (if present) and thus enjoy the powers of test case variable substitution. This is necessary to allow complete port number freedom. Test 309 is updated to work with a non-fixed port number thanks to this.
bagder
force-pushed
the
bagder/tests-on-dynport
branch
from
a05fa6c
to
f99b007
Compare
FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic ports Test 842-845 are unfortunately a bit hard to move over to this concept right now and require "default port" still...
bagder
changed the title
Makes the test servers (sws and ftpserver) for HTTP, Gopher, FTP, IMAP, POP3, SMTP listen on a currently unused port and runtests adapts to that! Removed the need and use of a fixed port number for these tests.
Test case preprocessing
In order to work correctly with flexible port number for all test cases and all protocols, runtests now preprocesses every test case before it is started and the updated test case file is stored as
log/test$num. Each protocol test server should now look for the preprocessed version first, then fall back to the original test file only if not found. This allows proper variable substitution in the entire test case and also makes it work for all protocols and test servers at once.