Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Some questions about tests
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 10 Jan 2022 10:33:12 +0100 (CET)
On Sun, 9 Jan 2022, Dennis Clarke via curl-library wrote:
> However when I scroll through the full test report output I see plenty of
> little things that make me wonder what is happening :
If you don't want to know how the sausage is made, then the final "OK" or
"FAIL" for a test case is really the only thing you need to care about.
In order to reach that final conclusive binary result, there's a lot of woodo
going on to make it work on the widest possible range of systems. Both in
terms of speed and available system resources etc but also in terms of
different operating systems.
All platforms need their special attention. Especially platforms that are not
covered by CI jobs are not fixed by me and other curl regulars since we won't
see the problems (in time).
> test 0029...Warning: server logs lock timeout (2 seconds) expired
> Here I have to guess that the timeout is what was desired given
> that I see "OK" there.
It wouldn't say OK if it wasn't OK. Every test is checked in multiple ways and
everything needs to be a perfect match for the test to do OK.
The "advisor read lock file" for tests are explained and handled here:
https://github.com/curl/curl/blob/7edb0fcf3f92f4b58620142f41a830f1d465c833/tests/runtests.pl#L4229-L4245
... they're a somewhat of a failed attempt to deal mostly with oddities on
Windows and slower systems iirc.
> test 0111...Killed
>
> Again I see "OK" but the output "Killed" has me wondering if that
> is really what was desired? Same for test 0117, 0118 and 0119
> and a pile of others.
It's not "desired", no. It's shown because the test system killed a test
server. Someone could work away that output with some effort, but it's been
shown there for a very long time so I take it it's not a big deal to most
people.
> What is going on here? I have OpenSSH_8.6p1 installed and in the PATH
> but here I see ye ancient old SunSSH ? There are pages and pages of
> output where all manner of things go wrong and then I see this :
> RUN: failed to start the SSH server on 22654
It sounds pretty clear to me: it failed to start the SSH server.
The error message is slighly misleading, as the server is actually attempted
to run on a range of different (random) ports until it gives up. Since a while
back, the test suite uses no fixed port numbers for any servers.
> Here I must ask, do I need to run the testsuite as root user and
Never.
> also provide some more information via env vars or magic incantation
> such that curl knows where to find OpenSSH ?
Here's the function that finds the path to your sshd:
https://github.com/curl/curl/blob/7edb0fcf3f92f4b58620142f41a830f1d465c833/tests/sshhelp.pm#L368
Feel free to propose improvements.
> To continue there are many many tests where there are gaps and leaps from
> one test to another with numbers skipped :
The test suite runs over the numbers for matching test files in
tests/data/tests[num]. It is a range with gaps. See tests/data/Makefile.inc
for all of them.
Every test has preconditions that need to be met for the particular test to
run. If the precondition fails, the test won't run. The test suite will show
*some* of those failed preconditions, but not all, just to avoid too many
repeats.
> ld.so.1: sh: fatal:
> /opt/bw/build/curl-7.81.0_sunos5.10_sparcv9.001/tests/libtest/.libs/libhostname.so:
> wrong ELF class: ELFCLASS64
>
> I don't know what to make of that.
That looks like a failed precondition for a test (that uses that libhostname
trick). To fix that, identify the test and then research the specifics.
> Also there is no reason that we
> should be doing anything with 32-bit libs or 32-bit binaries on
> this machine.
Well, something in your build clearly made that happen. It's not like we force
anyone into anything 32 bit.
> Later on in the testsuite I see something that makes no sense to me
> whatsoever. I see the testsuite trying to use the blake2s and also
> blake2b message digest algorithms
>
> Perhaps this is all just a problem with Python ?
I don't know what it is. I never saw that before.
> Perhaps we can takes these one at a time and at least address the strange
> message digest issue.
Sounds like a smart approach.
Date: Mon, 10 Jan 2022 10:33:12 +0100 (CET)
On Sun, 9 Jan 2022, Dennis Clarke via curl-library wrote:
> However when I scroll through the full test report output I see plenty of
> little things that make me wonder what is happening :
If you don't want to know how the sausage is made, then the final "OK" or
"FAIL" for a test case is really the only thing you need to care about.
In order to reach that final conclusive binary result, there's a lot of woodo
going on to make it work on the widest possible range of systems. Both in
terms of speed and available system resources etc but also in terms of
different operating systems.
All platforms need their special attention. Especially platforms that are not
covered by CI jobs are not fixed by me and other curl regulars since we won't
see the problems (in time).
> test 0029...Warning: server logs lock timeout (2 seconds) expired
> Here I have to guess that the timeout is what was desired given
> that I see "OK" there.
It wouldn't say OK if it wasn't OK. Every test is checked in multiple ways and
everything needs to be a perfect match for the test to do OK.
The "advisor read lock file" for tests are explained and handled here:
https://github.com/curl/curl/blob/7edb0fcf3f92f4b58620142f41a830f1d465c833/tests/runtests.pl#L4229-L4245
... they're a somewhat of a failed attempt to deal mostly with oddities on
Windows and slower systems iirc.
> test 0111...Killed
>
> Again I see "OK" but the output "Killed" has me wondering if that
> is really what was desired? Same for test 0117, 0118 and 0119
> and a pile of others.
It's not "desired", no. It's shown because the test system killed a test
server. Someone could work away that output with some effort, but it's been
shown there for a very long time so I take it it's not a big deal to most
people.
> What is going on here? I have OpenSSH_8.6p1 installed and in the PATH
> but here I see ye ancient old SunSSH ? There are pages and pages of
> output where all manner of things go wrong and then I see this :
> RUN: failed to start the SSH server on 22654
It sounds pretty clear to me: it failed to start the SSH server.
The error message is slighly misleading, as the server is actually attempted
to run on a range of different (random) ports until it gives up. Since a while
back, the test suite uses no fixed port numbers for any servers.
> Here I must ask, do I need to run the testsuite as root user and
Never.
> also provide some more information via env vars or magic incantation
> such that curl knows where to find OpenSSH ?
Here's the function that finds the path to your sshd:
https://github.com/curl/curl/blob/7edb0fcf3f92f4b58620142f41a830f1d465c833/tests/sshhelp.pm#L368
Feel free to propose improvements.
> To continue there are many many tests where there are gaps and leaps from
> one test to another with numbers skipped :
The test suite runs over the numbers for matching test files in
tests/data/tests[num]. It is a range with gaps. See tests/data/Makefile.inc
for all of them.
Every test has preconditions that need to be met for the particular test to
run. If the precondition fails, the test won't run. The test suite will show
*some* of those failed preconditions, but not all, just to avoid too many
repeats.
> ld.so.1: sh: fatal:
> /opt/bw/build/curl-7.81.0_sunos5.10_sparcv9.001/tests/libtest/.libs/libhostname.so:
> wrong ELF class: ELFCLASS64
>
> I don't know what to make of that.
That looks like a failed precondition for a test (that uses that libhostname
trick). To fix that, identify the test and then research the specifics.
> Also there is no reason that we
> should be doing anything with 32-bit libs or 32-bit binaries on
> this machine.
Well, something in your build clearly made that happen. It's not like we force
anyone into anything 32 bit.
> Later on in the testsuite I see something that makes no sense to me
> whatsoever. I see the testsuite trying to use the blake2s and also
> blake2b message digest algorithms
>
> Perhaps this is all just a problem with Python ?
I don't know what it is. I never saw that before.
> Perhaps we can takes these one at a time and at least address the strange
> message digest issue.
Sounds like a smart approach.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2022-01-10