Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Parallel curl testing project
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Dan Fandrich via curl-library <curl-library_at_lists.haxx.se>
Date: Wed, 22 Mar 2023 10:54:20 -0700
On the long road to 8.0.0, curl has taken on close to 1600 test cases
[1] that help verify that it stays running correctly. These tests are
one of the ways that curl stays operating reliably year after year, but
the downside is that they can take an annoyingly long time to run.
Normal test runs in the CI builds take between 6 and 25 minutes, and
that's not including Valgrind or torture runs which take much longer
than that. The test suite runs tests sequentially, so running them on a
multi-core CPU makes hardly any difference to the run time.
Several CI services we rely on run builds sequentially, so it can take many
hours between submitting a PR and seeing the final results. Developers
working on their own machines are also slowed down when testing adds 10
minutes to a edit-compile-run cycle. Speeding up a test run would make
developers' lives that much better.
I looked into running tests in parallel a few years ago as a way to
speed them up [2], but the testing infrastructure had various
assumptions baked-in that would have required a commitment to do some
major refactoring. Since then, at least one of the hurdles has already
been been overcome (running servers on random ports [3]) and the number
of test cases being added keeps increasing. As CPUs advance more by
increasing the number of cores rather than making each one faster, the
test suite's serial nature is becoming more of a bottleneck that needs
to be addressed.
I"m glad that I'm finally going to be able to tackle this problem. I'll
be working on parallelizing the test suite over the next few weeks,
funded by the curl project itself. I've put together an outline of what I
intend to do [4] and would welcome comments as I dive in. Commits
will reference this issue [5] if you want to follow along.
Dan Fandrich
[1]: https://curl.se/dashboard1.html#tests
[2]: https://curl.se/mail/lib-2018-10/0004.html
[3]: https://github.com/curl/curl/pull/5247
[4]: https://github.com/curl/curl/files/11023995/curl.parallel.testing.proposal.pdf
[5]: https://github.com/curl/curl/issues/10818
Date: Wed, 22 Mar 2023 10:54:20 -0700
On the long road to 8.0.0, curl has taken on close to 1600 test cases
[1] that help verify that it stays running correctly. These tests are
one of the ways that curl stays operating reliably year after year, but
the downside is that they can take an annoyingly long time to run.
Normal test runs in the CI builds take between 6 and 25 minutes, and
that's not including Valgrind or torture runs which take much longer
than that. The test suite runs tests sequentially, so running them on a
multi-core CPU makes hardly any difference to the run time.
Several CI services we rely on run builds sequentially, so it can take many
hours between submitting a PR and seeing the final results. Developers
working on their own machines are also slowed down when testing adds 10
minutes to a edit-compile-run cycle. Speeding up a test run would make
developers' lives that much better.
I looked into running tests in parallel a few years ago as a way to
speed them up [2], but the testing infrastructure had various
assumptions baked-in that would have required a commitment to do some
major refactoring. Since then, at least one of the hurdles has already
been been overcome (running servers on random ports [3]) and the number
of test cases being added keeps increasing. As CPUs advance more by
increasing the number of cores rather than making each one faster, the
test suite's serial nature is becoming more of a bottleneck that needs
to be addressed.
I"m glad that I'm finally going to be able to tackle this problem. I'll
be working on parallelizing the test suite over the next few weeks,
funded by the curl project itself. I've put together an outline of what I
intend to do [4] and would welcome comments as I dive in. Commits
will reference this issue [5] if you want to follow along.
Dan Fandrich
[1]: https://curl.se/dashboard1.html#tests
[2]: https://curl.se/mail/lib-2018-10/0004.html
[3]: https://github.com/curl/curl/pull/5247
[4]: https://github.com/curl/curl/files/11023995/curl.parallel.testing.proposal.pdf
[5]: https://github.com/curl/curl/issues/10818
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-03-22