Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Parallel curl testing project
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Stefan Eissing via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 23 Mar 2023 08:57:20 +0100
Dan,
very happy that you will work on this. I like to run the test suite locally before a large push and the time it takes on my machine is around 10 minutes. I'd very much appreciate that to go down!
Kind Regards,
Stefan
> Am 22.03.2023 um 18:54 schrieb Dan Fandrich via curl-library <curl-library_at_lists.haxx.se>:
>
> On the long road to 8.0.0, curl has taken on close to 1600 test cases [1] that help verify that it stays running correctly. These tests are one of the ways that curl stays operating reliably year after year, but the downside is that they can take an annoyingly long time to run. Normal test runs in the CI builds take between 6 and 25 minutes, and that's not including Valgrind or torture runs which take much longer than that. The test suite runs tests sequentially, so running them on a multi-core CPU makes hardly any difference to the run time.
>
> Several CI services we rely on run builds sequentially, so it can take many hours between submitting a PR and seeing the final results. Developers working on their own machines are also slowed down when testing adds 10 minutes to a edit-compile-run cycle. Speeding up a test run would make developers' lives that much better.
>
> I looked into running tests in parallel a few years ago as a way to speed them up [2], but the testing infrastructure had various assumptions baked-in that would have required a commitment to do some major refactoring. Since then, at least one of the hurdles has already been been overcome (running servers on random ports [3]) and the number of test cases being added keeps increasing. As CPUs advance more by increasing the number of cores rather than making each one faster, the test suite's serial nature is becoming more of a bottleneck that needs to be addressed.
>
> I"m glad that I'm finally going to be able to tackle this problem. I'll be working on parallelizing the test suite over the next few weeks, funded by the curl project itself. I've put together an outline of what I intend to do [4] and would welcome comments as I dive in. Commits will reference this issue [5] if you want to follow along.
>
> Dan Fandrich
>
> [1]: https://curl.se/dashboard1.html#tests
> [2]: https://curl.se/mail/lib-2018-10/0004.html
> [3]: https://github.com/curl/curl/pull/5247
> [4]: https://github.com/curl/curl/files/11023995/curl.parallel.testing.proposal.pdf
> [5]: https://github.com/curl/curl/issues/10818
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html
Date: Thu, 23 Mar 2023 08:57:20 +0100
Dan,
very happy that you will work on this. I like to run the test suite locally before a large push and the time it takes on my machine is around 10 minutes. I'd very much appreciate that to go down!
Kind Regards,
Stefan
> Am 22.03.2023 um 18:54 schrieb Dan Fandrich via curl-library <curl-library_at_lists.haxx.se>:
>
> On the long road to 8.0.0, curl has taken on close to 1600 test cases [1] that help verify that it stays running correctly. These tests are one of the ways that curl stays operating reliably year after year, but the downside is that they can take an annoyingly long time to run. Normal test runs in the CI builds take between 6 and 25 minutes, and that's not including Valgrind or torture runs which take much longer than that. The test suite runs tests sequentially, so running them on a multi-core CPU makes hardly any difference to the run time.
>
> Several CI services we rely on run builds sequentially, so it can take many hours between submitting a PR and seeing the final results. Developers working on their own machines are also slowed down when testing adds 10 minutes to a edit-compile-run cycle. Speeding up a test run would make developers' lives that much better.
>
> I looked into running tests in parallel a few years ago as a way to speed them up [2], but the testing infrastructure had various assumptions baked-in that would have required a commitment to do some major refactoring. Since then, at least one of the hurdles has already been been overcome (running servers on random ports [3]) and the number of test cases being added keeps increasing. As CPUs advance more by increasing the number of cores rather than making each one faster, the test suite's serial nature is becoming more of a bottleneck that needs to be addressed.
>
> I"m glad that I'm finally going to be able to tackle this problem. I'll be working on parallelizing the test suite over the next few weeks, funded by the curl project itself. I've put together an outline of what I intend to do [4] and would welcome comments as I dive in. Commits will reference this issue [5] if you want to follow along.
>
> Dan Fandrich
>
> [1]: https://curl.se/dashboard1.html#tests
> [2]: https://curl.se/mail/lib-2018-10/0004.html
> [3]: https://github.com/curl/curl/pull/5247
> [4]: https://github.com/curl/curl/files/11023995/curl.parallel.testing.proposal.pdf
> [5]: https://github.com/curl/curl/issues/10818
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-03-23