Skip to content

test/certs: clean up #16593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

test/certs: clean up #16593

wants to merge 7 commits into from

Conversation

icing
Copy link
Contributor

@icing icing commented Mar 6, 2025

Keep only the generated files needed for tests. Place generated intermediaries in tests/certs/gen where they are ignored by git. No longer generated *.dhp files.

Have a shorter naming scheme: test-ca instead of EdelCurlRoot-ca and test-localhost instead of Server-localhost-sv, etc.

Remove the stunnel certificate as it was nearly a duplicate of test-localhost.

No longer copy a generated certificates to tests/stunnel.pem. Let test server default to certs/test-localhost.pem instead.

icing added 3 commits March 6, 2025 11:01

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing
Copy results out, leave the mess in gen, ignored by git.

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing
No longer copy stunnel.pem to tests dir, let test servers use the files
in the tests/certs dir as needed.
@github-actions github-actions bot added the tests label Mar 6, 2025
icing added 3 commits March 6, 2025 12:39

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing
… that

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing
@github-actions github-actions bot added the CI Continuous Integration label Mar 6, 2025

Verified

This commit was signed with the committer’s verified signature.
icing Stefan Eissing
@bagder bagder closed this in 556f722 Mar 6, 2025
vszakats added a commit that referenced this pull request Mar 27, 2025

Verified

This commit was signed with the committer’s verified signature.
vszakats Viktor Szakats
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.

Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.

As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.

Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.

Details:

- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
  separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
  confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
  auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
  noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
  ```
  + openssl genrsa -out test-ca.key -passout fd:0 2048
  Invalid password argument, starting with "fd:"
  ```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
  (`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
  the Visual Studio generator miss to create the `clean-certs`,
  `build-certs` targets. No target depend on them, so they don't execute
  implicitly anyway. Fixes:
  ```
  MSBUILD : error MSB1009: Project file does not exist.
  Switch: clean-certs.vcxproj
  ```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.

Follow-up to 556f722 #16593
Follow-up to fa461b4 #14486

Closes #16824
pps83 pushed a commit to pps83/curl that referenced this pull request Apr 26, 2025
Keep only the generated files needed for tests. Place generated
intermediaries in `tests/certs/gen` where they are ignored by git. No
longer generated `*.dhp` files.

Have a shorter naming scheme: `test-ca` instead of `EdelCurlRoot-ca` and
`test-localhost` instead of `Server-localhost-sv`, etc.

Remove the `stunnel` certificate as it was nearly a duplicate of
`test-localhost`.

No longer copy a generated certificates to `tests/stunnel.pem`. Let test
server default to `certs/test-localhost.pem` instead.

Closes curl#16593
pps83 pushed a commit to pps83/curl that referenced this pull request Apr 26, 2025

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.

Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.

As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.

Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.

Details:

- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
  separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
  confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
  auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
  noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
  ```
  + openssl genrsa -out test-ca.key -passout fd:0 2048
  Invalid password argument, starting with "fd:"
  ```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
  (`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
  the Visual Studio generator miss to create the `clean-certs`,
  `build-certs` targets. No target depend on them, so they don't execute
  implicitly anyway. Fixes:
  ```
  MSBUILD : error MSB1009: Project file does not exist.
  Switch: clean-certs.vcxproj
  ```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.

Follow-up to 556f722 curl#16593
Follow-up to fa461b4 curl#14486

Closes curl#16824
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration tests
Development

Successfully merging this pull request may close these issues.

None yet

3 participants