Re: A CI job inventory

From: Timothe Litt <litt_at_acm.org>
Date: Mon, 7 Feb 2022 17:34:55 -0500

Oh, and there might be benefit in having a real database, or a script to
load this file into one.

You really don't want to create custom query/management code when SQL
utilities abound.

Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.

On 07-Feb-22 17:10, Daniel Stenberg via curl-library wrote:
> Hello team!
>
> # Background
>
> At the moment we have a little over 100 CI jobs that run for each PR
> push and for each merge to the master repository on github.
>
> The dashboard [1] shows some graphs for the CI development and changes
> over time, but I really feel that I am starting to lose track of all
> the builds and exactly what they verify or not.
>
> In order to get better overview and control of the jobs we run, I'm
> proposing that we create and maintain a single file that lists all the
> jobs we run. This "database" of jobs could then be used to run checks
> against and maybe generate some tables or charts and what not to help
> us make sure our CI jobs really covers as many build combinations as
> possible and perhaps it can help us reduce duplications or too-similar
> builds.
>
> # The plan
>
> Create docs/CI.md to that documents the file format for the inventory
>
> Create tests/ci-jobs.txt which is the actual list with all the jobs.
>
> I'll create a first PR soon when I have a few jobs added, then
> probably add/convert more jobs in subsequent PRs in a batch manner.
>
> # File format
>
> The file format is a simple "key: value" design where "id" is the
> first one for a job and then we have a long range of keys to use. See
> below.
>
> - id (short unique identifier among the CI jobs). The CI job should
> refer back
>   to this id.
> - desc (optional human redable description)
> - service (running it: zuul, gha, azure, cirrus, circle, appveyor)
> - time (number of minutes a typical run goes)
> - source (file for the CI job in the git repo)
> - compiler (gcc, clang etc)
> - special compiler options (sanitizers etc)
> - OS (linux, windows, freebsd, macos)
> - CPU (x86_64, x86, arm)
> - build system (autotools, cmake, winbuild, vstudio)
> - debug enabled (true/false)
> - runs tests (true/false)
> - valgrind (true/false)
> - amissl (true/false)
> - bearssl (true/false)
> - gnutls (true/false)
> - mbedtls (true/false)
> - nss (true/false)
> - openssl (true/false)
> - BoringSSL (true/false)
> - libressl (true/false)
> - rustls (true/false)
> - schannel (true/false)
> - secure-transport (true/false)
> - wolfssl (true/false)
> - libssh2 (true/false)
> - libssh (true/false)
> - wolfssh (true/false)
> - zlib (true/false)
> - zstd (true/false)
> - brotli (true/false)
> - libidn2 (true/false)
> - winidn (true/false)
> - c-ares (true/false)
>
> # Basic verification
>
> I hope I can also write up some basic checks that the jobs used in
> this list is also actually defined in the yaml files and vice versa,
> to help us make sure that the files are all in sync even when going
> forward.
>
> # Feedback
>
> Thoughts? Anything major I've forgotten, did wrong or should add to
> this while we're at it? Does the file format work or should I do it
> differently?
>
> [1] = https://curl.se/dashboard.html
>
>

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Received on 2022-02-07