Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: A CI job inventory
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 8 Feb 2022 08:38:26 +0100 (CET)
On Mon, 7 Feb 2022, Timothe Litt via curl-library wrote:
> Maybe not precisely, but an approximation is possible. See my response to
> Dan's note. A real database can be kept. A CI job's exit status can be
> captured, timestamped, and recorded. PRs can be tagged with the CI job that
> found an issue. Then you can get found bugs found/job - and a time series.
> That's a reasonable proxy for effectiveness.
Very few CI jobs turning red result in a new issue. They typically show that
the PR in progress need furter attention. Usually ending up in a commit or two
more getting added to the same branch, which then are squashed into the same
single commit at the time of merge. The problem is then never present in the
master branch.
> Tags can also let you categorize bug types - e.g. memory leaks, buffer
> overflows, protocol errors...
That would be awesome indeed, and massive amount of work to setup. We don't
even have the enbryo for such a setup now.
>> The CI jobs do their best work on PRs, and for those there are no "less
>> frequently"
> You can manually trigger once a week, or before a release, every 10th PR,
> or... You can also script "manual" triggers.
Yes that is theoretically possible, and that would indeed make them run more
seldomly. It then converts from from the effective CI jobs we have into the
autobuild style we have had for a very long time. That's very ineffective
because: 1. very few people watch the progress of them and 2. the original
author of the commit already moved on in a lot of cases. That's why it is so
important to test and verify as much as possible before the PR is accepting in
the first place. That's when the incentive and motivation is still with the
original author to keep polishing the work until no further flaws can be
found.
> If the protocol or platform that you fixed is the 199th job in the queue,
> you're waiting for feedback. That's expensive too. Even if you do something
> else, you lose task switching time. More intelligent scheduling helps.
Waiting 2-3 hours for the CI jobs to complete is usually not a problem anyway,
as in a majority of cases they are still completed by the same a human has had
the opportunity to review the proposed change. If waiting 2-3 hours is a
problem for a contributor, then I think we have another problem than the CI
setup.
Date: Tue, 8 Feb 2022 08:38:26 +0100 (CET)
On Mon, 7 Feb 2022, Timothe Litt via curl-library wrote:
> Maybe not precisely, but an approximation is possible. See my response to
> Dan's note. A real database can be kept. A CI job's exit status can be
> captured, timestamped, and recorded. PRs can be tagged with the CI job that
> found an issue. Then you can get found bugs found/job - and a time series.
> That's a reasonable proxy for effectiveness.
Very few CI jobs turning red result in a new issue. They typically show that
the PR in progress need furter attention. Usually ending up in a commit or two
more getting added to the same branch, which then are squashed into the same
single commit at the time of merge. The problem is then never present in the
master branch.
> Tags can also let you categorize bug types - e.g. memory leaks, buffer
> overflows, protocol errors...
That would be awesome indeed, and massive amount of work to setup. We don't
even have the enbryo for such a setup now.
>> The CI jobs do their best work on PRs, and for those there are no "less
>> frequently"
> You can manually trigger once a week, or before a release, every 10th PR,
> or... You can also script "manual" triggers.
Yes that is theoretically possible, and that would indeed make them run more
seldomly. It then converts from from the effective CI jobs we have into the
autobuild style we have had for a very long time. That's very ineffective
because: 1. very few people watch the progress of them and 2. the original
author of the commit already moved on in a lot of cases. That's why it is so
important to test and verify as much as possible before the PR is accepting in
the first place. That's when the incentive and motivation is still with the
original author to keep polishing the work until no further flaws can be
found.
> If the protocol or platform that you fixed is the 199th job in the queue,
> you're waiting for feedback. That's expensive too. Even if you do something
> else, you lose task switching time. More intelligent scheduling helps.
Waiting 2-3 hours for the CI jobs to complete is usually not a problem anyway,
as in a majority of cases they are still completed by the same a human has had
the opportunity to review the proposed change. If waiting 2-3 hours is a
problem for a contributor, then I think we have another problem than the CI
setup.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2022-02-08