Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
On more stable curl releases
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Dan Fandrich via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 21 Mar 2023 09:22:00 -0700
I've been getting a feeling that there have been more curl functional
regressions (compared to plain bugs) in releases over the last few years. I
don't know if that's true or not, but the pace of bug fixes generally has been
going up (see https://curl.se/dashboard1.html#bugfix-frequency). I'm splitting
off regressions into a separate category of bugs since those directly fail
curl's objective of allowing applications to upgrade and without
noticing any negative change in behaviour. The remaining bugs genrally
fix issues that existed for a long time which applications presumably
hadn't hit yet, or aren't concerned about, or fix a problem created
since the last release.
Some examples of regressions would be the crash that prompted yesterday's 8.0.1
point release, the noproxy host matching bug #9842 (fixed in 7.86.0) and the
wrong units bug in --write-out %{time_total} (fixed in 7.75.0). Of these
three examples, one resulted in a point release while the other two did
not.
The problem with allowing regressions to live until the next scheduled release
without fixing them is that as the number of such regressions goes up,
the chances of everybody being able to upgrade to a scheduled release
without a problem goes down. If there ends up being one regression in
every release cycle then each curl release is no longer better than the
last one because it's going break somebody's application. And the one
after that is going to break something else. There ends up being no
release that that meets our goals for users (assuming such a goal is
even possible).
curl probably isn't at that point yet, but I would hate for it to get
there. To try to see how things stand, I looked at three Linux
distributions and counted the patches they added to curl over the past
two years of releases. These three distros, Fedora, Debian and Alpine,
are widely used and have frequent curl releases, and typically upgrade
to every new release, so I could get enough data points to be
interesting. I looked at patches between 7.75.0 (or 7.74.0 in one case
that didn't ship 7.75.0) and 7.88.1. I only counted patches that seemed
to be fixing curl functionality and ignored patches to do with the
packaging process itself.
Fedora: released 18 curl versions in that time, of which 10 included
functional patches: 56% were patched releases
Debian: released 11 curl versions in that time, of which 6 included
functional patches: 55% were patched releases
Alpine: released 17 curl versions in that time, of which 4 included
functional patches: 24% were patched releases
So, between all of them, 43% of their releases included patches that the
distros felt could not wait until the next release to include.
To look at it another way, between tags 7.75.0 and 7.88.1 (17 versions
of commits), 32 commits say "regression" in the comment, averaging 1.9
regressions per release.
The EARLY-RELEASE.md document states the criteria needed to do an
extraordinary release, but I think the bar should be lowered to include
more regression fixes. For example, the noproxy bug did not get an out
of band release, and we received bug reports about it for weeks (IIRC,
somebody was asking even in the last week about Apple's release policy
because they shipped the problematic version). I personally developed a
script relying on %{time_total} on a distro that shipped 7.74.0 without
realizing that version had the wrong time units. When I ran the script
on another system with a newer curl, it mysteriously failed because of
it.
With an installation base of 1e+10 even a "minor" regression can still
affect a huge number of people. Not all users have a distro maintainer
to curate patches and ensure they have a stable version.
I would like to see patch releases more often that include fixes that
target regressions. That's easy for me to say since I'm not the one
doing the work of making those extra releases, but that work could be
reduced by increasing the time between scheduled releases from 8 weeks
to perhaps 12 weeks. If we include a "soft" point release date 3-4 weeks
after each main release date (maybe less if the main release contained a
security fix) and half the time we use that date to do a point release
(using the distributions' ratio of patching curl), both the average time
between releases and the number of releases per year stays the same. So
it's free! (Assuming cherry-picking commits is considered to be fun and
not work...) "Release early, release often" can also relate to frequency
of releases with respect to code churn and not just calendar time.
A point release would only contain regression fixes and other fixes
deemed low risk. The idea would be that a curl .1 release is
(practically) always strictly better than anything that preceded it so
people can upgrade from a .0 release without thinking, and those who are
looking to maximize stability can just wait for the .1 (or timeout
waiting).
Dan Fandrich
Date: Tue, 21 Mar 2023 09:22:00 -0700
I've been getting a feeling that there have been more curl functional
regressions (compared to plain bugs) in releases over the last few years. I
don't know if that's true or not, but the pace of bug fixes generally has been
going up (see https://curl.se/dashboard1.html#bugfix-frequency). I'm splitting
off regressions into a separate category of bugs since those directly fail
curl's objective of allowing applications to upgrade and without
noticing any negative change in behaviour. The remaining bugs genrally
fix issues that existed for a long time which applications presumably
hadn't hit yet, or aren't concerned about, or fix a problem created
since the last release.
Some examples of regressions would be the crash that prompted yesterday's 8.0.1
point release, the noproxy host matching bug #9842 (fixed in 7.86.0) and the
wrong units bug in --write-out %{time_total} (fixed in 7.75.0). Of these
three examples, one resulted in a point release while the other two did
not.
The problem with allowing regressions to live until the next scheduled release
without fixing them is that as the number of such regressions goes up,
the chances of everybody being able to upgrade to a scheduled release
without a problem goes down. If there ends up being one regression in
every release cycle then each curl release is no longer better than the
last one because it's going break somebody's application. And the one
after that is going to break something else. There ends up being no
release that that meets our goals for users (assuming such a goal is
even possible).
curl probably isn't at that point yet, but I would hate for it to get
there. To try to see how things stand, I looked at three Linux
distributions and counted the patches they added to curl over the past
two years of releases. These three distros, Fedora, Debian and Alpine,
are widely used and have frequent curl releases, and typically upgrade
to every new release, so I could get enough data points to be
interesting. I looked at patches between 7.75.0 (or 7.74.0 in one case
that didn't ship 7.75.0) and 7.88.1. I only counted patches that seemed
to be fixing curl functionality and ignored patches to do with the
packaging process itself.
Fedora: released 18 curl versions in that time, of which 10 included
functional patches: 56% were patched releases
Debian: released 11 curl versions in that time, of which 6 included
functional patches: 55% were patched releases
Alpine: released 17 curl versions in that time, of which 4 included
functional patches: 24% were patched releases
So, between all of them, 43% of their releases included patches that the
distros felt could not wait until the next release to include.
To look at it another way, between tags 7.75.0 and 7.88.1 (17 versions
of commits), 32 commits say "regression" in the comment, averaging 1.9
regressions per release.
The EARLY-RELEASE.md document states the criteria needed to do an
extraordinary release, but I think the bar should be lowered to include
more regression fixes. For example, the noproxy bug did not get an out
of band release, and we received bug reports about it for weeks (IIRC,
somebody was asking even in the last week about Apple's release policy
because they shipped the problematic version). I personally developed a
script relying on %{time_total} on a distro that shipped 7.74.0 without
realizing that version had the wrong time units. When I ran the script
on another system with a newer curl, it mysteriously failed because of
it.
With an installation base of 1e+10 even a "minor" regression can still
affect a huge number of people. Not all users have a distro maintainer
to curate patches and ensure they have a stable version.
I would like to see patch releases more often that include fixes that
target regressions. That's easy for me to say since I'm not the one
doing the work of making those extra releases, but that work could be
reduced by increasing the time between scheduled releases from 8 weeks
to perhaps 12 weeks. If we include a "soft" point release date 3-4 weeks
after each main release date (maybe less if the main release contained a
security fix) and half the time we use that date to do a point release
(using the distributions' ratio of patching curl), both the average time
between releases and the number of releases per year stays the same. So
it's free! (Assuming cherry-picking commits is considered to be fun and
not work...) "Release early, release often" can also relate to frequency
of releases with respect to code churn and not just calendar time.
A point release would only contain regression fixes and other fixes
deemed low risk. The idea would be that a curl .1 release is
(practically) always strictly better than anything that preceded it so
people can upgrade from a .0 release without thinking, and those who are
looking to maximize stability can just wait for the .1 (or timeout
waiting).
Dan Fandrich
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-03-21