Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: On more stable curl releases
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Timothe Litt <litt_at_acm.org>
Date: Tue, 21 Mar 2023 12:40:28 -0400
I agree with the problem statement. But there's no easy answer.
I expect that with frequent patch releases, curl would end up in the
situation of most M$ releases whose strategy is- "wait for the other
people to find the bugs and only take the nth patch release." And with
fewer users, the bugs would take longer to turn up...which is a
worsening spiral.
To make a more frequent patch release scheme work, you'd need to find a
group of aggressive users/testers who'd find and report bugs early.
And be willing to take the intermediate releases.
And if you can do that, you can also pull them into the regular release
process...
Many projects have a separate release stream/channel to support this
sort of model. The challenge is recruiting and retaining the right
number and type of early release testers...
I've been down this road with OS releases - it's much more difficult to
implement an effective, lasting scheme than it first appears.
On 21-Mar-23 12:22, Dan Fandrich via curl-library wrote:
> I've been getting a feeling that there have been more curl functional
> regressions (compared to plain bugs) in releases over the last few
> years. I don't know if that's true or not, but the pace of bug fixes
> generally has been going up (see
> https://curl.se/dashboard1.html#bugfix-frequency). I'm splitting off
> regressions into a separate category of bugs since those directly fail
> curl's objective of allowing applications to upgrade and without
> noticing any negative change in behaviour. The remaining bugs genrally
> fix issues that existed for a long time which applications presumably
> hadn't hit yet, or aren't concerned about, or fix a problem created
> since the last release.
>
> Some examples of regressions would be the crash that prompted
> yesterday's 8.0.1 point release, the noproxy host matching bug #9842
> (fixed in 7.86.0) and the wrong units bug in --write-out %{time_total}
> (fixed in 7.75.0). Of these three examples, one resulted in a point
> release while the other two did not.
>
> The problem with allowing regressions to live until the next scheduled
> release without fixing them is that as the number of such regressions
> goes up, the chances of everybody being able to upgrade to a scheduled
> release without a problem goes down. If there ends up being one
> regression in every release cycle then each curl release is no longer
> better than the last one because it's going break somebody's
> application. And the one after that is going to break something else.
> There ends up being no release that that meets our goals for users
> (assuming such a goal is even possible).
>
> curl probably isn't at that point yet, but I would hate for it to get
> there. To try to see how things stand, I looked at three Linux
> distributions and counted the patches they added to curl over the past
> two years of releases. These three distros, Fedora, Debian and
> Alpine, are widely used and have frequent curl releases, and typically
> upgrade to every new release, so I could get enough data points to be
> interesting. I looked at patches between 7.75.0 (or 7.74.0 in one
> case that didn't ship 7.75.0) and 7.88.1. I only counted patches that
> seemed to be fixing curl functionality and ignored patches to do with
> the packaging process itself.
>
> Fedora: released 18 curl versions in that time, of which 10 included
> functional patches: 56% were patched releases
>
> Debian: released 11 curl versions in that time, of which 6 included
> functional patches: 55% were patched releases
>
> Alpine: released 17 curl versions in that time, of which 4 included
> functional patches: 24% were patched releases
>
> So, between all of them, 43% of their releases included patches that
> the distros felt could not wait until the next release to include.
>
> To look at it another way, between tags 7.75.0 and 7.88.1 (17 versions
> of commits), 32 commits say "regression" in the comment, averaging 1.9
> regressions per release.
>
> The EARLY-RELEASE.md document states the criteria needed to do an
> extraordinary release, but I think the bar should be lowered to
> include more regression fixes. For example, the noproxy bug did not
> get an out of band release, and we received bug reports about it for
> weeks (IIRC, somebody was asking even in the last week about Apple's
> release policy because they shipped the problematic version). I
> personally developed a script relying on %{time_total} on a distro
> that shipped 7.74.0 without realizing that version had the wrong time
> units. When I ran the script on another system with a newer curl, it
> mysteriously failed because of it.
>
> With an installation base of 1e+10 even a "minor" regression can still
> affect a huge number of people. Not all users have a distro
> maintainer to curate patches and ensure they have a stable version.
>
> I would like to see patch releases more often that include fixes that
> target regressions. That's easy for me to say since I'm not the one
> doing the work of making those extra releases, but that work could be
> reduced by increasing the time between scheduled releases from 8 weeks
> to perhaps 12 weeks. If we include a "soft" point release date 3-4
> weeks after each main release date (maybe less if the main release
> contained a security fix) and half the time we use that date to do a
> point release (using the distributions' ratio of patching curl), both
> the average time between releases and the number of releases per year
> stays the same. So it's free! (Assuming cherry-picking commits is
> considered to be fun and not work...) "Release early, release often"
> can also relate to frequency of releases with respect to code churn
> and not just calendar time.
>
> A point release would only contain regression fixes and other fixes
> deemed low risk. The idea would be that a curl .1 release is
> (practically) always strictly better than anything that preceded it so
> people can upgrade from a .0 release without thinking, and those who
> are looking to maximize stability can just wait for the .1 (or timeout
> waiting).
>
> Dan Fandrich
Received on 2023-03-21
Date: Tue, 21 Mar 2023 12:40:28 -0400
I agree with the problem statement. But there's no easy answer.
I expect that with frequent patch releases, curl would end up in the
situation of most M$ releases whose strategy is- "wait for the other
people to find the bugs and only take the nth patch release." And with
fewer users, the bugs would take longer to turn up...which is a
worsening spiral.
To make a more frequent patch release scheme work, you'd need to find a
group of aggressive users/testers who'd find and report bugs early.
And be willing to take the intermediate releases.
And if you can do that, you can also pull them into the regular release
process...
Many projects have a separate release stream/channel to support this
sort of model. The challenge is recruiting and retaining the right
number and type of early release testers...
I've been down this road with OS releases - it's much more difficult to
implement an effective, lasting scheme than it first appears.
On 21-Mar-23 12:22, Dan Fandrich via curl-library wrote:
> I've been getting a feeling that there have been more curl functional
> regressions (compared to plain bugs) in releases over the last few
> years. I don't know if that's true or not, but the pace of bug fixes
> generally has been going up (see
> https://curl.se/dashboard1.html#bugfix-frequency). I'm splitting off
> regressions into a separate category of bugs since those directly fail
> curl's objective of allowing applications to upgrade and without
> noticing any negative change in behaviour. The remaining bugs genrally
> fix issues that existed for a long time which applications presumably
> hadn't hit yet, or aren't concerned about, or fix a problem created
> since the last release.
>
> Some examples of regressions would be the crash that prompted
> yesterday's 8.0.1 point release, the noproxy host matching bug #9842
> (fixed in 7.86.0) and the wrong units bug in --write-out %{time_total}
> (fixed in 7.75.0). Of these three examples, one resulted in a point
> release while the other two did not.
>
> The problem with allowing regressions to live until the next scheduled
> release without fixing them is that as the number of such regressions
> goes up, the chances of everybody being able to upgrade to a scheduled
> release without a problem goes down. If there ends up being one
> regression in every release cycle then each curl release is no longer
> better than the last one because it's going break somebody's
> application. And the one after that is going to break something else.
> There ends up being no release that that meets our goals for users
> (assuming such a goal is even possible).
>
> curl probably isn't at that point yet, but I would hate for it to get
> there. To try to see how things stand, I looked at three Linux
> distributions and counted the patches they added to curl over the past
> two years of releases. These three distros, Fedora, Debian and
> Alpine, are widely used and have frequent curl releases, and typically
> upgrade to every new release, so I could get enough data points to be
> interesting. I looked at patches between 7.75.0 (or 7.74.0 in one
> case that didn't ship 7.75.0) and 7.88.1. I only counted patches that
> seemed to be fixing curl functionality and ignored patches to do with
> the packaging process itself.
>
> Fedora: released 18 curl versions in that time, of which 10 included
> functional patches: 56% were patched releases
>
> Debian: released 11 curl versions in that time, of which 6 included
> functional patches: 55% were patched releases
>
> Alpine: released 17 curl versions in that time, of which 4 included
> functional patches: 24% were patched releases
>
> So, between all of them, 43% of their releases included patches that
> the distros felt could not wait until the next release to include.
>
> To look at it another way, between tags 7.75.0 and 7.88.1 (17 versions
> of commits), 32 commits say "regression" in the comment, averaging 1.9
> regressions per release.
>
> The EARLY-RELEASE.md document states the criteria needed to do an
> extraordinary release, but I think the bar should be lowered to
> include more regression fixes. For example, the noproxy bug did not
> get an out of band release, and we received bug reports about it for
> weeks (IIRC, somebody was asking even in the last week about Apple's
> release policy because they shipped the problematic version). I
> personally developed a script relying on %{time_total} on a distro
> that shipped 7.74.0 without realizing that version had the wrong time
> units. When I ran the script on another system with a newer curl, it
> mysteriously failed because of it.
>
> With an installation base of 1e+10 even a "minor" regression can still
> affect a huge number of people. Not all users have a distro
> maintainer to curate patches and ensure they have a stable version.
>
> I would like to see patch releases more often that include fixes that
> target regressions. That's easy for me to say since I'm not the one
> doing the work of making those extra releases, but that work could be
> reduced by increasing the time between scheduled releases from 8 weeks
> to perhaps 12 weeks. If we include a "soft" point release date 3-4
> weeks after each main release date (maybe less if the main release
> contained a security fix) and half the time we use that date to do a
> point release (using the distributions' ratio of patching curl), both
> the average time between releases and the number of releases per year
> stays the same. So it's free! (Assuming cherry-picking commits is
> considered to be fun and not work...) "Release early, release often"
> can also relate to frequency of releases with respect to code churn
> and not just calendar time.
>
> A point release would only contain regression fixes and other fixes
> deemed low risk. The idea would be that a curl .1 release is
> (practically) always strictly better than anything that preceded it so
> people can upgrade from a .0 release without thinking, and those who
> are looking to maximize stability can just wait for the .1 (or timeout
> waiting).
>
> Dan Fandrich
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
- application/pgp-signature attachment: OpenPGP digital signature