Buy commercial curl support from
WolfSSL. We help you work out your issues, debug your libcurl
applications, use the API, port to new platforms, add new features and more.
With a team lead by the curl founder himself.
Re: Reproducing the release tarballs
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Dan Fandrich via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 1 Apr 2024 16:55:03 -0700
On Sun, Mar 31, 2024 at 11:24:27AM +0200, Daniel Stenberg wrote:
> On Sat, 30 Mar 2024, Dan Fandrich via curl-library wrote:
>
> > SPDX seems to be the standard SBOM format for this that tools are
> > starting to expect. The format is able to handle complex situations,
> > but given the very limited scope needed in curl and for source releases
> > only, once you get a template file set up the first time filling in the
> > details for every release should be simple.
>
> I can't but to feel that this is aiming (much) higher than what I want to
> do. If someone truly thinks SPDX is a better way to provide this information
> then I hope someone will step up and convert the scripts to instead use this
> format.
>
> This is a SBOM for the tarball creation, not for curl.
Well, what is the tarball but the tarball of "curl"? SPDX can provide
information on the files in the tarball as well as the files used to create the
tarball. How much you provide is up to you, but the more information available,
the more possibilities there are for others to use it.
> I rather start with something basic and simple, as we don't even know if
> anyone cares or wants this information.
That makes sense. SPDX is definitely heavier weight than a few version numbers
in an .md file. But, a lot more useful, too.
> > Even running "reuse spdx" in the curl tree (the same tool that's keeping
> > curl in REUSE compliance in that CI build) will output a SPDX file for
> > curl.
>
> I tried it just now. It produces 86,000 lines of output! And yet I can't
> find a lot of helpful content within the output for our purpose here.
That example was just the first one I thought of that you might already have on
your system (due to the work in getting REUSE compliance some time ago). It
doesn't solve the problem at hand, but it shows what SPDX looks like and it
could still be integrated into a final curl SPDX file provided with each
release if we wanted it to. Few projects provide SPDX files right now which is why
companies using SPDX only for license compatibility checking need to run "reuse
spdx" on the source code themselves. But if curl provided that SPDX file already
filled in with each release, including the additional information on the
dependencies used to create the tar ball itself, that single file can serve two
purposes. Even more purposes, actually, since it could be additionally be used
for security scanning, such as finding that curl used a back-door autoconf m4
macro found only in the tarball (if that ends up happening one day).
> It does not seem like a suitable tool for this.
Agreed. It just gives a flavour of one of the kinds of things a SPDX file can
provide, but could become part of a solution.
A tool that might actually do what you want is
https://pypi.org/project/distro2sbom/ That creates a SPDX file listing all the
packages in the current system (e.g. Debian packages on Debian). You probably
don't want to run that on your personal system (way too many irrelevant
packages), but it could be run from a minimal container used just to create a
tarball to provide a more easily reproducible set of packages for others to
fall on want to completely reproduce that build process.
Dan
Date: Mon, 1 Apr 2024 16:55:03 -0700
On Sun, Mar 31, 2024 at 11:24:27AM +0200, Daniel Stenberg wrote:
> On Sat, 30 Mar 2024, Dan Fandrich via curl-library wrote:
>
> > SPDX seems to be the standard SBOM format for this that tools are
> > starting to expect. The format is able to handle complex situations,
> > but given the very limited scope needed in curl and for source releases
> > only, once you get a template file set up the first time filling in the
> > details for every release should be simple.
>
> I can't but to feel that this is aiming (much) higher than what I want to
> do. If someone truly thinks SPDX is a better way to provide this information
> then I hope someone will step up and convert the scripts to instead use this
> format.
>
> This is a SBOM for the tarball creation, not for curl.
Well, what is the tarball but the tarball of "curl"? SPDX can provide
information on the files in the tarball as well as the files used to create the
tarball. How much you provide is up to you, but the more information available,
the more possibilities there are for others to use it.
> I rather start with something basic and simple, as we don't even know if
> anyone cares or wants this information.
That makes sense. SPDX is definitely heavier weight than a few version numbers
in an .md file. But, a lot more useful, too.
> > Even running "reuse spdx" in the curl tree (the same tool that's keeping
> > curl in REUSE compliance in that CI build) will output a SPDX file for
> > curl.
>
> I tried it just now. It produces 86,000 lines of output! And yet I can't
> find a lot of helpful content within the output for our purpose here.
That example was just the first one I thought of that you might already have on
your system (due to the work in getting REUSE compliance some time ago). It
doesn't solve the problem at hand, but it shows what SPDX looks like and it
could still be integrated into a final curl SPDX file provided with each
release if we wanted it to. Few projects provide SPDX files right now which is why
companies using SPDX only for license compatibility checking need to run "reuse
spdx" on the source code themselves. But if curl provided that SPDX file already
filled in with each release, including the additional information on the
dependencies used to create the tar ball itself, that single file can serve two
purposes. Even more purposes, actually, since it could be additionally be used
for security scanning, such as finding that curl used a back-door autoconf m4
macro found only in the tarball (if that ends up happening one day).
> It does not seem like a suitable tool for this.
Agreed. It just gives a flavour of one of the kinds of things a SPDX file can
provide, but could become part of a solution.
A tool that might actually do what you want is
https://pypi.org/project/distro2sbom/ That creates a SPDX file listing all the
packages in the current system (e.g. Debian packages on Debian). You probably
don't want to run that on your personal system (way too many irrelevant
packages), but it could be run from a minimal container used just to create a
tarball to provide a more easily reproducible set of packages for others to
fall on want to completely reproduce that build process.
Dan
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2024-04-02