Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Curl feature idea, download hash checking
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Ian Norton via curl-users <curl-users_at_lists.haxx.se>
Date: Thu, 18 Jun 2026 09:38:17 +0100
> On Fri, 12 Jun 2026, Daniel Stenberg via curl-users wrote:
> > On Fri, 12 Jun 2026, Ian Norton via curl-users wrote:
> > > curl -O URL —-hash sha256:HASH_VALUE
> >
> > Does this sound like a good idea?
> It is an idea that is often brought up but has never been materialized. Is it good? I think it depends on how it is done so that in the end people would use it.
> One additional idea is adding the hash into the URL itself somehow so that copy and pasting a single URL is enough and you can't forget the hash:
> curl -O URL https://host.com/HASHMARKER-HASH/the/path/file.tar.gz
> It also makes the URL not work at all if the download tool doesn't support the hash, which I think is a bonus. Doing the hash embedded in the URL is of course a side-track, can be worked on independently and is for sure hard to do with buy-in from others as well.
I'd not thought about url for conveying the hash, perhaps we could
abuse the URL scheme for that kind of idea (I'd initially thought of
URL#digest=hastype:hash but that would trip up a lot of shell script
languages)
another form of the argument might be:
curl sha256:HASH:https://host.com/whatever
I suppose these particular bits are just variations around what the
command line style would be for these. I think having the digest
criteria separate allows it to be applied to any resource.
> Others have suggested a similar thing before but instead of just doing a hash it would verify the *signature* provided.
Interesting! That probably ties too deeply into how the signature gets
verified (and often signatures are detached files at a different URL)
> Challenges involved include:
> What if you download to stdout and yet want it verified? Like in the classic:
> "curl https://installer.example.com | sh" case. It seems like that case is still a decent amount of current use cases this could help strengthen.
So so many people living dangerously with those :D.
> Then curl needs to download to a temporary location, check the hash and *then* pass it on. This can lead to challenges if you instead for example pipe a
> petabyte stream: "curl https://never-ending.example.com | videoplayer"
I'd not considered never-ending streams, The pipe case was why I'd
leant towards only allowing hash checking when the destination is a
disk file (-o or -O). But since you could never actually digest a
never-ending video stream anyway..?
I'm coming from a place where in my org I've already "solved" this
use-case with a python app that will exit non-zero if it can't
download, or if the thing it downloaded didn't have a matching digest.
But of course, this is a python tool and requires all the associated
deps (and isn't especially fast)
I'll try to make a curl PR and also try out the different input/arg
variations and see if I can find one that feels right.
Ian
Date: Thu, 18 Jun 2026 09:38:17 +0100
> On Fri, 12 Jun 2026, Daniel Stenberg via curl-users wrote:
> > On Fri, 12 Jun 2026, Ian Norton via curl-users wrote:
> > > curl -O URL —-hash sha256:HASH_VALUE
> >
> > Does this sound like a good idea?
> It is an idea that is often brought up but has never been materialized. Is it good? I think it depends on how it is done so that in the end people would use it.
> One additional idea is adding the hash into the URL itself somehow so that copy and pasting a single URL is enough and you can't forget the hash:
> curl -O URL https://host.com/HASHMARKER-HASH/the/path/file.tar.gz
> It also makes the URL not work at all if the download tool doesn't support the hash, which I think is a bonus. Doing the hash embedded in the URL is of course a side-track, can be worked on independently and is for sure hard to do with buy-in from others as well.
I'd not thought about url for conveying the hash, perhaps we could
abuse the URL scheme for that kind of idea (I'd initially thought of
URL#digest=hastype:hash but that would trip up a lot of shell script
languages)
another form of the argument might be:
curl sha256:HASH:https://host.com/whatever
I suppose these particular bits are just variations around what the
command line style would be for these. I think having the digest
criteria separate allows it to be applied to any resource.
> Others have suggested a similar thing before but instead of just doing a hash it would verify the *signature* provided.
Interesting! That probably ties too deeply into how the signature gets
verified (and often signatures are detached files at a different URL)
> Challenges involved include:
> What if you download to stdout and yet want it verified? Like in the classic:
> "curl https://installer.example.com | sh" case. It seems like that case is still a decent amount of current use cases this could help strengthen.
So so many people living dangerously with those :D.
> Then curl needs to download to a temporary location, check the hash and *then* pass it on. This can lead to challenges if you instead for example pipe a
> petabyte stream: "curl https://never-ending.example.com | videoplayer"
I'd not considered never-ending streams, The pipe case was why I'd
leant towards only allowing hash checking when the destination is a
disk file (-o or -O). But since you could never actually digest a
never-ending video stream anyway..?
I'm coming from a place where in my org I've already "solved" this
use-case with a python app that will exit non-zero if it can't
download, or if the thing it downloaded didn't have a matching digest.
But of course, this is a python tool and requires all the associated
deps (and isn't especially fast)
I'll try to make a curl PR and also try out the different input/arg
variations and see if I can find one that feels right.
Ian
-- PS (apologies for the mail thread breaking, I've switched to my personal account as I recoil in horror at what outlook does to plain text email) -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2026-06-18