Re: Naming ETag files?

From: Timothe Litt <litt_at_acm.org>
Date: Sun, 15 Jan 2023 07:19:32 -0500

On 14-Jan-23 20:01, Paul Gilmartin via curl-users wrote:
> On 1/14/23 13:57:19, Timothe Litt via curl-users wrote:
>> On 14-Jan-23 13:21, Daniel Stenberg via curl-users wrote:
>>>
>>>> I would like to save my --etag-compare and --etag-save (probably
>>>> the same name) files with names generated from the
>>>> --remote-header-name; perhaps as simple as
>>>> "$remote-header-name.ETag".  Is there an orderly and reliable way
>>>> to do this?
>>>
>>> No there isn't. With some clever thinking maybe we can offer this in
>>> the future!
>
> My fallback is to do a trial download to a write-protected directory.
> It promptly fails, but "--write-out %{filename_effective}" works
> regardless,
> giving the filename I need.
>
>>> Having "${variables}" work more broadly would be cool but tricky
>>> since they are all available/done at different point in times, and
>>> generally safe to get *after* the transfer is complete...
>>>
>> I'd much rather see command lines get shorter and simpler than longer.
>>
>> Rather than more command line clutter, consider that dotfiles are a
>> convenient place to store metadata.
>
> But don't move the clutter from the command line to the users' home
> directories.
> Put them in ~/.curl/*.
>
No.  That's not desirable.  I am NOT saying to (always) put the .<meta>
files in the users' home directories.

The e-tag is a property of the file, not the user.

e-tags don't belong in the user's home directory, unless that's where
the local file lives.  They belong wherever the file lives.

e.g. curl https://example.net/somewhere/foo.pl --output
/usr/local/sbin/foo.pl

The e-tag for that file belongs in /usr/local/sbin, not ~paul/.anything,
~daniel/.anything.

It doesn't matter who does the download.  The tag validates the file
data.  The tag should be stored with the file data.

If the filesystem supports extended attributes, an even better place
would be to store the etag there, since in that case, moving, copying,
renaming would automagically ensure that the etag is carried along. 
Linux xattr, Mac forks, NTFS 'streams', etc..

A dotfile is universally understood, and on most systems, hidden.  (Can
set "hidden" on windows).   That's consistent with the way curl does
things.  Which is why I suggested it.

But I'd be even happier to see e-tags stored in xattrs.  (man 7 xattr,
man 2 getxattr, man 2 setxattr)...  It's a bigger project, since
Windows, BSD, Solaris, IOS, VMS, and others each have their
equivalents.  And some don't - in which case you fall back to dotfiles...



> Command substitution enables this with no changes to curl, as in:
>     curl $( ~/.curl/optXXX.rc ) URL
>

Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2023-01-15