Re: --etag-compare vs. --time-cond

From: Timothe Litt <litt_at_acm.org>
Date: Sat, 4 Mar 2023 17:41:44 -0500

On 04-Mar-23 17:10, Paul Gilmartin via curl-users wrote:
> On 3/4/23 13:59:42, Timothe Litt via curl-users wrote:
>
>> On 04-Mar-23 15:12, Paul Gilmartin via curl-users wrote:
>>> If I specify both --etag-compare and --time-cond and they disagree on
>>> whether a file is up-to-date, which wins?
>>>
>>> (It might depend on the HTTPD)
>>
>> RFC 7232 <https://www.rfc-editor.org/rfc/rfc7232> Section3.3
>> specifies that the E-Tag wins for GET class requests.
>
> Thanks.  So in a test I (caused to be) supplied:
>> If-Modified-Since: Sat, 04 Mar 2023 21:52:30 GMT
>> If-None-Match: wombat  # I'm guessing this does *NOT* match
>
> ... and got:
> < HTTP/1.1 304 Not Modified
> < Last-Modified: Thu, 17 Nov 2022 02:01:41 GMT
>
> ... apparently the If-None-Match: did not dominate.  But this man mean
> only that the HTTPD violates RFC 7232.
>
Section 6 also discusses precedence and evaluation order.

As I said, read the whole RFC.

Your test isn't valid.  An E-Tag validator is only meaningful to the
server that sent it.

The origin server defines the format of a validator (the E-Tag and
If-(None)-Match headers have one or more).

You're only supposed to send a server validators that it originated. 
What may be happening here is that the server sees that "wombat" could
not have been generated by it, and therefore ignores the If-None-Match -
falling back to the If-Modified-Since.

For example, consider a validator that internally looks like
<mtime>-<size>-<expires>.  If the server doesn't keep a side database of
validators, it might try to parse it, and since "wombat" isn't in that
format, give up.

The RFC doesn't discuss that case.  It's reasonable for the server to
decide that a client sent a validator that came from another server (or
an older version of itself) and rather than fail the request, ignore
it.  Since you conveniently provide an alternative validator
(If-Modified-Since), it uses that.

Validators can be anything from a hash of file data, to some combination
of Inode, mtime, size, to a database index, GUID, or...  They're opaque
to the client.  Designing one is tricky, especially when clusters or
network file systems are involved.  In theory, one could have an EDC to
detect tampering.

IF the validator originated with the server that receives it, the RFC
applies.  Otherwise, all bets are off.

Also note that E-Tags are an optimization - the client must always be
prepared to get the full response.  Reasons can include a server
restored from a backup, clock skew, resource limits, and many others.

For a better test, you could use a server that doesn't include mtime in
ETags.  Then you could modify the file, set the mtime back and see what
happens.  But you're breaking rules to do it. If your server is Apache
HTTPD, see the FileEtag directive.

"It isn't nice to fool mother nature" applies to servers, too.

Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2023-03-04