Re: HTTP response header access in curl

From: Timothe Litt <litt_at_acm.org>
Date: Thu, 17 Mar 2022 19:11:31 -0400

On 17-Mar-22 19:00, Daniel Stenberg via curl-users wrote:
> On Thu, 17 Mar 2022, Daniel Stenberg via curl-users wrote:
>
>> $ curl -sw '%{header_json}\n' curl.se | jq
>
> As was quickly pointed out to me on IRC, that format will not work for
> duplicated headers - and such are still valid and frequently used. I
> think header values therefor need to be made JSON arrays and thus
> rather look like this:
>
> {"Date":["Tue, 09 Nov 2010 14:49:00 GMT"],
> "Server":["test-server/fake"],
> "Last-Modified":["Tue, 13 Jun 2000 12:10:00 GMT"],
> "ETag":["\"21025-dc7-39462498\""],
> "Accept-Ranges":["bytes"],
> "Set-Cookie":["firstcookie=want1; path=/","2cookie=want2;
> path=/","cookie3=want3; path=/"],
> "Content-Length":["6"],
> "Connection":["close"]
> }

Does the JSON have to be an exact representation of the received
headers?  Or semantically equivalent?

Duplicated headers are to be interpreted as if a single header with a
comma-separated list for values.

This comes down to how much interpretation of the values is
appropriate.  They could be parsed down to subfield level - that isn't
the current proposal.  It's arguably easier for consumers to have one
format than two.

So does curl:

 1. Provide values in an array as delivered (Daniel's latest)
 2. Merge all duplicates into a comma separated list
 3. Split values in each header on commas, and provide an array that's
    the union of all segments

1 is easy to implement, looks like what's received.  But the consumer
has to join, or join & split

2 gives the consumer one value/header comma separated

3 is the most (but not much) work, makes life simplest for the
consumer.  But is furthest from what's received.

See RFC2616:

Multiple message-header fields with the same field-name MAY be present
in a message if and only if the entire field-value for that header field
is defined as a comma-separated list [i.e., #(values)]. It MUST be
possible to combine the multiple header fields into one "field-name:
field-value" pair, without changing the semantics of the message, by
appending each subsequent field-value to the first, each separated by a
comma. The order in which header fields with the same field-name are
received is therefore significant to the interpretation of the combined
field value, and thus a proxy MUST NOT change the order of these field
values when a message is forwarded


Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-users
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Received on 2022-03-18