curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: An API for extracing (HTTP) headers?

From: Timothe Litt <litt_at_acm.org>
Date: Tue, 22 Mar 2022 13:15:12 -0400


On 22-Mar-22 12:49, Daniel Stenberg wrote:
> On Tue, 22 Mar 2022, Timothe Litt wrote:
>
>> A single API call can guarantee a consistent result.  Since you allow
>> this to be called while headers are arriving, multiple calls may
>> return different counts - presumably only increasing.
>
> That's true. But someone polling header info while headers are still
> incoming should hopefully be aware of that not all headers have
> arrived yet...
>
How?  Unless they wait to end of transfer - in which case, why poll?  Or
there's a flag in what's returned...
>> OK.  But it's only the structs -- 1 (for the end marker) + #headers
>> vs. 1.  You're holding the names and values, which hopefully are the
>> large items.
>
> Yes, but also since we would then return a dynamic set of a variable
> amount of structs we would have to allocate that memory dynamically.
> Which we can avoid now, since we always need the same fixed amount.
> Simpler.
>
For the library, yes.  For the user, perhaps not.
>> One more thought - it might be useful to have a flag bit that
>> indicates when the last header has been received.
>
> I had a flag like that in the "spec" for a while and my first
> implementation did, but I removed it later when I added support for
> the whole "chain" of requests as it then got a little complicated.
>
> There can be "the last header received" several times during a
> transfer and I couldn't decide how to signal that proper!
>
There's only one "last header received" for a given response. Your chain
results from a series of (requests and) responses. Each response can
have a last_header_received flag.  (You call response 'request" in the
API, which is somewhat confusing.  The request headers are outgoing...)
   Anyhow, the API selects the desired set of response headers; the flag
applies to that set. I'd put it in the 'origin' element of the struct
(and rename it 'flags').

For the case of multiple responses (e.g. when you follow a redirect
chain), a simple approach would be to also have a "last response" flag,
which is set when headers are retrieved for the last request.  I think
the status code (which comes before any header) tells you whether you'll
be making another request.  It certainly does for redirects.

I guess if you support chunked encoding, there can be headers in the
trailer, so in that (rather rare, and removed in http/2) case, you
couldn't set 'last header" until the trailer arrives.

Are there other cases?



-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2022-03-22