Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: An API for extracing (HTTP) headers?
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Timothe Litt <litt_at_acm.org>
Date: Tue, 22 Mar 2022 13:15:12 -0400
On 22-Mar-22 12:49, Daniel Stenberg wrote:
> On Tue, 22 Mar 2022, Timothe Litt wrote:
>
>> A single API call can guarantee a consistent result. Since you allow
>> this to be called while headers are arriving, multiple calls may
>> return different counts - presumably only increasing.
>
> That's true. But someone polling header info while headers are still
> incoming should hopefully be aware of that not all headers have
> arrived yet...
>
How? Unless they wait to end of transfer - in which case, why poll? Or
there's a flag in what's returned...
>> OK. But it's only the structs -- 1 (for the end marker) + #headers
>> vs. 1. You're holding the names and values, which hopefully are the
>> large items.
>
> Yes, but also since we would then return a dynamic set of a variable
> amount of structs we would have to allocate that memory dynamically.
> Which we can avoid now, since we always need the same fixed amount.
> Simpler.
>
For the library, yes. For the user, perhaps not.
>> One more thought - it might be useful to have a flag bit that
>> indicates when the last header has been received.
>
> I had a flag like that in the "spec" for a while and my first
> implementation did, but I removed it later when I added support for
> the whole "chain" of requests as it then got a little complicated.
>
> There can be "the last header received" several times during a
> transfer and I couldn't decide how to signal that proper!
>
There's only one "last header received" for a given response. Your chain
results from a series of (requests and) responses. Each response can
have a last_header_received flag. (You call response 'request" in the
API, which is somewhat confusing. The request headers are outgoing...)
Anyhow, the API selects the desired set of response headers; the flag
applies to that set. I'd put it in the 'origin' element of the struct
(and rename it 'flags').
For the case of multiple responses (e.g. when you follow a redirect
chain), a simple approach would be to also have a "last response" flag,
which is set when headers are retrieved for the last request. I think
the status code (which comes before any header) tells you whether you'll
be making another request. It certainly does for redirects.
I guess if you support chunked encoding, there can be headers in the
trailer, so in that (rather rare, and removed in http/2) case, you
couldn't set 'last header" until the trailer arrives.
Are there other cases?
Received on 2022-03-22
Date: Tue, 22 Mar 2022 13:15:12 -0400
On 22-Mar-22 12:49, Daniel Stenberg wrote:
> On Tue, 22 Mar 2022, Timothe Litt wrote:
>
>> A single API call can guarantee a consistent result. Since you allow
>> this to be called while headers are arriving, multiple calls may
>> return different counts - presumably only increasing.
>
> That's true. But someone polling header info while headers are still
> incoming should hopefully be aware of that not all headers have
> arrived yet...
>
How? Unless they wait to end of transfer - in which case, why poll? Or
there's a flag in what's returned...
>> OK. But it's only the structs -- 1 (for the end marker) + #headers
>> vs. 1. You're holding the names and values, which hopefully are the
>> large items.
>
> Yes, but also since we would then return a dynamic set of a variable
> amount of structs we would have to allocate that memory dynamically.
> Which we can avoid now, since we always need the same fixed amount.
> Simpler.
>
For the library, yes. For the user, perhaps not.
>> One more thought - it might be useful to have a flag bit that
>> indicates when the last header has been received.
>
> I had a flag like that in the "spec" for a while and my first
> implementation did, but I removed it later when I added support for
> the whole "chain" of requests as it then got a little complicated.
>
> There can be "the last header received" several times during a
> transfer and I couldn't decide how to signal that proper!
>
There's only one "last header received" for a given response. Your chain
results from a series of (requests and) responses. Each response can
have a last_header_received flag. (You call response 'request" in the
API, which is somewhat confusing. The request headers are outgoing...)
Anyhow, the API selects the desired set of response headers; the flag
applies to that set. I'd put it in the 'origin' element of the struct
(and rename it 'flags').
For the case of multiple responses (e.g. when you follow a redirect
chain), a simple approach would be to also have a "last response" flag,
which is set when headers are retrieved for the last request. I think
the status code (which comes before any header) tells you whether you'll
be making another request. It certainly does for redirects.
I guess if you support chunked encoding, there can be headers in the
trailer, so in that (rather rare, and removed in http/2) case, you
couldn't set 'last header" until the trailer arrives.
Are there other cases?
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
- application/pgp-signature attachment: OpenPGP digital signature