Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bufq+dynhds #10720

Closed
wants to merge 2 commits into from
Closed

Bufq+dynhds #10720

wants to merge 2 commits into from

Conversation

icing
Copy link
Contributor

@icing icing commented Mar 9, 2023

Adding 2 pieces to curl's infrastructure for use in various places:

Adding bufq:

  • at init() time configured to hold up to n chunks of m bytes each.
  • various methods for reading from and writing to it.
  • peek support to get access to buffered data without copy
  • pass support to allow buffer flushing on write if it becomes full
  • use case: IO buffers for dynamic reads and writes that do not blow up
  • distinct from dynbuf in that:
    • it maintains a read position
    • writes on a full bufq return CURLE_AGAIN instead of nuking itself
  • Init options:
    • SOFT_LIMIT: allow writes into a full bufq
    • NO_SPARES: free empty chunks right away
    • a bufc_pool that can keep a number of spare chunks to
      be shared between different bufq instances

Adding dynhds:

  • a straightforward list of name+value pairs as used for HTTP headers
  • headers can be appended dynamically
  • headers can be removed again
  • headers can be replaced
  • headers can be looked up
  • http/1.1 formatting into a dynbuf
  • configured at init() with limits on header counts and total string sizes
  • use case: pass a HTTP request or response around without being version specific
  • express a HTTP request without a curl easy handle (used in h2 proxy tunnels)
  • future extension possibilities:
    • conversions of dynhds to nghttp2/nghttp3 name+value arrays

        Adding `bufq`:
        - at init() time configured to hold up to `n` chunks of `m` bytes each.
        - various methods for reading from and writing to it.
        - `peek` support to get access to buffered data without copy
        - `pass` support to allow buffer flushing on write if it becomes full
        - use case: IO buffers for dynamic reads and writes that do not blow up
        - distinct from `dynbuf` in that:
          - it maintains a read position
          - writes on a full bufq return CURLE_AGAIN instead of nuking itself
        - Init options:
          - SOFT_LIMIT: allow writes into a full bufq
          - NO_SPARES: free empty chunks right away
        - a `bufc_pool` that can keep a number of spare chunks to
          be shared between different `bufq` instances

        Adding `dynhds`:
        - a straightforward list of name+value pairs as used for HTTP headers
        - headers can be appended dynamically
        - headers can be removed again
        - headers can be replaced
        - headers can be looked up
        - http/1.1 formatting into a `dynbuf`
        - configured at init() with limits on header counts and total string sizes
        - use case: pass a HTTP request or response around without being version specific
        - express a HTTP request without a curl easy handle (used in h2 proxy tunnels)
        - future extension possibilities:
          - conversions of `dynhds` to nghttp2/nghttp3 name+value arrays
@bagder
Copy link
Member

bagder commented Mar 30, 2023

Thanks!

@bagder bagder closed this in 61f52a9 Mar 30, 2023
dfandrich added a commit that referenced this pull request Mar 31, 2023
Values greater than 125 have special meanings, so cap it there.  Unit
tests and lib tests use the number of failures as the return code, so a
large number of failures (such as test 2601 as a torture test) can
exceed this causing the test to be erroneously reported as having
failed.

Ref: #10720
bch pushed a commit to bch/curl that referenced this pull request Jul 19, 2023
Adding `bufq`:
- at init() time configured to hold up to `n` chunks of `m` bytes each.
- various methods for reading from and writing to it.
- `peek` support to get access to buffered data without copy
- `pass` support to allow buffer flushing on write if it becomes full
- use case: IO buffers for dynamic reads and writes that do not blow up
- distinct from `dynbuf` in that:
  - it maintains a read position
  - writes on a full bufq return CURLE_AGAIN instead of nuking itself
- Init options:
  - SOFT_LIMIT: allow writes into a full bufq
  - NO_SPARES: free empty chunks right away
- a `bufc_pool` that can keep a number of spare chunks to
  be shared between different `bufq` instances

Adding `dynhds`:
- a straightforward list of name+value pairs as used for HTTP headers
- headers can be appended dynamically
- headers can be removed again
- headers can be replaced
- headers can be looked up
- http/1.1 formatting into a `dynbuf`
- configured at init() with limits on header counts and total string
  sizes
- use case: pass a HTTP request or response around without being version
  specific
- express a HTTP request without a curl easy handle (used in h2 proxy
  tunnels)
- future extension possibilities:
  - conversions of `dynhds` to nghttp2/nghttp3 name+value arrays

Closes curl#10720
bch pushed a commit to bch/curl that referenced this pull request Jul 19, 2023
Values greater than 125 have special meanings, so cap it there.  Unit
tests and lib tests use the number of failures as the return code, so a
large number of failures (such as test 2601 as a torture test) can
exceed this causing the test to be erroneously reported as having
failed.

Ref: curl#10720
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

2 participants