Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confused by --compressed documentation #11091

Closed
dbanty opened this issue May 8, 2023 · 5 comments
Closed

Confused by --compressed documentation #11091

dbanty opened this issue May 8, 2023 · 5 comments

Comments

@dbanty
Copy link

dbanty commented May 8, 2023

At https://curl.se/docs/manpage.html#--compressed the document states that "Headers are not modified" which led me to believe that if I did not pass an Accept-Encoding that no compression would actually happen. However, it seems that if no Accept-Encoding is passed, then a default of "deflate, gzip, br, zstd" will be set.

On a second read, with this knowledge in mind, it seems more clear that the header will be added if missing, but not modified if present. Maybe there is a rewording of this paragraph that better indicates the behavior? My suggestion:

(HTTP) Request a compressed response using one of the algorithms curl supports, and automatically decompress the content. Existing headers are not modified, but required headers (like Accept-Encoding) will be added if missing.

Someone who is more familiar with the feature could probably explain it better 😅

@bagder
Copy link
Member

bagder commented May 8, 2023

Headers are not modified

This (comes from #5217) means that curl does not remove any headers from the response when it decompresses the content and passes on/saves the headers. It could possibly be clarified further, perhaps to say "response headers".

@bagder
Copy link
Member

bagder commented May 8, 2023

Maybe this makes it clearer?

diff --git a/docs/cmdline-opts/compressed.d b/docs/cmdline-opts/compressed.d
index 97ab1b10b..4921bdb47 100644
--- a/docs/cmdline-opts/compressed.d
+++ b/docs/cmdline-opts/compressed.d
@@ -8,10 +8,14 @@ Example: --compressed $URL
 See-also: compressed-ssh
 Added: 7.10
 Multi: boolean
 ---
 Request a compressed response using one of the algorithms curl supports, and
-automatically decompress the content. Headers are not modified.
+automatically decompress the content.
+
+Response headers are not modified when saved, so if they are "interpreted"
+separately again at a later point they might appear to be saying that the
+content is (still) compressed; while in fact it has already been decompressed.
 
 If this option is used and the server sends an unsupported encoding, curl will
 report an error. This is a request, not an order; the server may or may not
 deliver data compressed.

@dbanty
Copy link
Author

dbanty commented May 9, 2023

Oh that's much clearer, thanks!

@bagder
Copy link
Member

bagder commented May 9, 2023

I'll turn into a PR then!

bagder added a commit that referenced this issue May 9, 2023
@bagder bagder closed this as completed in 6708498 May 9, 2023
bch pushed a commit to bch/curl that referenced this issue Jul 19, 2023
@shvchk
Copy link

shvchk commented Sep 13, 2023

Just to confirm what's exactly going on for future lurkers:

> nc -l 9999 & curl localhost:9999 -m 1
GET / HTTP/1.1
Host: localhost:9999
User-Agent: curl/7.88.1
Accept: */*
> nc -l 9999 & curl localhost:9999 -m 1 --compressed
GET / HTTP/1.1
Host: localhost:9999
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: deflate, gzip, br, zstd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

3 participants