Re: Does curl do anything special with non-ASCII characters in HTTP header values?

On Sat, Sep 19, 2020 at 12:19:57AM +0000, Steve Perkins via curl-users wrote:
> When I send that character as a header value in Postman, or from a
> server-to-server HTTP request from another application, then it is received in
> mangled form. However, it comes through perfectly fine when I test like this:
>
> curl --request POST --header "my-header: ø" --data "dummy payload" http://
> localhost:8080/my-endpoint

To curl, the value of the --header option is an opaque series of bytes which it
sends along. The bytes you pass it in this case are in whatever your local
character set is, probably UTF-8, but it might be ISO 8859/1 or something else.
Whatever it is, it sounds like that's what the server wants.

> I should probably refactor this application, to not take potentially-non-ASCII
> values from headers. Or perhaps just manually encode and decode on both ends.
> But regardless, I’m just dying to understand why this works with curl! Could
> anyone offer any insight on what curl might be doing with header values… such
> that non-ASCII characters transmit cleanly, even when they don’t from other
> clients?

Other clients are probably sending other bytes for that header. Are they
hard-coded in a source code file? Maybe that source code file is in a different
character set than the command-line you use. The program may need to perform a
character set conversion (using iconv(3) or similar) before sending the bytes
to the server.

Dan
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2020-09-19