curl / Mailing Lists / curl-users / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: Does curl do anything special with non-ASCII characters in HTTP header values?

From: Dan Fandrich via curl-users <>
Date: Sat, 19 Sep 2020 08:59:42 -0700

On Sat, Sep 19, 2020 at 12:19:57AM +0000, Steve Perkins via curl-users wrote:
> When I send that character as a header value in Postman, or from a
> server-to-server HTTP request from another application, then it is received in
> mangled form. However, it comes through perfectly fine when I test like this:
> curl --request POST --header "my-header: ø" --data "dummy payload" http://
> localhost:8080/my-endpoint

To curl, the value of the --header option is an opaque series of bytes which it
sends along. The bytes you pass it in this case are in whatever your local
character set is, probably UTF-8, but it might be ISO 8859/1 or something else.
Whatever it is, it sounds like that's what the server wants.

> I should probably refactor this application, to not take potentially-non-ASCII
> values from headers. Or perhaps just manually encode and decode on both ends.
> But regardless, I’m just dying to understand why this works with curl! Could
> anyone offer any insight on what curl might be doing with header values… such
> that non-ASCII characters transmit cleanly, even when they don’t from other
> clients?

Other clients are probably sending other bytes for that header. Are they
hard-coded in a source code file? Maybe that source code file is in a different
character set than the command-line you use. The program may need to perform a
character set conversion (using iconv(3) or similar) before sending the bytes
to the server.

Received on 2020-09-19