curl-library

From: Demi Obenour via curl-library <curl-library_at_cool.haxx.se> Date: Wed, 20 Sep 2017 11:16:20 -0400 · > i am C++ developer and i am having trouble modifying this example to use

--
  / daniel.haxx.se
------------------------------
Message: 2
Date: Tue, 19 Sep 2017 15:40:14 +0100
From: Patrick Monnerat <patrick_at_monnerat.net>
To: libcurl development <curl-library_at_cool.haxx.se>
Subject: nul bytes in form field names
Message-ID: <5ff006dc-a1da-f6a2-6310-f6c11ac94f62_at_monnerat.net>
Content-Type: text/plain; charset=utf-8; format=flowed
According to standards, nul bytes are allowed and significant characters
in form field names. This practice is however discouraged since it may
lead to header parsing errors or be understood by servers as an attack
attempt. In addition, header syntaxes containing nul bytes are now
deprecated [1].
There is obviously an intention to support this feature in curl: apart
from limiting the length of a non nul-terminated name string, the name
length parameter of curl_mime_name() is supposed to allow it. Is it the
primary reason for this parameter ? Do we really want to allow such zero
bytes in names.
Currently, curl's mime API wrongly escapes those bytes as '\0'. This
should be changed as this won't have the expected result when parsed by
the server. There is only one alternative to transmitting a binary zero
byte in a header: Use of RFC 2047 escaping [2].
Both methods (escaping or not) have their pros and cons:
No escaping pros:
- Compatibility with the old form API (that currently does not work
properly anymore, BTW).
- No need of RFC 2047 support in the server.
Cons:
- May fool servers (parsing errors, false positive attack detection).
- Needs special support in libcurl's code (no curl_slist).
Escaping pros:
- No nul bytes in text (may use curl_slist).
- No "control" character in stream.
- No triggering of strange server behaviors or exploits.
Cons:
- The server must support it.
- The encoding character set must be known.
- The header length increases.
Your comments are welcomed about which solution to implement:
a) remove nul byte support.
b) include nul bytes in output stream.
c) encode words containing nul bytes as RFC 2047.
Thanks in advance.
[1] = https://tools.ietf.org/html/rfc2822#section-4.1
[2] = https://www.ietf.org/rfc/rfc2047.txt
------------------------------
Message: 3
Date: Tue, 19 Sep 2017 16:55:42 +0200 (CEST)
From: Daniel Stenberg <daniel_at_haxx.se>
To: Patrick Monnerat via curl-library <curl-library_at_cool.haxx.se>
Subject: Re: nul bytes in form field names
Message-ID: <alpine.DEB.2.20.1709191653270.7744_at_tvnag.unkk.fr>
Content-Type: text/plain; charset=US-ASCII; format=flowed
On Tue, 19 Sep 2017, Patrick Monnerat via curl-library wrote:
> According to standards, nul bytes are allowed and significant characters
in
> form field names. This practice is however discouraged since it may lead
to
> header parsing errors or be understood by servers as an attack attempt. In
> addition, header syntaxes containing nul bytes are now deprecated [1].
I don't see any good reason to support that edge case. I don't think I've
ever
encountered such a use case in the real world and I've never seen a
curl/libcurl user who wanted it or used it.
I think it is better and clearer if we explictly do not support it, and
document this limitation.
--
  / daniel.haxx.se
------------------------------
Message: 4
Date: Tue, 19 Sep 2017 17:07:15 +0100
From: Patrick Monnerat <patrick_at_monnerat.net>
To: libcurl development <curl-library_at_cool.haxx.se>
Subject: Re: nul bytes in form field names
Message-ID: <2b66f8d0-f6e3-b483-e434-3d295d72cfa1_at_monnerat.net>
Content-Type: text/plain; charset=windows-1252; format=flowed
On 09/19/2017 03:55 PM, Daniel Stenberg wrote:
>> According to standards, nul bytes are allowed and significant
>> characters in form field names. This practice is however discouraged
>> since it may lead to header parsing errors or be understood by
>> servers as an attack attempt. In addition, header syntaxes containing
>> nul bytes are now deprecated [1].
>
> I don't see any good reason to support that edge case.
Thanks for your reply Daniel,
I'll remove this support and update the documentation.
Please note this will affect the form API compatibility.
Do we keep the length parameter for names or suppress it and require
names are always nul-terminated ?
If we keep it, I'll check in curl_mime_name() for invalid names.
> I don't think I've ever encountered such a use case in the real world
> and I've never seen a curl/libcurl user who wanted it or used it.
>
The only place I've ever seen it is in the pre Apr 2011 formdata.c
_FORM_DEBUG enabled code :-)
Patrick
------------------------------
Message: 5
Date: Wed, 20 Sep 2017 00:18:38 +0200 (CEST)
From: Daniel Stenberg <daniel_at_haxx.se>
To: Patrick Monnerat via curl-library <curl-library_at_cool.haxx.se>
Subject: Re: nul bytes in form field names
Message-ID: <alpine.DEB.2.20.1709200016120.7744_at_tvnag.unkk.fr>
Content-Type: text/plain; charset=US-ASCII; format=flowed
On Tue, 19 Sep 2017, Patrick Monnerat via curl-library wrote:
> Please note this will affect the form API compatibility.
Mostly theoretically, right? I mean the old API didn't really send names
properly with embedded zeroes either, did it?
> Do we keep the length parameter for names or suppress it and require names
> are always nul-terminated ?
I vote for requiring nul termination, just to make the API a little bit
simpler.
> The only place I've ever seen it is in the pre Apr 2011 formdata.c
> _FORM_DEBUG enabled code :-)
I don't think we should count that as real use! =)
--
  / daniel.haxx.se
------------------------------
Message: 6
Date: Wed, 20 Sep 2017 01:05:06 +0100
From: Patrick Monnerat <patrick_at_monnerat.net>
To: libcurl development <curl-library_at_cool.haxx.se>
Subject: Re: nul bytes in form field names
Message-ID: <a3f45986-67e9-082a-1a35-a583e1a6528a_at_monnerat.net>
Content-Type: text/plain; charset=windows-1252; format=flowed
On 09/19/2017 11:18 PM, Daniel Stenberg wrote:
>> Please note this will affect the form API compatibility.
>
> Mostly theoretically, right? I mean the old API didn't really send
> names properly with embedded zeroes either, did it?
I never tested it in real conditions, but considering the now defunct
_FORM_DEBUG code, it probably worked some day.
I'll reject names with embedded null bytes (CURL_FORMADD_NULL) in
curl_formadd(). Then document this deviance.
>
>> Do we keep the length parameter for names or suppress it and require
>> names are always nul-terminated ?
>
> I vote for requiring nul termination, just to make the API a little
> bit simpler.
Agreed. Thanks for hints and advice.
>
>> The only place I've ever seen it is in the pre Apr 2011 formdata.c
>> _FORM_DEBUG enabled code :-)
>
> I don't think we should count that as real use! =)
This was just a Precambrian code reference (the earth was still flat at
this time !)
Patrick
------------------------------
Subject: Digest Footer
_______________________________________________
curl-library mailing list
curl-library_at_cool.haxx.se
https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-library
------------------------------
End of curl-library Digest, Vol 145, Issue 24
*********************************************

curl-library

Re: Form data