Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Limit the URL size more?
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Jeffrey Walton via curl-library <curl-library_at_lists.haxx.se>
Date: Sun, 14 Dec 2025 08:44:44 -0500
On Sun, Dec 14, 2025 at 6:09 AM Daniel Stenberg via curl-library
<curl-library_at_lists.haxx.se> wrote:
>
> Hey friends,
>
> Yesterday we received a security report [1] on curl that identified a possible
> performance issue. We deemed it not a security problem, but it still triggered
> this question:
>
> When a user sets a URL in libcurl it allows up to an 8 megabyte string. If
> that URL uses a maximum amount of "../" occurances, all those need to be
> "optimized away" in the dotdot normalization phase of the URL parsing.
>
> That makes 2,666,663 three-byte sequences to remove in the worst possible
> case.
When using cURL as a user agent or client, I think this is a layering
violation. cURL should not be fiddling with URLs in general, and dots
in particular since a dot is not something that needs to be encoded in
HTML URLs anyways. It is up to the receiving server to interpret what
the user asked for in a locale-specific kind of way. The server is
the host that should canonicalize the URL.
This smells of the WHATWG trying to determine what a valid email
address is. The email folks already defined what a valid email
address is. It is not the place of the WHATWG to try to validate an
email address against its own arbitrary standards since the WHATWG
does not know how the destination mail server handles an email
address. For example, <input type="email"> will reject a valid email,
like a "plus" address of john.doe+test_at_example.com. See
<https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address>.
This is a slightly different question than asking "how long a URL
should cURL support"? I think that's a valid question.
> While asking ourselves how much faster we can make that code, I figure we
> could also open up the question:
>
> How long URL do we really need to support? In practise, a URL longer than a
> few tens of kilobytes is not likely to actually work on the internet. Most
> HTTP servers for example have a limit of somewhere around 8-9 kilobytes for
> the path component.
>
> Maybe we could save ourselves and our users some future trouble by reducing
> the max URL limit size? Maybe we can set it to 1 megabyte (or even lower)
> without any legitiate user ever noticing?
>
> Thoughts?
>
> [1] = https://hackerone.com/reports/3463608
Jeff
Date: Sun, 14 Dec 2025 08:44:44 -0500
On Sun, Dec 14, 2025 at 6:09 AM Daniel Stenberg via curl-library
<curl-library_at_lists.haxx.se> wrote:
>
> Hey friends,
>
> Yesterday we received a security report [1] on curl that identified a possible
> performance issue. We deemed it not a security problem, but it still triggered
> this question:
>
> When a user sets a URL in libcurl it allows up to an 8 megabyte string. If
> that URL uses a maximum amount of "../" occurances, all those need to be
> "optimized away" in the dotdot normalization phase of the URL parsing.
>
> That makes 2,666,663 three-byte sequences to remove in the worst possible
> case.
When using cURL as a user agent or client, I think this is a layering
violation. cURL should not be fiddling with URLs in general, and dots
in particular since a dot is not something that needs to be encoded in
HTML URLs anyways. It is up to the receiving server to interpret what
the user asked for in a locale-specific kind of way. The server is
the host that should canonicalize the URL.
This smells of the WHATWG trying to determine what a valid email
address is. The email folks already defined what a valid email
address is. It is not the place of the WHATWG to try to validate an
email address against its own arbitrary standards since the WHATWG
does not know how the destination mail server handles an email
address. For example, <input type="email"> will reject a valid email,
like a "plus" address of john.doe+test_at_example.com. See
<https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address>.
This is a slightly different question than asking "how long a URL
should cURL support"? I think that's a valid question.
> While asking ourselves how much faster we can make that code, I figure we
> could also open up the question:
>
> How long URL do we really need to support? In practise, a URL longer than a
> few tens of kilobytes is not likely to actually work on the internet. Most
> HTTP servers for example have a limit of somewhere around 8-9 kilobytes for
> the path component.
>
> Maybe we could save ourselves and our users some future trouble by reducing
> the max URL limit size? Maybe we can set it to 1 megabyte (or even lower)
> without any legitiate user ever noticing?
>
> Thoughts?
>
> [1] = https://hackerone.com/reports/3463608
Jeff
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2025-12-14