Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Issue with MAX_COOKIE_HEADER_LEN
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Dmitry Karpov via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 19 May 2023 19:13:17 +0000
I heard too many often from cloud folks something like “we control the server and your client should do this…”.
So, I think having an option allowing to override MAX_COOKIE_HEADER_LEN is a good idea allowing to meet different client/server needs.
It is much better than making a custom libcurl build, and potentially there may be cases when different servers may require different limits.
And I also agree with Daniel that verbose output should have some indication that a custom header cookie limit is used.
The 8k limit seems like a right default value, reflecting commonly accepted use cases, so any deviations from it may cause surprising errors,
and giving some hints about potential errors when non-default values are used will be helpful.
Thanks,
Dmitry Karpov
From: curl-library <curl-library-bounces_at_lists.haxx.se> On Behalf Of Henrik Holst via curl-library
Sent: Friday, May 19, 2023 7:56 AM
To: Daniel Stenberg <daniel_at_haxx.se>
Cc: Henrik Holst <henrik.holst_at_millistream.com>; libcurl development <curl-library_at_lists.haxx.se>
Subject: [EXTERNAL] Re: Issue with MAX_COOKIE_HEADER_LEN
Yeah it is ofc a complex situation, the web servers themselves does not help much either since basically none of them responds with a proper 413 and instead send a generic 400.
When it comes to your #1 to #3 I think that all we can do there is make sure that the documentation for the setting is very clear on that this must match the setting at the server side. However since this is configurable on the server side there can very well be servers out there with a size < 8kb which means that curl could currently fail in the exact same way with no indication on why. AFAIK e.g nginx uses the page size by default so on many systems it currently have a default size of 4k.
/HH
Den fre 19 maj 2023 kl 16:30 skrev Daniel Stenberg <daniel_at_haxx.se<mailto:daniel_at_haxx.se>>:
On Fri, 19 May 2023, Henrik Holst wrote:
> I did some more digging around on what the various http servers uses and
> while it is 8k on most servers (apache, nginx) it is 16k on IIS but it is
> also configurable on all http servers. So it can be set to be higher on
> apache, nginx, IIS and most other http servers out there. So having it be
> configurable in libcurl as well does IMHO not sound that outlandish.
No, it is perfectly logical. It's just that this is one of those tough ones.
1. basically no user will understand when to change such an option
2. setting it to a "too high" value and using that size, risks making your
curl transfers to fail with rather inexplainable failure modes
3. A user might change the value and find it works fine *for years* (perhaps
because they never reach the limit) and then they try another server or
even worse: the primary server updates and then suddenly without anything
in curl or the application having changed, the transfers fail.
But yes: by *not* offering such an option we also have users that cannot make
their application perform correctly because libcurl blocks it from doing what
it needs to do simply because there are *other* servers out there that will
not appreciate a request like that.
Maybe a small thing to help would be to make curl use a verbose output when it
sends a cookie header that is longer than the current limit? That at least
could be a clue for users who get a 400 back without understanding why...
--
/ daniel.haxx.se<http://daniel.haxx.se>
| Commercial curl support up to 24x7 is available!
| Private help, bug fixes, support, ports, new features
| https://curl.se/support.html
Date: Fri, 19 May 2023 19:13:17 +0000
I heard too many often from cloud folks something like “we control the server and your client should do this…”.
So, I think having an option allowing to override MAX_COOKIE_HEADER_LEN is a good idea allowing to meet different client/server needs.
It is much better than making a custom libcurl build, and potentially there may be cases when different servers may require different limits.
And I also agree with Daniel that verbose output should have some indication that a custom header cookie limit is used.
The 8k limit seems like a right default value, reflecting commonly accepted use cases, so any deviations from it may cause surprising errors,
and giving some hints about potential errors when non-default values are used will be helpful.
Thanks,
Dmitry Karpov
From: curl-library <curl-library-bounces_at_lists.haxx.se> On Behalf Of Henrik Holst via curl-library
Sent: Friday, May 19, 2023 7:56 AM
To: Daniel Stenberg <daniel_at_haxx.se>
Cc: Henrik Holst <henrik.holst_at_millistream.com>; libcurl development <curl-library_at_lists.haxx.se>
Subject: [EXTERNAL] Re: Issue with MAX_COOKIE_HEADER_LEN
Yeah it is ofc a complex situation, the web servers themselves does not help much either since basically none of them responds with a proper 413 and instead send a generic 400.
When it comes to your #1 to #3 I think that all we can do there is make sure that the documentation for the setting is very clear on that this must match the setting at the server side. However since this is configurable on the server side there can very well be servers out there with a size < 8kb which means that curl could currently fail in the exact same way with no indication on why. AFAIK e.g nginx uses the page size by default so on many systems it currently have a default size of 4k.
/HH
Den fre 19 maj 2023 kl 16:30 skrev Daniel Stenberg <daniel_at_haxx.se<mailto:daniel_at_haxx.se>>:
On Fri, 19 May 2023, Henrik Holst wrote:
> I did some more digging around on what the various http servers uses and
> while it is 8k on most servers (apache, nginx) it is 16k on IIS but it is
> also configurable on all http servers. So it can be set to be higher on
> apache, nginx, IIS and most other http servers out there. So having it be
> configurable in libcurl as well does IMHO not sound that outlandish.
No, it is perfectly logical. It's just that this is one of those tough ones.
1. basically no user will understand when to change such an option
2. setting it to a "too high" value and using that size, risks making your
curl transfers to fail with rather inexplainable failure modes
3. A user might change the value and find it works fine *for years* (perhaps
because they never reach the limit) and then they try another server or
even worse: the primary server updates and then suddenly without anything
in curl or the application having changed, the transfers fail.
But yes: by *not* offering such an option we also have users that cannot make
their application perform correctly because libcurl blocks it from doing what
it needs to do simply because there are *other* servers out there that will
not appreciate a request like that.
Maybe a small thing to help would be to make curl use a verbose output when it
sends a cookie header that is longer than the current limit? That at least
could be a clue for users who get a 400 back without understanding why...
--
/ daniel.haxx.se<http://daniel.haxx.se>
| Commercial curl support up to 24x7 is available!
| Private help, bug fixes, support, ports, new features
| https://curl.se/support.html
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-05-19