New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
libcurl violates RFC7230 when constructing a proxy request with an explicit port ":80" in the URL #6769
Comments
The S3 uploader will currently always include an explicit port number within URIs (e.g. "http://mybucket.s3.us-east-1.amazonaws.com:80") even when using the default HTTP port 80. This is not incorrect, but unfortunately triggers a bug within libcurl (see curl/curl#6769) that causes it to construct requests that will be rejected if they happen to pass through HAProxy. Work around this libcurl bug by omitting an explicit port number from the constructed URI when the default port is used. Signed-off-by: Michael Brown <mbrown@fensystems.co.uk>
Let me address the three suggestions one by one for completeness:
This doesn't work on the Internet. Servers out there will break on port numbers for default ports in the
I fear that this will end up in the same bucket as the above, since the
I believe this alternative has the highest chance of working with the least amount of friction and problems. |
It struck me that the fix is really easy! PR coming up in a jiffy. |
To make sure the Host: header and the URL provide the same authority portion when sent to the proxy, strip the default port number from the URL if one was provided. Reported-by: Michael Brown Fixes #6769 Closes #[fill in]
I did this
With anything listening locally on port 3128 (e.g.
nc -l 3128
), issue a request that uses a proxy server and specifies port 80 explicitly within the request URL:Note that:
-x
option to specify the use of a proxy serverexample.org:80
(i.e. including the port number)Host
header contains the valueexample.org
(i.e. not including the port number)This violates RFC 7230 section 5.4, which states in part that
This discrepancy between target URI authority portion and
Host
header causes a failure when the request happens to pass through HAProxy, which will report a400 Bad Request
error due to the mismatch.Within curl, the
Host
header is constructed to explicitly omit the port number if it matches the default (80 for HTTP or 443 for HTTPS):curl/lib/http.c
Lines 2108 to 2123 in 03c8cef
and the target URI is constructed to exclude the userinfo subcomponent but will leave the port number present even if it would be omitted from the
Host
header:curl/lib/http.c
Lines 2176 to 2188 in 03c8cef
For reference, the relevant code within HAProxy that rejects the mismatched request target URI and
Host
header seems to be: https://github.com/haproxy/haproxy/blob/19d14710e941a366afd5b4ff8720090c011c83c1/src/h1.c#L871-L896I expected the following
curl should construct a request that conforms to RFC 7230. This could be achieved by any of:
Host
header unconditionally, orHost
header when issuing a request via a proxy, orCurl_http_host()
I am happy to put together a pull request if a maintainer could indicate which of the above would be the preferred approach.
curl/libcurl version
operating system
Linux 5.10.16-200.fc33.x86_64 #1 SMP Sun Feb 14 03:02:32 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: