Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bundle collision when using the same proxy for two remotes #3951

Closed
TvdW opened this issue May 27, 2019 · 8 comments
Closed

bundle collision when using the same proxy for two remotes #3951

TvdW opened this issue May 27, 2019 · 8 comments
Labels
HTTP/2

Comments

@TvdW
Copy link

TvdW commented May 27, 2019

I did this

https://gist.github.com/TvdW/bea189b3926993fff584471457350d1d

* Issue another request to this URL: 'http://tvdw.eu/'
* Found bundle for host tvdw.eu: 0xc5c8d0 [can multiplex]
* Multiplexed connection found!
* Found connection 8, with 1 requests on it
* Re-using existing connection! (#8) with proxy my-proxy-host
* Send failure: Bad file descriptor
* Failed sending HTTP request
* Connection died, retrying a fresh connect
* Issue another request to this URL: 'http://tvdw.eu/'
* Found bundle for host tvdw.eu: 0xc5c8d0 [can multiplex]
* Multiplexed connection found!
* Found connection 8, with 1 requests on it
* Re-using existing connection! (#8) with proxy my-proxy-host
* Send failure: Bad file descriptor
* Failed sending HTTP request
* Connection died, retrying a fresh connect

"Bad file descriptor" after a few requests. Seems to happen when mixing HTTP/1.1 and HTTP/2 in the same multi.

It looks like the fd gets closed while a request is still waiting for a response. A new connection may be opened, reusing the same fd, making it a valid file descriptor once again. curl may then issue a new request on the reused fd, but when reading the response could become available to the original request. Since the connection is not closed afterwards, the second request never finishes and times out.

(To be clear, that's not speculation: I've actually seen several responses arrive for the wrong request, and a lot of timeouts.)

Seems to be a new bug in 7.65.0, 7.64.1 did not have it.

I expected the following

All requests should happily execute and the program should finish. There should be no responses arriving for the wrong request either.

curl/libcurl version

curl 7.65.0 (x86_64-pc-linux-gnu) libcurl/7.65.0 OpenSSL/1.0.2k-fips zlib/1.2.7 brotli/1.0.1 c-ares/1.15.0 nghttp2/1.38.0
Release-Date: 2019-05-22
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS brotli HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL UnixSockets

operating system

CentOS 7 with custom-built curl
@bagder
Copy link
Member

bagder commented May 28, 2019

Possibly related to #3952 and I think #3953 might fix or at least change this.

@bagder bagder added the HTTP/2 label May 28, 2019
@TvdW
Copy link
Author

TvdW commented May 28, 2019

#3953 doesn't seem to change this.

I filed #3951 and #3952 separately because 3951 is new in 7.65.0, and 3952 was already in 7.64.1. I suspect they are different bugs.

bagder added a commit that referenced this issue May 28, 2019
@bagder
conncache: make "bundles" per host name when doing proxy tunnels
fc4b007 
Only HTTP proxy use where multiple host names can be used over the same
connection should use the proxy host name for bundles.

Reported-by: Tom van der Woerdt
Fixes #3951
@bagder bagder mentioned this issue May 28, 2019
Closed
@bagder
Copy link
Member

bagder commented May 28, 2019

I didn't manage to reproduce your symptoms exactly, but I found this issue that I think could be one that causes these problems for you. Let's start out trying this at least. See #3955

@TvdW
Copy link
Author

TvdW commented May 28, 2019 •
edited

Hm, I don't think #3955 will fix this, or at least not the issue I started out with, as it didn't involve a proxy at all... (edit: github issue has a proxy listed, but the bug I originally started debugging was with connections to localhost, no proxy, but hard to replicate outside of my specific environment)

Will try the patch when I'm near a compiler, and report back.

@bagder
Copy link
Member

bagder commented May 28, 2019

No, if you make it happen without proxy, #3955 will not change anything for you...

@TvdW
Copy link
Author

TvdW commented May 28, 2019

Beautiful.

#3955 resolves the problem as reproducible using the code in this issue. However, it turns out that during test minimization yesterday, I hit a different bug and then started minimizing towards that instead. 😆 🎉 I reported a different bug than I was experiencing.

So let's try this again... here's a gist trying to reproduce the issue I was originally having. https://gist.github.com/TvdW/f4815f8080d9dc30daa5d74b1f878dae

There's a nginx server running on 127.0.0.1, listening on port 80 for http/1.1, and on port 81 for http/2 (no TLS on either). I guess a simple nginx config to reproduce would be along the lines of

server {
listen *:80;
root /var/empty;
location / { }
}
server {
listen *:81 http2;
root /var/empty;
location / { }
}

Full log: https://gist.github.com/TvdW/37f7e01bf7b3b8763557d0a1c204379f

@bagder
Copy link
Member

bagder commented May 28, 2019

Ah, ok. Would you mind submitting this as a new issue as then we can consider #3955 a fix of this and I can work on this new one separately?

@TvdW TvdW changed the title multi: requests may get response for request issued in parallel (or timeout, or errors) bundle collision when using the same proxy for two remotes May 28, 2019
@TvdW
Copy link
Author

TvdW commented May 28, 2019

#3956

@TvdW TvdW mentioned this issue May 28, 2019
Closed
@bagder bagder closed this as completed in 2f11fbf May 28, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Aug 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
HTTP/2
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

conncache: make "bundles" per host name when doing proxy tunnels curl/curl
2 participants
@TvdW @bagder