Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Socket use after close when connecting to remote HTTPS via HTTPS proxy #8193

Closed
intxgo opened this issue Dec 27, 2021 · 6 comments
Closed

Socket use after close when connecting to remote HTTPS via HTTPS proxy #8193

intxgo opened this issue Dec 27, 2021 · 6 comments
Labels
connecting & proxies

Comments

@intxgo
Copy link

intxgo commented Dec 27, 2021 •
edited

I did this

  1. Connect to non-existent HTTPS server via HTTPS proxy tunnel.
  2. Observe the process crash

We use ProcessStrictHandleCheckPolicy which reveals socket use after close.

I expected the following

Curl returns "Received HTTP code 502 from proxy after CONNECT" and gracefully close the HTTPS tunnel.

curl/libcurl version

7.79.1

[curl -V output]
curl 7.81.0-DEV (x86_64-pc-win32) libcurl/7.81.0-DEV OpenSSL/1.1.1i zlib/1.2.11 WinIDN
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI UnixSockets alt-svc libz

operating system

Windows 10 20H2
@intxgo intxgo mentioned this issue Dec 27, 2021
Closed
@bagder
Copy link
Member

bagder commented Dec 27, 2021

You didn't show the curl -V output though, so we're clueless about your TLS config.

@intxgo
Copy link
Author

intxgo commented Dec 27, 2021 •
edited

Thanks, I see, but I'm only linking with curl lib in my project.

Running curl.exe built from this repo, top of master branch, the problem is still there. I added curl -V output to the description above.

Patch which enables strict handle check policy on Windows:
https://github.com/intxgo/curl/blob/8c04acbaecd4071b86c962998545c77fcf49b066/src/tool_main.c#L262-L275

With the above policy, unpatched code crash when executed like this (with no server on port 9999):

.\curl --proxy-cacert ca.pem https://localhost:9999 -x https://localhost:8899

with my PR all is fine:

.\curl --proxy-cacert ca.pem https://localhost:9999 -x https://localhost:8899
curl: (56) Received HTTP code 502 from proxy after CONNECT

@bagder
Copy link
Member

bagder commented Dec 27, 2021

I want to A) reproduce the problem and then B) verify that the fix is fine.

@bagder
Copy link
Member

bagder commented Dec 27, 2021

It does not reproduce for me on Linux, with or without valgrind:

./src/curl -x https://localhost:8843 https://localhost:7999/ --proxy-insecure

Where is the socket use after close happening? Can you get a stack trace from your crash?

@intxgo
Copy link
Author

intxgo commented Dec 28, 2021

The stack trace:

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007ffbc7670d2a (ntdll!KiRaiseUserExceptionDispatcher+0x000000000000003a)
   ExceptionCode: c0000008 (Invalid handle)
  ExceptionFlags: 00000000
NumberParameters: 0
Thread tried to close a handle that was invalid or illegal to close

FAULTING_THREAD:  00002f00

PROCESS_NAME:  curl.exe

ERROR_CODE: (NTSTATUS) 0xc0000008 - An invalid HANDLE was specified.

EXCEPTION_CODE_STR:  c0000008

STACK_TEXT:  
0000008d`8b8fdbc0 00007ffb`c4f048b0     : 00000000`67f2bc60 00000247`9566f880 00000247`95677d90 00000000`00000003 : ntdll!KiRaiseUserExceptionDispatcher+0x3a
0000008d`8b8fdc90 00007ffb`c5854b31     : 00000247`9566f720 00000247`956776a0 0000008d`8b8fde48 00000000`00000000 : KERNELBASE!GetHandleInformation+0x40
0000008d`8b8fdcd0 00007ffb`c584627b     : 00000247`95677fbb 00000000`00000000 00000000`67e0db00 00000247`95665fb0 : WS2_32!DSOCKET::FindIFSSocket+0x21
0000008d`8b8fdd00 00000000`67e4070d     : 00000000`00000018 0000008d`8b8fde58 00000247`95677fb3 0000008d`8b8fde53 : WS2_32!send+0x13f5b
0000008d`8b8fdda0 00000000`67e3b12f     : 0000008d`8b8fde70 00000000`6d48e55a cccccccc`cccccccc cccccccc`cccccccc : libcrypto_1_1_x64!BIO_s_null+0x1dd
0000008d`8b8fdde0 00000000`67e39585     : 0000008d`00000003 00000247`00000001 0000008d`8b8fdf68 00000000`00000015 : libcrypto_1_1_x64!BIO_number_written+0x8f
0000008d`8b8fde10 00000000`6d487aed     : 0000008d`00000000 0000008d`8b8fe580 cccccccc`cccccccc 0000008d`8b8fdfb0 : libcrypto_1_1_x64!BIO_write+0xb5
0000008d`8b8fde90 00000000`6d488c05     : cccccccc`cccccccc cccccccc`cccccccc cccccccc`cccccccc cccccccc`cccccccc : libssl_1_1_x64!SSL_rstate_string+0x4ad
0000008d`8b8fdef0 00000000`6d4941c6     : cccccccc`cccccccc cccccccc`00000015 00000247`95663e10 0000008d`8b8fef10 : libssl_1_1_x64!SSL_rstate_string+0x15c5
0000008d`8b8feed0 00000000`6d493116     : cccccccc`cccccccc cccccccc`cccccccc cccccccc`cccccccc 00000000`0000000b : libssl_1_1_x64!SSL_rstate_string+0xcb86
0000008d`8b8fef40 00007ff6`a528f4d7     : cccccccc`cccccccc cccccccc`cccccccc 00000001`00000001 cccccccc`cccccccc : libssl_1_1_x64!SSL_rstate_string+0xbad6
0000008d`8b8fefb0 00007ff6`a528f59c     : 00000247`9547b560 00000247`954792a0 00000247`95479510 cccccccc`cccccccc : curl!ossl_closeone+0xa7
0000008d`8b8ff040 00007ff6`a525d476     : 00000247`9547b560 00000247`954792a0 cccccccc`00000000 0000edfe`e00c382a : curl!ossl_close+0x3c
0000008d`8b8ff070 00007ff6`a5261a6c     : 00000247`9547b560 00000247`954792a0 00000000`00000000 00000247`9547b598 : curl!Curl_ssl_close+0x36
0000008d`8b8ff0a0 00007ff6`a5260846     : 00000247`9547b560 00000247`954792a0 00000247`954975c0 00000247`954769e8 : curl!conn_shutdown+0xbc
0000008d`8b8ff0d0 00007ff6`a52513c7     : 00000247`9547b560 00000247`954792a0 0000008d`8b8ff200 0000008d`8b8ff1e4 : curl!Curl_disconnect+0xf6
0000008d`8b8ff100 00007ff6`a524b13d     : 00000247`9547a170 0000008d`8b8ff478 00000247`9547b560 ffffffff`ffffffff : curl!multi_runsingle+0x1c47
0000008d`8b8ff420 00007ff6`a523a389     : 00000247`9547a170 0000008d`8b8ff544 cccccccc`00000000 cccccccc`000003e8 : curl!curl_multi_perform+0xdd
0000008d`8b8ff500 00007ff6`a523a5a4     : 00000247`9547a170 00000247`9547b560 00000000`00000005 0000008d`8b8ff4f9 : curl!easy_transfer+0x99
0000008d`8b8ff5a0 00007ff6`a5239906     : 00000247`9547b560 cccccccc`cccccc00 cccccccc`cccccccc 0000edfe`e00c255a : curl!easy_perform+0x174
0000008d`8b8ff5f0 00007ff6`a522db29     : 00000247`9547b560 00000247`95479c50 0000008d`8b8ff664 00007ffb`8f79e1c7 : curl!curl_easy_perform+0x16
0000008d`8b8ff620 00007ff6`a522dfed     : 0000008d`8b8ff810 00000247`954769c0 00000000`00000005 00007ff6`a522f6e3 : curl!serial_transfers+0x109
0000008d`8b8ff6d0 00007ff6`a5223df6     : 0000008d`8b8ff810 00000247`954769c0 00000000`00000000 00007ff6`a5272b60 : curl!run_all_transfers+0x9d
0000008d`8b8ff760 00007ff6`a52236d1     : 0000008d`8b8ff810 00000000`00000005 00000247`9545b3a0 00000003`00000003 : curl!operate+0x2d6
0000008d`8b8ff7d0 00007ff6`a52d2a29     : 0000edfe`00000005 00000247`9545b3a0 00000000`00000000 00007ff6`a52d25bd : curl!main+0x1f1
0000008d`8b8ff8d0 00007ff6`a52d297e     : 00007ff6`a52d8e00 00007ff6`a52d8e30 00000000`00000000 00000000`00000000 : curl!invoke_main+0x39
0000008d`8b8ff920 00007ff6`a52d283e     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : curl!__scrt_common_main_seh+0x12e
0000008d`8b8ff990 00007ff6`a52d2a9e     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : curl!__scrt_common_main+0xe
0000008d`8b8ff9c0 00007ffb`c6e97034     : 0000008d`8b750000 00000000`00000000 00000000`00000000 00000000`00000000 : curl!mainCRTStartup+0xe
0000008d`8b8ff9f0 00007ffb`c7622651     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x14
0000008d`8b8ffa20 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21


STACK_COMMAND:  ~0s ; .cxr ; kb

SYMBOL_NAME:  kernelbase!GetHandleInformation+40

MODULE_NAME: KERNELBASE

IMAGE_NAME:  KERNELBASE.dll

FAILURE_BUCKET_ID:  INVALID_HANDLE_c0000008_KERNELBASE.dll!GetHandleInformation

OS_VERSION:  10.0.19041.1

BUILDLAB_STR:  vb_release

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

IMAGE_VERSION:  10.0.19041.1151

FAILURE_ID_HASH:  {9770812b-4494-dd8c-5dcb-c58d3d8140ae}

Followup:     MachineOwner
---------

0:000> k
 # Child-SP          RetAddr               Call Site
00 0000008d`8b8fdbc0 00007ffb`c4f048b0     ntdll!KiRaiseUserExceptionDispatcher+0x3a
01 0000008d`8b8fdc90 00007ffb`c5854b31     KERNELBASE!GetHandleInformation+0x40
02 0000008d`8b8fdcd0 00007ffb`c584627b     WS2_32!DSOCKET::FindIFSSocket+0x21
03 0000008d`8b8fdd00 00000000`67e4070d     WS2_32!send+0x13f5b
04 0000008d`8b8fdda0 00000000`67e3b12f     libcrypto_1_1_x64!BIO_s_null+0x1dd
05 0000008d`8b8fdde0 00000000`67e39585     libcrypto_1_1_x64!BIO_number_written+0x8f
06 0000008d`8b8fde10 00000000`6d487aed     libcrypto_1_1_x64!BIO_write+0xb5
07 0000008d`8b8fde90 00000000`6d488c05     libssl_1_1_x64!SSL_rstate_string+0x4ad
08 0000008d`8b8fdef0 00000000`6d4941c6     libssl_1_1_x64!SSL_rstate_string+0x15c5
09 0000008d`8b8feed0 00000000`6d493116     libssl_1_1_x64!SSL_rstate_string+0xcb86
0a 0000008d`8b8fef40 00007ff6`a528f4d7     libssl_1_1_x64!SSL_rstate_string+0xbad6
0b 0000008d`8b8fefb0 00007ff6`a528f59c     curl!ossl_closeone+0xa7 [C:\Users\lesio\code\curl\lib\vtls\openssl.c @ 1444] 
0c 0000008d`8b8ff040 00007ff6`a525d476     curl!ossl_close+0x3c [C:\Users\lesio\code\curl\lib\vtls\openssl.c @ 1463] 
0d 0000008d`8b8ff070 00007ff6`a5261a6c     curl!Curl_ssl_close+0x36 [C:\Users\lesio\code\curl\lib\vtls\vtls.c @ 685] 
0e 0000008d`8b8ff0a0 00007ff6`a5260846     curl!conn_shutdown+0xbc [C:\Users\lesio\code\curl\lib\url.c @ 754] 
0f 0000008d`8b8ff0d0 00007ff6`a52513c7     curl!Curl_disconnect+0xf6 [C:\Users\lesio\code\curl\lib\url.c @ 874] 
10 0000008d`8b8ff100 00007ff6`a524b13d     curl!multi_runsingle+0x1c47 [C:\Users\lesio\code\curl\lib\multi.c @ 2571] 
11 0000008d`8b8ff420 00007ff6`a523a389     curl!curl_multi_perform+0xdd [C:\Users\lesio\code\curl\lib\multi.c @ 2638] 
12 0000008d`8b8ff500 00007ff6`a523a5a4     curl!easy_transfer+0x99 [C:\Users\lesio\code\curl\lib\easy.c @ 606] 
13 0000008d`8b8ff5a0 00007ff6`a5239906     curl!easy_perform+0x174 [C:\Users\lesio\code\curl\lib\easy.c @ 696] 
14 0000008d`8b8ff5f0 00007ff6`a522db29     curl!curl_easy_perform+0x16 [C:\Users\lesio\code\curl\lib\easy.c @ 716] 
15 0000008d`8b8ff620 00007ff6`a522dfed     curl!serial_transfers+0x109 [C:\Users\lesio\code\curl\src\tool_operate.c @ 2380] 
16 0000008d`8b8ff6d0 00007ff6`a5223df6     curl!run_all_transfers+0x9d [C:\Users\lesio\code\curl\src\tool_operate.c @ 2551] 
17 0000008d`8b8ff760 00007ff6`a52236d1     curl!operate+0x2d6 [C:\Users\lesio\code\curl\src\tool_operate.c @ 2665] 
18 0000008d`8b8ff7d0 00007ff6`a52d2a29     curl!main+0x1f1 [C:\Users\lesio\code\curl\src\tool_main.c @ 291] 
19 0000008d`8b8ff8d0 00007ff6`a52d297e     curl!invoke_main+0x39 [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 79] 
1a 0000008d`8b8ff920 00007ff6`a52d283e     curl!__scrt_common_main_seh+0x12e [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
1b 0000008d`8b8ff990 00007ff6`a52d2a9e     curl!__scrt_common_main+0xe [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 331] 
1c 0000008d`8b8ff9c0 00007ffb`c6e97034     curl!mainCRTStartup+0xe [d:\a01\_work\2\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp @ 17] 
1d 0000008d`8b8ff9f0 00007ffb`c7622651     KERNEL32!BaseThreadInitThunk+0x14
1e 0000008d`8b8ffa20 00000000`00000000     ntdll!RtlUserThreadStart+0x21

First I set a debug log on CloseHandle and I found out that we are closing the same handle in the place which I patched. When the code unwinds from there it triggers the stream_error on

curl/lib/multi.c

Line 2006 in e7f9c90

stream_error = TRUE;

then it falls into

curl/lib/multi.c

Line 2569 in e7f9c90

Curl_disconnect(data, conn, dead_connection);

then here

curl/lib/url.c

Line 753 in e7f9c90

Curl_ssl_close(data, conn, FIRSTSOCKET);

which is an interesting code. Curl_ssl_close will dereference SSL context at curl_easy_t->conn->ssl[0]->backend which has the same socket handle in it's read/write structures as curl_easy_t->conn->sock[0] which was just closed.

A blind try to fix it with

if(CURL_SOCKET_BAD != conn->sock[FIRSTSOCKET])
    Curl_ssl_close(data, conn, FIRSTSOCKET);

also worked but walking the code, reading comments, I made an educated guess to fix it as submitted in my PR

bagder added a commit that referenced this issue Dec 28, 2021
@bagder
http_proxy: don't close the socket (too early)
93ec5c3 
... and double-check in the OpenSSL shutdown that the socket is actually
still there before it is used.

Fixes #8193

Reported-by: Leszek Kubik
@bagder bagder mentioned this issue Dec 28, 2021
Closed
@bagder
Copy link
Member

bagder commented Dec 28, 2021

Awesome, that was just the info I needed to get a full picture.

@bagder bagder closed this as completed in f772926 Dec 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
connecting & proxies
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

http_proxy: don't close the socket (too early) curl/curl
2 participants
@bagder @intxgo