NTLM 401 on HTTPS IIS endpoints #3280

georgeok · 2018-11-15T19:35:00Z

NTLM fails consistently on Windows when targeting https endpoint that are WIA protected. This only happens when "Extended Protection" is set to Accept or Require in the IIS server (Accept is the default). The same endpoint work correct when plain http is in use.

I did this

curl.exe -v -k --ntlm -u user:Password https://my.dc.local/adfs/ls/wia/


*   Trying 10.211.55.65...
* TCP_NODELAY set
* Connected to my.dc.local (10.211.55.65) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / AES128-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=my.dc.local
*  start date: Nov 13 17:36:50 2018 GMT
*  expire date: Nov 13 00:00:00 2019 GMT
*  issuer: CN=my.dc.local
*  SSL certificate verify result: unable to get local issuer certificate (20), c
ontinuing anyway.
* Server auth using NTLM with user 'george'
> GET /adfs/ls/wia/ HTTP/1.1
> Host: my.dc.local
> Authorization: NTLM TlR---------------------------------------------AAAADw==
> User-Agent: curl/7.62.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM TlRMTVNTUAACAAAABAAEADgAAAAFgomixeUPaN/XXvgAAAAAAAAAAGI
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
----------------------------------------------------------------------------AQAAAAA=
< Date: Thu, 15 Nov 2018 19:37:18 GMT
< Content-Length: 341
<
* Ignoring the response-body
* Connection #0 to host my.dc.local left intact
* Issue another request to this URL: 'https://my.dc.local/adfs/ls/wia/'
* Found bundle for host my.dc.local: 0x1882600 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host my.dc.local
* Connected to my.dc.local (10.211.55.65) port 443 (#0)
* Server auth using NTLM with user 'george'
> GET /adfs/ls/wia/ HTTP/1.1
> Host: my.dc.local
> Authorization: NTLM TlRMTVNTUAADAAAAGAAYAHoAAAAKAQoBkgAAAAAAAABYAAAADAAMAFgAAA
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
AAAAAAAAAAAA==
> User-Agent: curl/7.62.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html
< Server: Microsoft-IIS/8.0
* NTLM handshake rejected
* Authentication problem. Ignoring this.
< WWW-Authenticate: NTLM
< X-Powered-By: ASP.NET
< Date: Thu, 15 Nov 2018 19:37:18 GMT
< Content-Length: 1293
<
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/x
html1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>401 - Unauthorized: Access is denied due to invalid credentials.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;b
ackground:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS
", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;positio
n:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>401 - Unauthorized: Access is denied due to invalid credentials.</h2>
  <h3>You do not have permission to view this directory or page using the creden
tials that you supplied.</h3>
 </fieldset></div>
</div>
</body>
</html>
* Connection #0 to host my.dc.local left intact

I expected the following

A 200 response from the server. Again everything works fine with plain HTTP

curl/libcurl version

I have tried curl 7.62, 7.55.1 and 7.57.

[curl -V output]

curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: [unreleased]
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL


curl 7.62.0 (i386-pc-win32) libcurl/7.62.0 OpenSSL/1.1.1 (WinSSL) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.8.0 nghttp2/1.34.0
Release-Date: 2018-10-31
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz brotli TLS-SRP HTTP2 HTTPS-proxy MultiSSL

operating system

Failing behaviour is consistent on Windows 7,8,10.
macOS and Linux work fine.

The text was updated successfully, but these errors were encountered:

danielgustafsson · 2018-11-15T19:46:04Z

Please amend this with more information as to what the issue actually is.

georgeok · 2018-11-15T19:50:51Z

Please amend this with more information as to what the issue actually is.

Sorry for the delay. Just amended the information.

bagder · 2018-11-15T23:03:57Z

These windows builds are built with SSPI enabled, meaning they use native windows function calls for the NTLM magic. That makes this a very windows-specific issue and requires someone to debug this on windows. It would be interesting to learn if someone would try this with a windows-build without SSPI and see if that makes it work.

NTLM debugging is challenging...

georgeok · 2018-11-19T10:56:00Z

I've just tried with curl 7.59 without the SSPI and everything work as expected. Looks like something is failing only when the SSPI is used

curl 7.59.0 (i686-pc-cygwin) libcurl/7.59.0 OpenSSL/1.0.2p zlib/1.2.11 libidn2/2.0.4 libpsl/0.18.0 (+libidn2/2.0.2) libssh2/1.7.0 nghttp2/1.31.0
Release-Date: 2018-03-14
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS Debug IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL Metalink```

georgeok · 2018-11-23T11:31:02Z

After looking deeper to the problem I can see why curl with the SSPI is failing. The problem is the Windows Extended Protection (aka channel binding).
https://blogs.msdn.microsoft.com/openspecification/2013/03/26/ntlm-and-channel-binding-hash-aka-extended-protection-for-authentication/

A solution would be to:

Export the CBT by the Schannel SPI (SSL) with QueryContextAttributes(SECPKG_ATTR_ENDPOINT_BINDINGS). It returns a SecPkgContext_Bindings structure.
Import with the function InitializeSecurityContext in a supplemental buffer of the type SECBUFFER_CHANNEL_BINDINGS when creating the type 3 somewhere message here.

Example implementations:
C:
https://github.com/eurogiciel-oss/freerdp/blob/ff1454a973fa1f5a938dc4af0563e89e95c3489f/libfreerdp/core/gateway/ntlm.c#L221

https://github.com/chromium/chromium/blob/7236fb7258ccb4095b21fd16219236b4835dd849/net/http/http_auth_sspi_win.cc#L376

Python:
https://github.com/brandond/requests-negotiate-sspi/blob/4a115e2cc6ad40929bf0c7722b017a5aee46f5d7/requests_negotiate_sspi/requests_negotiate_sspi.py#L65

I would be to patch and test it with some guidance from someone more experienced with libcurl.

Windos extended potection (aka ssl channel binding) is required to login to ntlm IIS endpoint, otherwise the server return 401 responses. [Fixes: curl#3280]

bagder added HTTP authentication labels Nov 15, 2018

bagder added the help wanted label Nov 15, 2018

georgeok added a commit to georgeok/curl that referenced this issue Nov 28, 2018

[ntlm] SPPI channel binding suport

d339d21

Windos extended potection (aka ssl channel binding) is required to login to ntlm IIS endpoint, otherwise the server return 401 responses. [Fixes: curl#3280]

georgeok mentioned this issue Nov 28, 2018

[ntlm] SSPI channel binding support #3321

Closed

MarcelRaad closed this as completed in 0966233 Jan 19, 2019

georgeok mentioned this issue Jan 28, 2019

Negotiate SSPI 401 Channel Bindings #3503

Closed

lock bot locked as resolved and limited conversation to collaborators Apr 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NTLM 401 on HTTPS IIS endpoints #3280

NTLM 401 on HTTPS IIS endpoints #3280

georgeok commented Nov 15, 2018 •

edited

danielgustafsson commented Nov 15, 2018

georgeok commented Nov 15, 2018

bagder commented Nov 15, 2018

georgeok commented Nov 19, 2018

georgeok commented Nov 23, 2018

NTLM 401 on HTTPS IIS endpoints #3280

NTLM 401 on HTTPS IIS endpoints #3280

Comments

georgeok commented Nov 15, 2018 • edited

I did this

I expected the following

curl/libcurl version

operating system

danielgustafsson commented Nov 15, 2018

georgeok commented Nov 15, 2018

bagder commented Nov 15, 2018

georgeok commented Nov 19, 2018

georgeok commented Nov 23, 2018

georgeok commented Nov 15, 2018 •

edited