Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In non-sspi builds, NTLM cannot identify against saslauthd (cyrus-sasl) #6813

Closed
monnerat opened this issue Mar 29, 2021 · 4 comments
Closed

Comments

@monnerat
Copy link
Contributor

Current curl NTLM implementation is based on https://curl.se/rfc/ntlm.html: this document seems to be an outstanding work of reverse engineering and/or information compilation done at a time when NTLM was a proprietary protocol and no specification were published.

Since then, M$ released the specification: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-nlmp/

Comparing both documents, there seems to be some misunderstanding in the first one, one of them explaining why curl is not able to authenticate against saslauthd using NTLM:

Trying to be short:

  • there are 3 ways to provide authentication information, NTLMv1, NTLMv1 with extended security and NTLMv2.
  • curl understand negotiated flags NTLMFLAG_NEGOTIATE_NTLM2_KEY as a request to use NTLMv2 and NTLMFLAG_NEGOTIATE_NTLM_KEY to use NTLMv1 with extended security.
  • according to the MS-NLMP document (that names these flags NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY and NTLMSSP_NEGOTIATE_NTLM respectively), this is a misuse: there is no possible negotiation of NTLMv2 in the protocol (client should have an external way of selecting it or not). If NTLMv2 is not used, NTLMFLAG_NEGOTIATE_NTLM2_KEY tells to use NTLMv1 extended security, while the meaning of NTLMFLAG_NEGOTIATE_NTLM_KEY is not clear (probably can be cleared to downgrade the authentication to LM-only).
  • curl negotiates the full capabilities (according to its protocol understanding and conditional compilation statements), that is: NTLMFLAG_NEGOTIATE_NTLM_KEY and NTLMFLAG_NEGOTIATE_NTLM2_KEY. But saslauthd does not support extended security and thus properly clears flag NTLMFLAG_NEGOTIATE_NTLM2_KEY in challenge message (type2). As NTLMFLAG_NEGOTIATE_NTLM_KEY is still set, curl uses NTLMv1 with extended security for authentication, which is wrong according to MS-NLMP.

Note that, although NTLMv2 is not negotiated, a server receiving an authentication (type3) message can determine it from the NTLM authentication data length. A client however, has no way to know if a server supports NTLMv2 or not.

Although MS-NLMP does not specifiy any meaning of NTLMFLAG_NEGOTIATE_NTLM2_KEY when using NTLMv2, it sets it in negotiation of all NTLMv2 examples.

This does not affect builds with ntlm-sspi.

@monnerat
Copy link
Contributor Author

Some other client libraries behavior:

  • cyrus-sasl checks a configuration parameter to use NTLMv2 or NTLMv1 without extended security.
  • libgsasl uses libntlm that always uses NTLMv1 without extended security.

Possible fixes in curl
There is a tradeoff in choice: one end is the maximum compatibility with old servers at the price of a possible rejection by a server requesting a more secure level of authentication, the other is the maximum security level.

  1. Maximum security level: Always use NTLMv2 if compiled in.
  2. Securely mitigated: if NTLMFLAG_NEGOTIATE_NTLM2_KEY is set and NTLMv2 is compiled in, use NTLMv2: the idea here is to use the maximum security level as a server that accepts extended security for NTLMv1 is likely to understand NTLMv2 too. If NTLMv2 is not compiled in, but extended security is, use NTLMv1 + extended security.
  3. Less securely mitigated: if NTLMFLAG_NEGOTIATE_NTLM2_KEY is set and extended security is compiled in, use NTLMV1 with extended security. If not compiled in, use NTLMv2 if available.
  4. Never use NTLMv2: if NTLMFLAG_NEGOTIATE_NTLM2_KEY is set and extended security is compiled in, use NTLMV1 with extended security.
  5. Never use extended security: Always use NTLMv1 without extended security.
  6. Something else, as a new CURLOPT option for example (needs to set the default anyway).

In all cases, if NTLMFLAG_NEGOTIATE_NTLM2_KEY is clear or neither NTLMv2 nor extended security are available, use NTLMv1 without extended security.
NTLMFLAG_NEGOTIATE_NTLM_KEY is not tested anymore.

A local build with fix 2 applied has been tested against saslauthd successfully.

Whatever fix is adopted, the test suite would have to be adapted accordingly. Real tests against a M$ server are also mandatory.

Questions:

  • Which is the preferred fix ?
  • As I don't have any M$ software, how can I test in real conditions ?

@dfandrich
Copy link
Contributor

dfandrich commented Mar 29, 2021 via email

@monnerat
Copy link
Contributor Author

monnerat commented Mar 30, 2021

Yes, there are. But I don't know if they were available at the time NTLM was introduced in curl.
As cited in #6813 (comment), I checked cyrus-sasl and libgsasl/libntlm: both uses NTLMv1 without extended security (the lowest security level). Is this what we want ?
NTLMv1 with extended security as well as NTLMv2 feature a client challenge, while NTLMv1 does not.
In all cases, the biggest work will be about testing.

@MarcelRaad
Copy link
Member

I like option 2. It's not only more secure, but also the smaller change in behavior.

monnerat added a commit to monnerat/curl that referenced this issue Apr 4, 2021
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes curl#6813
monnerat added a commit to monnerat/curl that referenced this issue Apr 4, 2021
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes curl#6813
monnerat added a commit to monnerat/curl that referenced this issue Apr 4, 2021
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes curl#6813
monnerat added a commit to monnerat/curl that referenced this issue Apr 5, 2021
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes curl#6813
monnerat added a commit to monnerat/curl that referenced this issue Apr 6, 2021
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes curl#6813
@bagder bagder closed this as completed in 9c1e1a6 Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

4 participants