ntlm: support version 2 on 32-bit platforms and fix negotiated flags usage #6849

monnerat · 2021-04-04T14:14:42Z

This does not affect sspi builds.

I successfully performed as many tests as my non-M$ environment allows to, but this must be tested against a genuine M$ server, although I'm quite confident in it.

Thanks for considering it.

bagder · 2021-04-05T10:40:43Z

lib/vauth/ntlm.c

    unsigned char ntbuffer[0x18];
    unsigned char tmp[0x18];
-    unsigned char md5sum[CURL_MD5_DIGEST_LENGTH];
+    unsigned char md5sum[MD5_DIGEST_LENGTH];


Why this change? We provide our own define to make sure it exists under a single name. There's no guarantee there's a MD5_DIGEST_LENGTH define provided in systems! See d80d419 that introduced our name.

You're right, thanks for pointing it.
I intended to replace by MD5_DIGEST_LEN which is defined in curl_md5.h. Will fix it.

Fixed by using MD5_DIGEST_LEN.

Do you remove the CURL_MD5_DIGEST_LENGTH define as well then?

Yes, because it was local to vauth/ntlm.c.

bagder · 2021-04-05T10:42:48Z

this must be tested against a genuine M$ server

We need to figure out how to get someone to run such tests! I don't have any such servers/services to test against and I don't know who does! 🤔

monnerat · 2021-04-05T10:57:09Z

We need to figure out how to get someone to run such tests! I don't have any such servers/services to test against and I don't know who does!

We're both linux guys, that's why we are in the same situation.
If the initial ntlm developer is still around, maybe we should ask him/her for a review. Else a W$ contributor.

I would prefer not to have it committed for "testing by the community".
One thing is sure: current flag usage is wrong!

bagder · 2021-04-05T15:37:56Z

If the initial ntlm developer is still around, maybe we should ask him/her for a review. Else a W$ contributor.

I wrote the initial NTLM implementation in curl...

One thing is sure: current flag usage is wrong!

... I believe you, but still it has been right enough to work pretty well for a lot of users over the years.

monnerat · 2021-04-05T16:40:32Z

I wrote the initial NTLM implementation in curl...

Congratulations! I guess it was not that easy when no official documentation was released.
And how did you test it?

bagder · 2021-04-05T20:51:32Z

Other users at the time tested it against actual NTLM-using servers before I wrote up the test cases that verifiy that we maintain the same behavior.

monnerat · 2021-04-06T01:13:58Z

Other users at the time tested it against actual NTLM-using servers before I wrote up the test cases that verifiy that we maintain the same behavior.

I had to adapt the tests as the NTLM response in type 3 message changes. I did it manually, checking that only the targeted message part was affected.

I guess NTLM is not much used nowadays.

bagder · 2021-04-06T14:24:43Z

I guess NTLM is not much used nowadays.

According to our user survey 2020, 9.9% of our users use it, down just a little from 10.3% the year before. I think that's a pretty sizable share after all.

monnerat · 2021-04-06T14:27:55Z

9.9% of our users use it

Maybe we should call them for evaluation, via the mailing list for example ?

MarcelRaad · 2021-04-06T15:13:52Z

FWIW, I'm one of those who use NTLM (via SSPI though), but NTLMv2 only would suffice for me.

I can try to find a test server and test this patch.

monnerat · 2021-04-06T18:24:03Z

Many thanks Marcel,
In the meantime, I managed to setup a minimal IIS7 for testing.
The first tests results with it are not satisfactory, thus I think we should put it on hold for a few days.
I'll come back to your kind proposal when I feel it more mature.

... as !defined(CURL_DISABLE_CRYPTO_AUTH) is a prerequisite for the whole NTLM.

According to Microsoft document MS-NLMP, current flags usage is not accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of extended security in an NTLM authentication message and NTLM version 2 cannot be negotiated within the protocol. The solution implemented here is: if the extended security flag is set, prefer using NTLM version 2 (as a server featuring extended security should also support version 2). If version 2 has been disabled at compile time, use extended security. Tests involving NTLM are adjusted to this new behavior. Fixes curl#6813

monnerat · 2021-04-07T11:11:17Z

OK, I performed additional test with IIS7 and it works.

Here is a table of what is used depending on curl and server capabilities:

curl \ server	V1ext	V1ext + V2	V2	V1 only
V1ext	V1ext	V1ext	V1	V1
V1ext + V2	V2 !	V2	V1	V1
V2	V2 !	V2	V1	V1
V1 only	V1	V1	V1	V1

Entries marked with ! are wrong and will fail, but these situations will probably never occur.
V2 cannot be negotiated: this explains oddities in this table.
By default curls supports V2 and V1ext (this can be changed by altering defines in lib/curl_ntlm_core.h).
Of course, this can fail if the security level used is not high enough for the server.
This should at least be compatible with M$ servers, samba and saslauthd.

It should be ready now.

MarcelRaad · 2021-04-07T11:19:01Z

The V2 column looks strange. Shouldn't the V1ext + V2 and V2 rows contain V2 instead of V1? And probably the V1ext row V1ext?

monnerat · 2021-04-07T11:24:46Z

Shouldn't the V1ext + V2 and V2 rows contain V2 instead of V1?

No because as V2 cannot be negotiated, curl does not see server supports V2. We thus assume that V2 is supported only if V1ext is. IMHO this is the safest way.

MarcelRaad · 2021-04-07T11:35:26Z

OK, makes sense. I thought that V1ext + V2 looked the same to the client instead of V1 + V2.

monnerat · 2021-04-07T12:00:02Z

I thought that V1ext + V2 looked the same to the client instead of V1 + V2.

No. In fact a V2-aware server does not care about the extended security flag when determining V2 is used: it only checks the NTLM response length.
The unknown here is: is the server V2-aware ?

The only other solution to the problem is to never use V2 (this will also remove a lot of code), but I don't think you will feel satisfied with it :-)

monnerat · 2021-04-08T14:31:55Z

Thanks for your review and work on this, Marcel.

bagder · 2021-04-09T07:41:27Z

Thanks!

Closes #6849

According to Microsoft document MS-NLMP, current flags usage is not accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of extended security in an NTLM authentication message and NTLM version 2 cannot be negotiated within the protocol. The solution implemented here is: if the extended security flag is set, prefer using NTLM version 2 (as a server featuring extended security should also support version 2). If version 2 has been disabled at compile time, use extended security. Tests involving NTLM are adjusted to this new behavior. Fixes #6813 Closes #6849

monnerat · 2021-04-09T11:01:58Z

Thanks for pulling.

monnerat force-pushed the ntlm branch 2 times, most recently from 5dc6fda to ea6fd4f Compare April 4, 2021 17:49

bagder reviewed Apr 5, 2021

View reviewed changes

monnerat force-pushed the ntlm branch from ea6fd4f to 9135bd5 Compare April 5, 2021 11:30

monnerat added 3 commits April 7, 2021 00:20

curl_ntlm_core.h: simplify conditionals for USE_NTLM2SESSION

5991a8e

... as !defined(CURL_DISABLE_CRYPTO_AUTH) is a prerequisite for the whole NTLM.

ntlm: support version 2 on 32-bit platforms

a5d8836

monnerat force-pushed the ntlm branch from 9135bd5 to 100e096 Compare April 6, 2021 22:21

MarcelRaad approved these changes Apr 8, 2021

View reviewed changes

bagder closed this in 10514d0 Apr 9, 2021

bagder pushed a commit that referenced this pull request Apr 9, 2021

ntlm: support version 2 on 32-bit platforms

cca455a

Closes #6849

monnerat deleted the ntlm branch April 9, 2021 11:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ntlm: support version 2 on 32-bit platforms and fix negotiated flags usage #6849

ntlm: support version 2 on 32-bit platforms and fix negotiated flags usage #6849

monnerat commented Apr 4, 2021

bagder Apr 5, 2021

monnerat Apr 5, 2021

monnerat Apr 5, 2021

bagder Apr 5, 2021

monnerat Apr 5, 2021

bagder commented Apr 5, 2021

monnerat commented Apr 5, 2021

bagder commented Apr 5, 2021

monnerat commented Apr 5, 2021

bagder commented Apr 5, 2021

monnerat commented Apr 6, 2021

bagder commented Apr 6, 2021

monnerat commented Apr 6, 2021

MarcelRaad commented Apr 6, 2021

monnerat commented Apr 6, 2021 •

edited

monnerat commented Apr 7, 2021

MarcelRaad commented Apr 7, 2021 •

edited

monnerat commented Apr 7, 2021 •

edited

MarcelRaad commented Apr 7, 2021

monnerat commented Apr 7, 2021

monnerat commented Apr 8, 2021

bagder commented Apr 9, 2021

monnerat commented Apr 9, 2021

ntlm: support version 2 on 32-bit platforms and fix negotiated flags usage #6849

ntlm: support version 2 on 32-bit platforms and fix negotiated flags usage #6849

Conversation

monnerat commented Apr 4, 2021

bagder Apr 5, 2021

Choose a reason for hiding this comment

monnerat Apr 5, 2021

Choose a reason for hiding this comment

monnerat Apr 5, 2021

Choose a reason for hiding this comment

bagder Apr 5, 2021

Choose a reason for hiding this comment

monnerat Apr 5, 2021

Choose a reason for hiding this comment

bagder commented Apr 5, 2021

monnerat commented Apr 5, 2021

bagder commented Apr 5, 2021

monnerat commented Apr 5, 2021

bagder commented Apr 5, 2021

monnerat commented Apr 6, 2021

bagder commented Apr 6, 2021

monnerat commented Apr 6, 2021

MarcelRaad commented Apr 6, 2021

monnerat commented Apr 6, 2021 • edited

monnerat commented Apr 7, 2021

MarcelRaad commented Apr 7, 2021 • edited

monnerat commented Apr 7, 2021 • edited

MarcelRaad commented Apr 7, 2021

monnerat commented Apr 7, 2021

monnerat commented Apr 8, 2021

bagder commented Apr 9, 2021

monnerat commented Apr 9, 2021

monnerat commented Apr 6, 2021 •

edited

MarcelRaad commented Apr 7, 2021 •

edited

monnerat commented Apr 7, 2021 •

edited