Is this a timing issue? If you run each build 10x do you get the same results? Could it have something to do with the way you built libssh2? If you are using the libssh2 dll use curl --dump-module-paths to confirm the right one is being loaded.

Is this a timing issue? If you run each build 10x do you get the same results?

I just ran it 100 times through a script, and the error appears every time.

Could it have something to do with the way you built libssh2?

Possibly, but it happens when I link against a static libssh2 built with
cmake -DCRYPTO_BACKEND=WinCNG -DENABLE_ZLIB_COMPRESSION=ON -DZLIB_LIBRARY=C:\dev\zlib\lib\zlib.lib -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -A Win32 ..
as well as with curl built with vcpkg, which uses
cmake "-DCMAKE_BUILD_TYPE=Release" "-DFETCHCONTENT_FULLY_DISCONNECTED=ON" "-DBUILD_EXAMPLES=OFF" "-DBUILD_TESTING=OFF" "-DENABLE_ZLIB_COMPRESSION=ON" "-DCMAKE_DISABLE_FIND_PACKAGE_ZLIB=OFF" "-DCRYPTO_BACKEND=OpenSSL" "-DBUILD_STATIC_LIBS:BOOL=OFF" "-DBUILD_SHARED_LIBS=ON"
and with the mingw64 version as well

If you are using the libssh2 dll use curl --dump-module-paths to confirm the right one is being loaded.

Since it also happens with static linking this doesn't seem to be the issue - i checked anyway for the vcpkg version and it shows the correct dll.

Is there any pre built version of libcurl (dll or static lib) that we could use? Or can we provide any other information to facilitate finding the cause of this problem

Thanks for that. Based on the verbose output the state when the error occurs is SSH_AUTH_PKEY:

Note LIBSSH2_ERROR_EAGAIN causes a break rather than show the error message SSH public key authentication failed: Would block requesting userauth list. libssh2 code has several places it sets that error message but it always then returns LIBSSH2_ERROR_EAGAIN. Either another libssh2 function is changing the return code somewhere or libcurl is pulling an old error message from the queue (in other words, a different libssh2 function failed but didn't set an error message to overwrite that one). Please try this patch so we can see the return code

diff --git a/lib/vssh/libssh2.c b/lib/vssh/libssh2.c
index 37040b4..1488739 100644
--- a/lib/vssh/libssh2.c
+++ b/lib/vssh/libssh2.c
@@ -1180,6 +1180,7 @@ static CURLcode ssh_statemach_act(struct Curl_easy *data, bool *block)
         char *err_msg = NULL;
         (void)libssh2_session_last_error(sshc->ssh_session,
                                          &err_msg, NULL, 0);
+        infof(data, "libssh2_userauth_publickey_fromfile_ex error: %d\n", rc);
         infof(data, "SSH public key authentication failed: %s", err_msg);
         state(data, SSH_AUTH_PASS_INIT);
         rc = 0; /* clear rc and continue */

I also suspect this function because you wrote earlier it doesn't matter if the key file exists or not.

thanks for the patch, output is:
* libssh2_userauth_publickey_fromfile_ex error: -1

I debugged it a little (had to remove the \n from your patch or it would throw an assert in log), and found the reason this is happening at least with libssh2 using wincng crypto backend:
in libssh2 wincng.c _libssh2_wincng_load_private, the key is expected to start with
-----BEGIN RSA PRIVATE KEY-----
or
-----BEGIN DSA PRIVATE KEY-----
and if it doesn't, it returns -1, which is the same as if the key file doesn't exist at all.
My key began with -----BEGIN OPENSSH PRIVATE KEY-----, which works nicely with curl.exe downloaded from the website, however, after converting it using ssh-keygen -m pem -f key.pem , neither my build nor the version from the website can authenticate successfully (curl: (67) Authentication failure), libssh2_userauth_publickey_fromfile_ex error: -18.

As this seems to be a libssh2 problem, should i open an issue there?

and if it doesn't, it returns -1, which is the same as if the key file doesn't exist at all.

what is the return code of libssh2_userauth_publickey_fromfile_ex in that case?

libssh2_userauth_publickey_fromfile_ex error: -18.

what is the error message curl shows you in that case? aside from your certificate issue I'm trying to determine if we're doing something wrong in curl processing the error messages or this is a libssh2 issue with not updating the error messages

As this seems to be a libssh2 problem, should i open an issue there?

Yes, please file at libssh2 and link to this issue and the two issues below which may be related:
#4568
libssh2/libssh2#431 (since determined those crossed out issues are likely not related)

Sorry for the delay,

what is the return code of libssh2_userauth_publickey_fromfile_ex in that case?

if the path supplied via curl --key parameter points to an OpenSSH private key (or any file not containing -----BEGIN RSA PRIVATE KEY----- / -----BEGIN DSA PRIVATE KEY-----) or even a nonexistent path, libssh2_userauth_publickey_fromfile_ex returns -1,

what is the error message curl shows you in that case [...]

SSH public key authentication failed: Username/PublicKey combination invalid
and
curl: (67) Authentication failure

Yes, please file at libssh2 and link to this issue

first I want to try linking curl against a libssh2 with the OpenSSL crypto backend and try how that behaves, but I had linking problems, I will dig into it tomorrow - also didn't have a chance trying with curl 8.3.0 yet.

Update: I managed to link curl 8.3.0 against libssh2 with OpenSSL crypto backend, and this solves the problem, i can connect to sftp hosts using a private key with -----BEGIN OPENSSH PRIVATE KEY----- format; however, when supplying an RSA private key, the error message is just (curl: (67) Authentication failure), and the verbose log message says SSH public key authentication failed: Username/PublicKey combination invalid.
So as far as I understand it's a combination of two problems:

• No check if the supplied certificate exists and if it's in the correct format
• libssh2 private key authentication doesn't work using wincng crypto backend

Since it took quite some time getting past these hurdles, i don't know when I'll have time to analyze this in detail (project deadlines approaching), but if you need any more information i'd be glad to help.

if the path supplied via curl --key parameter points to an OpenSSH private key (or any file not containing -----BEGIN RSA PRIVATE KEY----- / -----BEGIN DSA PRIVATE KEY-----) or even a nonexistent path, libssh2_userauth_publickey_fromfile_ex returns -1,

I've filed #11881 to address the incorrect error messages when libssh2_userauth_publickey_fromfile_ex returns -1.

So as far as I understand it's a combination of two problems:

• No check if the supplied certificate exists and if it's in the correct format
• libssh2 private key authentication doesn't work using wincng crypto backend

Ok. In my opinion these are libssh2 issues and you would have to file with them. #11881 is as far as I think we should take it in libcurl.

@consulion Did you report this to libssh2?

This looks like an @libssh2 issue.