issues with CURL using RSA keys + SFTP

[Ian Veach via curl-users]

The nutshell:  We get an "authentication failure" when using curl and sftp with 
RSA keys.  We don't know why, and debug/verbose/trace don't reveal anything 
useful, but we don't think it's an authentication failure due to 
user/keys/access.   Source is AIX 7.2 system running curl, destination is a 
RHEL9 system running DEFAULT cryptopolicy (which doesn't allow SHA1 signing, 
but does use SHA1 MAC and RSA keys).  In testing, used same source and 
destination servers:

- fails: curl + sftp + rsa keys
- works: scp or sftp + same rsa keys as above (i.e. Keys ok, network ok, 
access allowance ok)
- works: curl + sftp + rsa keys, but turning on SHA1 in cryptopolicy on 
destination (i.e. seems to be SHA1 related)
- works: curl + sftp + ed25519 keys (i.e. curl+sftp ok, access allowance ok)

Given above, we think our issue seems to be using curl+sftp+no-SHA1 signing 
specifically, but not why/how.  However, the curl is pretty modern and should 
(I assume) be using sha256 or sha512 for signing.  I think somehow it's 
reverting to sha1 internally/compat, but not sure how to verify that.  More 
background:

The curl is IBM provided through AIX Toolbox.  Although not current 8.12.1, 
it's as recent as we can get from them that is supported / without compiling:

$ curl --version
curl 8.6.0 (powerpc-ibm-aix7.1.5.0) libcurl/8.6.0 OpenSSL/1.1.1v 
zlib/1.2.13 libssh2/1.10.0 nghttp2/1.58.0 OpenLDAP/2.5.16
Release-Date: 2024-01-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs 
ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos 
Largefile libz NTLM SPNEGO SSL threadsafe UnixSockets

We could perhaps compile if we absolutely have to, but interactions with 
Peoplesoft and... oracle cruft make that a bit difficult/dangerous.

Actual runs:

[user@aixsource:~/test]$ # failing curl + sftp + rsa keys
[user@aixsource:~/test]$ curl --trace file -u remoteuser: --key 
$keydir/rsa --pubkey $keydir/rsa.pub -T foo \
 sftp://rhel9server.foo.com:/tmp/bar
  % Total% Received % Xferd  Average Speed   TimeTime Time  
Current
 Dload  Upload   Total   SpentLeft  
Speed
  0 00 00 0  0  0 --:--:-- --:--:-- 
--:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- 
--:--:-- 0
curl: (67) Authentication failure

[user@aixsource:~/test]$ # scp with same keys works
[user@aixsource:~/test]$ scp -s -i $keydir/rsa foo 
remoteu...@rhel9server.foo.com:/tmp/bar
SSH is not allowed for i2i accounts.  Only scp and sftp will be able to 
continue.
foo 100%0 
0.0KB/s   00:00

[user@aixsource:~/test]$ # same curl as above, but using ed25519 keys, 
works
[user@aixsource:~/test]$ curl --trace file -u remoteuser: --key 
$keydir/ed25519 --pubkey $keydir/ed25519.pub -T foo \
 sftp://rhel9server.foo.com:/tmp/bar
  % Total% Received % Xferd  Average Speed   TimeTime Time  
Current
 Dload  Upload   Total   SpentLeft  
Speed
  0 00 00 0  0  0 --:--:--  0:00:02 
--:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:02 
--:--:-- 0

[user@aixsource:~/test]$ # scp with ed25519 keys works
[user@aixsource:~/test]$ scp -s -i $keydir/ed25519 foo 
remoteu...@rhel9server.foo.com:/tmp/bar
SSH is not allowed for i2i accounts.  Only scp and sftp will be able to 
continue.
foo 100%0 
0.0KB/s   00:00

Tracefile:

== Info: Host rhel9server.foo.com:22 was resolved.
== Info: IPv6: (none)
== Info: IPv4: 10.10.10.1
== Info:   Trying 10.10.10.1:22...
== Info: Connected to rhel9server.foo.com (10.10.10.1) port 22
== Info: Found host rhel9server.foo.com in 
/home/localuser/.ssh/known_hosts
== Info: Set "ecdsa-sha2-nistp256" as SSH hostkey type
== Info: SSH MD5 public key: NULL
== Info: SSH SHA256 public key: NULL
== Info: SSH host check: 0, key: KEYREDACTED
== Info: SSH authentication methods available: 
publickey,gssapi-keyex,gssapi-with-mic,password
== Info: Using SSH public key file '/home/localuser/test/rsa.pub'
== Info: Using SSH private key file '/home/localuser/test/rsa'
== Info: SSH public key authentication failed: Invalid signature for 
supplied public key, or bad username/public key combination
== Info: Failure connecting to agent
== Info: Authentication failure
== Info: Closing connection

So... anyone have ideas (other than perhaps recompiling newer source... our 
last resort)?

Thanks!
I

Re: issues with CURL using RSA keys + SFTP

[Ian Veach via curl-users]

> curl uses a SSH library to perform all the SSH protocol options. The curl
> version probably won't make as much of a difference as you think in this
> regard.
> > curl 8.6.0 (powerpc-ibm-aix7.1.5.0) libcurl/8.6.0 OpenSSL/1.1.1v 
> > zlib/1.2.13 libssh2/1.10.0 nghttp2/1.58.0 OpenLDAP/2.5.16
> This shows that curl is using libssh2 in your case, and that the version is
> over 3 years old. I'd try upgrading that before investigating too much deeper.

Thanks Dan.

I completely understand your valid points.  To answer:

I've taken a swing at this before, from the lib angle.  Everyone says it's 
someone else, including the libssh2 people.  I'm no crypto expert, but looking 
at the code it was hard to tell what did what when (to me).  But I do agree 
that the older code simply may have the issues simply because it's older.

Looks like one of our engineers told me porky pies though.  AIX Toolbox DOES 
offer a newer curl (8.11.1) with libssh2 1.11.0.  So I agree/think that's our 
first step (after a million steps to satisfy our customer base before "they 
allow us" to upgrade).  Then I can test again and see if our issue is fixed.

A follow up question:  Until we get to a newer version  what does curl (and 
presumably libssh2) honor as far as configuration options?  Is there any way or 
a workaround to add configuration on the client side to e.g. not use SHA1?

I fiddled with ssh_config, but curl (or libssh2) doesn't seem to honor that 
like ssh/scp does.  I see a LIBSSH2_NO_RSA_SHA1 option was added to libssh2 in 
1.11, but that's obviously compile time (and a newer version).  Again, we're 
trying to avoid custom compiling and interfering with the AIX Toolbox galaxy if 
we can.

Thanks,
Ian



PUBLIC RECORDS NOTICE: In accordance with NRS Chapter 239, this email and 
responses, unless otherwise made confidential by law, may be subject to the 
Nevada Public Records laws and may be disclosed to the public upon request.
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html

Re: issues with CURL using RSA keys + SFTP

[Ian Veach via curl-users]

I wrote:
> Looks like one of our engineers told me porky pies though.  AIX Toolbox DOES 
> offer a newer curl (8.11.1) with libssh2 1.11.0.  So I
> agree/think that's our first step (after a million steps to satisfy our 
> customer base before "they allow us" to upgrade).  Then I can test again
> and see if our issue is fixed.

We managed to upgrade to that curl+libssh2 in a test environment, and that 
solved the issue I reported.  Go figure.  🙂  Thanks!
PUBLIC RECORDS NOTICE: In accordance with NRS Chapter 239, this email and 
responses, unless otherwise made confidential by law, may be subject to the 
Nevada Public Records laws and may be disclosed to the public upon request.
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html