Bug 2068423
| Summary: | OpenSSH client should fail with a human-readable error in case a connection can't be established with a server that uses an insecure host key (rsa-ssh) | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Thorsten Scherf <tscherf> |
| Component: | openssh | Assignee: | Zoltan Fridrich <zfridric> |
| Status: | CLOSED ERRATA | QA Contact: | Marek Havrila <mhavrila> |
| Severity: | high | Docs Contact: | Jan Fiala <jafiala> |
| Priority: | medium | ||
| Version: | 9.0 | CC: | ajschorr, davide, dbelyavs, ifelmail, jafiala, jjelen, mhavrila, ngompa13, omoris, zfridric |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openssh-8.7p1-17.el9 | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-11-15 11:21:44 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Thorsten Scherf
2022-03-25 09:22:16 UTC
*** Bug 2069495 has been marked as a duplicate of this bug. *** It's not enough to "disable" ssh-rsa* in libcrypto, they should be removed from the defaults from lists of the supported algos in ssh client and server both (directives hostkeyalgorithms hostbasedkeytypes casignaturealgorithms pubkeyacceptedkeytypes). The reason behind that is, it tries to find the algo in the specified order that the both ends support. If you just disable support of them in the libcrypto without removing from the lists (what's done now), if there are no other supported by the both ends algos with higher prio, it will pick this one, but connection will fail, while if it's not on the list, it will not be chosen. So, without updating the lists, you should expect your users to experience random ssh connection failures to different host (one client can connect to host A, but cannot to host B, while another client can connect to host B, but can connect to A), depending on versions and the orders in their configs. Ideally, you would need to check if the algo specified in ssh config is supported by libcrypto. IDK if it's possible to do that and if so, how easy it would be. But please, at least remove the disabled in libcrypto algorithms from the ssh defaults to prevent client/server handshake from choosing them and let the handshake a chance to pick supported algo at least by default. *** Bug 2088916 has been marked as a duplicate of this bug. *** *** Bug 2088386 has been marked as a duplicate of this bug. *** Let me try again, since my bug on this issue https://bugzilla.redhat.com/show_bug.cgi?id=2088916 was peremptorily closed. It seems that ssh_keysign is puking on a key that was generated by your very own sshd-keygen.target: [root@localhost ssh]# cat /usr/lib/systemd/system/sshd-keygen.target [Unit] Wants=sshd-keygen Wants=sshd-keygen Wants=sshd-keygen PartOf=sshd.service Just remove "rsa" if it is no longer supported. Please take a look at #2088916 again. Why are you generating a key that your own code refuses to sign? I show in that bug specific steps to generate an inscrutable error message. I wasted a lot of time debugging this before realizing that ssh-rsa keys simply aren't working any more. (In reply to Andrew Schorr from comment #5) > Please take a look at #2088916 again. Why are you generating a key that your > own code > refuses to sign? I show in that bug specific steps to generate an > inscrutable error > message. I wasted a lot of time debugging this before realizing that ssh-rsa > keys > simply aren't working any more. The RSA key is still ok. The SHA1 signatures are not ok. Please, provide complete debug log if you want somebody to look into your issue. My assumption is that you have "ssh-rsa" somewhere in some configuration file or you are connecting to some legacy system which does not support SHA2 signatures with RSA, which I already mentioned in the previous bug. Either I'm losing my mind, or running "/usr/libexec/openssh/sshd-keygen rsa" on a CentOS Stream 9
system (as it is invoked by sshd-keygen.target via sshd-keygen@.service) results in an ssh-rsa key,
not an SHA2 variant. I just did it, and here's what I get:
[root@ti138 system]# cat sshd-keygen@.service
[Unit]
Description=OpenSSH %i Server Key Generation
ConditionFileNotEmpty=|!/etc/ssh/ssh_host_%i_key
[Service]
Type=oneshot
EnvironmentFile=-/etc/sysconfig/sshd
ExecStart=/usr/libexec/openssh/sshd-keygen %i
[Install]
WantedBy=sshd-keygen.target
[root@ti138 system]# /usr/libexec/openssh/sshd-keygen rsa
[root@ti138 system]# ls -l /etc/ssh
total 584
-rw-r--r-- 1 root root 578094 Feb 22 07:32 moduli
-rw-r--r-- 1 root root 1921 Feb 22 07:32 ssh_config
drwxr-xr-x 2 root root 51 May 23 23:05 ssh_config.d
-rw-r----- 1 root ssh_keys 2578 May 24 08:03 ssh_host_rsa_key
-rw-r--r-- 1 root root 554 May 24 08:03 ssh_host_rsa_key.pub
lrwxrwxrwx 1 root root 23 May 13 2012 ssh_known_hosts -> ../conf/ssh_known_hosts
-rw------- 1 root root 3669 Feb 22 07:32 sshd_config
drwx------ 2 root root 51 May 23 23:05 sshd_config.d
[root@ti138 system]# cat /etc/ssh/ssh_host_rsa_key.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb2UjEJfzljkCiCvgy68U4kYhOwrbEQt2IoY9N4RLniqSKcAgmmPVnVlYEUPAoGVwAPlQoPp8N1etJgsJ8DV0FEwVIqwtoDkCCH48CmxOmQVj16UpwSbSpq3DvtQKYweQ+ziqf0mMci4dOfb2NyUQ6C5BZsxPPOBmgaqXFFETBYMaCqxRistkIHd/nLnVkRjni72Vy4PPQ8oxMibwRGokY9FLX1KyUgP9raSwy8RrEvNLFNzNGw7F1NzQMEkTQpnzXynGPhbwMBbDSTHii3rERTp/CkUruOzm18XWPSq22FJ6ROSjy8vd+swmEvc0dNp2CNAuzNkQmXoCmT1jgAqJs/cGqN7ViFC+qR8ioZne+RfevhIVKIUp0GXYmQ8+hZ2iPnaI+Z5SOgm0On8D/RZRuSaNmT0vhORNsPYZSYtK/3Ypj/AWSEmznGli6iYWfoucaRP01jHqZbRe/36pXJDZTKPt9JSieWPcj8AnzCPTOEcm47v5SYuzH64bihV0JlZM=
[root@ti138 system]# cat sshd-keygen.target
[Unit]
Wants=sshd-keygen
Wants=sshd-keygen
Wants=sshd-keygen
PartOf=sshd.service
So is the system supposed to be generating an ssh-rsa key by default? That seems weird
if they're no longer supported. Or is the idea that you're doing that for people
who are going to set the crypto policy to add back support for SHA1 RSA keys?
The ssh-keygen man page says:
-t dsa | ecdsa | ecdsa-sk | ed25519 | ed25519-sk | rsa
Specifies the type of key to create. The possible values are “dsa”, “ecdsa”, “ecdsa-sk”, “ed25519”, “ed25519-sk”, or “rsa”.
This flag may also be used to specify the desired signature type when signing certificates using an RSA CA key. The available RSA sig‐
nature variants are “ssh-rsa” (SHA1 signatures, not recommended), “rsa-sha2-256”, and “rsa-sha2-512” (the default).
It appears to me that it's actually producing ssh-rsa, not rsa-sha2-512, so this seems like a bug
in ssh-keygen. Is there some other option to ssh-keygen in addition to -t rsa that's required
to select rsa-sha2-512 instead of ssh-rsa?
But the main point remains that the error messages are really bad and offer no useful guidance about what might be going wrong. Please, check RFC 8332 [1]. The key type is not the same as a signature type. The key itself does not have anything to do with the digest. It comes into the play only when the signature is created and the algorithm is negotiated during the connection. In any case, we should try to make the error message more clear. https://datatracker.ietf.org/doc/html/rfc8332 OK, fair enough. But I don't think that affects the point I think I demonstrated (but maybe I'm hallucinating): the ssh-keygen -t rsa command is producing a key that is not accepted by the openssh keysign code. Why is that? Also, is there some way to examine a key labeled ssh-rsa to identify whether it's actually ssh-rsa (SHA1) or rsa-sha2-256 or rsa-sha2-512? How can one tell them apart? Sorry if this is a stupid question. Can you provide the debug log and affected server and client configuration? The key is ssh-rsa always. There are no labels on them. The difference is only in the way how it is used by the client and servers to create signature (and verify it on the other hand indeed, but this side does not have any control of what the signer did). Copying from bug 2088916: Steps to Reproduce: 1. In a freshly installed CentOS Stream 9 qemu kvm, create /etc/ssh/sshd_config.d/99-local.conf with these contents: HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostbasedAuthentication yes PasswordAuthentication no Then create /etc/ssh/ssh_config.d/99-local.conf with these contents: HostbasedAuthentication yes EnableSSHKeysign yes 2. systemctl restart sshd 3. ssh -n localhost uptime Actual results: [root@localhost ssh]# ssh -n localhost uptime ssh-keysign: sshkey_sign failed: error in libcrypto ssh_keysign: no reply sign using hostkey ssh-rsa SHA256:RUgzE2VqS2tcUzf1UM+x6UQX2GzQF25mL+4HPn+lScU failed root@localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,hostbased). [root@localhost ~]# ssh -v -n localhost uptime OpenSSH_8.7p1, OpenSSL 3.0.1 14 Dec 2021 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config debug1: Reading configuration data /etc/ssh/ssh_config.d/99-local.conf debug1: configuration requests final Match pass debug1: re-parsing configuration debug1: Reading configuration data /etc/ssh/ssh_config debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config debug1: Reading configuration data /etc/ssh/ssh_config.d/99-local.conf debug1: Connecting to localhost [::1] port 22. debug1: Connection established. debug1: identity file /root/.ssh/id_rsa type -1 debug1: identity file /root/.ssh/id_rsa-cert type -1 debug1: identity file /root/.ssh/id_dsa type -1 debug1: identity file /root/.ssh/id_dsa-cert type -1 debug1: identity file /root/.ssh/id_ecdsa type -1 debug1: identity file /root/.ssh/id_ecdsa-cert type -1 debug1: identity file /root/.ssh/id_ecdsa_sk type -1 debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1 debug1: identity file /root/.ssh/id_ed25519 type -1 debug1: identity file /root/.ssh/id_ed25519-cert type -1 debug1: identity file /root/.ssh/id_ed25519_sk type -1 debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1 debug1: identity file /root/.ssh/id_xmss type -1 debug1: identity file /root/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_8.7 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.7 debug1: compat_banner: match: OpenSSH_8.7 pat OpenSSH* compat 0x04000000 debug1: Authenticating to localhost:22 as 'root' debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: aes256-gcm MAC: <implicit> compression: none debug1: kex: client->server cipher: aes256-gcm MAC: <implicit> compression: none debug1: kex: curve25519-sha256 need=32 dh_need=32 debug1: kex: curve25519-sha256 need=32 dh_need=32 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: SSH2_MSG_KEX_ECDH_REPLY received debug1: Server host key: ecdsa-sha2-nistp256 SHA256:54MUE5WrCw0N8jfadIPyz5PezFb0QvitCSjcFlUHh68 debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: Host 'localhost' is known and matches the ECDSA host key. debug1: Found key in /root/.ssh/known_hosts:1 debug1: rekey out after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: rekey in after 4294967296 blocks debug1: Will attempt key: /root/.ssh/id_rsa debug1: Will attempt key: /root/.ssh/id_dsa debug1: Will attempt key: /root/.ssh/id_ecdsa debug1: Will attempt key: /root/.ssh/id_ecdsa_sk debug1: Will attempt key: /root/.ssh/id_ed25519 debug1: Will attempt key: /root/.ssh/id_ed25519_sk debug1: Will attempt key: /root/.ssh/id_xmss debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256,webauthn-sk-ecdsa-sha2-nistp256> debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,hostbased debug1: Next authentication method: gssapi-with-mic debug1: No credentials were supplied, or the credentials were unavailable or inaccessible No Kerberos credentials available (default cache: KCM:) debug1: No credentials were supplied, or the credentials were unavailable or inaccessible No Kerberos credentials available (default cache: KCM:) debug1: Next authentication method: hostbased debug1: userauth_hostbased: trying hostkey ssh-ed25519 SHA256:RwdE2ivivfuECyGCYfJQL3bFjr82J+IRA2JGHdM57Pg debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,hostbased debug1: userauth_hostbased: trying hostkey ecdsa-sha2-nistp256 SHA256:54MUE5WrCw0N8jfadIPyz5PezFb0QvitCSjcFlUHh68 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,hostbased debug1: userauth_hostbased: trying hostkey ssh-rsa SHA256:JmPpNluivxIVVtgozzkc35z7wH3asKWddp3WcKQUtsM ssh-keysign: sshkey_sign failed: error in libcrypto ssh_keysign: no reply sign using hostkey ssh-rsa SHA256:JmPpNluivxIVVtgozzkc35z7wH3asKWddp3WcKQUtsM failed debug1: Next authentication method: publickey debug1: Trying private key: /root/.ssh/id_rsa debug1: Trying private key: /root/.ssh/id_dsa debug1: Trying private key: /root/.ssh/id_ecdsa debug1: Trying private key: /root/.ssh/id_ecdsa_sk debug1: Trying private key: /root/.ssh/id_ed25519 debug1: Trying private key: /root/.ssh/id_ed25519_sk debug1: Trying private key: /root/.ssh/id_xmss debug1: No more authentication methods to try. root@localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,hostbased). [root@localhost ~]# Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (openssh bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:8375 https://bugzilla.redhat.com/show_bug.cgi?id=2209096#c11 contains the explanation why this approach was chosen. |