Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2150710

Summary:	Cannot connect ssh server when a client attempted to connect with ssh-rsa
Product:	Red Hat Enterprise Linux 9	Reporter:	yatanaka
Component:	openssh	Assignee:	Dmitry Belyavskiy <dbelyavs>
Status:	CLOSED WONTFIX	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	9.0	CC:	jjelen
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-02-07 12:22:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description yatanaka 2022-12-05 05:51:11 UTC

Description of problem:

My customer is experiencing an ssh issue on RHOSP 17.0 environment, which is based on RHEL 9.0.
And I reproduced the issue on pure RHEL 9.0 lab.

I commented out "Include /etc/ssh/sshd_config.d/*.conf" line in sshd_config.
Note: /etc/ssh/sshd_config.d/*.conf is not included in RHOSP 17.0 environment by default.
~~~
[root@utils ~]# grep Include /etc/ssh/sshd_config
#Include /etc/ssh/sshd_config.d/*.conf <===============(*)comment out

[root@utils ~]# grep Include /etc/ssh/sshd_config.d/50-redhat.conf 
Include /etc/crypto-policies/back-ends/opensshserver.config <===============(*)config file of crypto-policies
~~~

Then, SSH with HostKeyAlgorithms=ssh-rsa,ecdsa-sha2-nistp256 fails.
~~~
# ssh -o HostKeyAlgorithms=ssh-rsa,ecdsa-sha2-nistp256 test.0.253 
Connection closed by 10.0.0.253 port 22
~~~

When I only enabled ecdsa-sha2-nistp256, this command succeeds.
~~~
# ssh  -o HostKeyAlgorithms=ecdsa-sha2-nistp256 test.0.253 
The authenticity of host '10.0.0.253 (10.0.0.253)' can't be established.
ECDSA key fingerprint is SHA256:pP+iApY5S9ZwxBkOWjARnrAz40sMF/4pFtxh96sUGYA.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.253' (ECDSA) to the list of known hosts.
Password: 
Register this system with Red Hat Insights: insights-client --register
Create an account or view all your systems at https://red.ht/insights-dashboard
Last login: Mon Dec  5 14:36:20 2022 from 10.0.0.1
[test@utils ~]$ 
~~~

I updated to RHEL 9.1, but this issue still remain.
~~~
[root@utils ~]# rpm -qa |grep  openssh
openssh-8.7p1-24.el9_1.x86_64
openssh-server-8.7p1-24.el9_1.x86_64
openssh-clients-8.7p1-24.el9_1.x86_64
~~~

I think ssh should succeed even if HostKeyAlgorithms contains ssh-rsa because ecdsa-sha2-nistp256 is also enabled and ecdsa-sha2-nistp256 works.



Version-Release number of selected component (if applicable):
RHEL 9.0, RHEL 9.1

How reproducible:

Steps to Reproduce:
1. Install RHEL 9.0
2. Comment out "Include /etc/ssh/sshd_config.d/*.conf" line in sshd_config.
3. Run "systemctl restart sshd"
3. Attempt to connect to this RHEL 9.0 server by "ssh -o HostKeyAlgorithms=ssh-rsa,ecdsa-sha2-nistp256" command


Actual results:
SSH fails


Expected results:
SSH succeeds


Additional info:
The following case looks a very similar to this issue.
https://bugzilla.redhat.com/show_bug.cgi?id=2088750
But I created this new bugzilla ticket because my customer is experiencing this issue on RHOSP 17.0 and we want fix on RHEL 9.0, not RHEL 9.1.

Comment 1 yatanaka 2022-12-05 06:05:32 UTC

The following is -vvv result.
~~~
# ssh  -o HostKeyAlgorithms=ssh-rsa,ecdsa-sha2-nistp256 stack.0.253 -vvv 
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.0.253 originally 10.0.0.253
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug2: resolve_canonicalize: hostname 10.0.0.253 is address
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.0.253 originally 10.0.0.253
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: ssh_connect_direct
debug1: Connecting to 10.0.0.253 [10.0.0.253] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.7
debug1: match: OpenSSH_8.7 pat OpenSSH* compat 0x04000000
debug2: fd 5 setting O_NONBLOCK
debug1: Authenticating to 10.0.0.253:22 as 'stack'
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa,ecdsa-sha2-nistp256
debug2: ciphers ctos: aes256-gcm,chacha20-poly1305,aes256-ctr,aes256-cbc,aes128-gcm,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm,chacha20-poly1305,aes256-ctr,aes256-cbc,aes128-gcm,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: compression ctos: none,zlib,zlib
debug2: compression stoc: none,zlib,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm
debug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm
debug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib
debug2: compression stoc: none,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes256-gcm MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
Connection closed by 10.0.0.253 port 22
~~~

I can see no logs in /var/log/secure.
~~~
[root@utils ~]# tail -f /var/log/secure -n0

~~~

Comment 2 Dmitry Belyavskiy 2022-12-06 09:35:12 UTC

The immediate problem looks related to lack of support of SHA1 hash algorithm for signature by default. In case of normal RHEL installation you can enable it via crypto-policy DEFAULT:SHA1. You also have a workaround to not explicitly enable ssh-rsa (or disable it via local ssh config).

Could you please also send a log when you enable only ecdsa key?

Comment 4 yatanaka 2022-12-06 23:58:24 UTC

> In case of normal RHEL installation you can enable it via crypto-policy DEFAULT:SHA1. 

Yes, I confirmed that ssh works in case of normal RHEL installation and crypto-policy DEFAULT:SHA1. 

> You also have a workaround to not explicitly enable ssh-rsa (or disable it via local ssh config).

Yes, I think so, too.

> Could you please also send a log when you enable only ecdsa key?

I'm pasting the log when I only enabled ecdsa key below.
I think the most important difference is "debug1: kex: host key algorithm:" line.
In this case, ecdsa-sha2-nistp256 is selected.
~~~
[root@dell-r730-056 ~]# ssh  -o HostKeyAlgorithms=ecdsa-sha2-nistp256 test.0.253  -vvv
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.0.253 originally 10.0.0.253
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug2: resolve_canonicalize: hostname 10.0.0.253 is address
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.0.253 originally 10.0.0.253
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: ssh_connect_direct
debug1: Connecting to 10.0.0.253 [10.0.0.253] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.7
debug1: match: OpenSSH_8.7 pat OpenSSH* compat 0x04000000
debug2: fd 5 setting O_NONBLOCK
debug1: Authenticating to 10.0.0.253:22 as 'test'
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256
debug2: ciphers ctos: aes256-gcm,chacha20-poly1305,aes256-ctr,aes256-cbc,aes128-gcm,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm,chacha20-poly1305,aes256-ctr,aes256-cbc,aes128-gcm,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: compression ctos: none,zlib,zlib
debug2: compression stoc: none,zlib,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm
debug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm
debug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib
debug2: compression stoc: none,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:PtvxQ/Pjk4Yb3LUCCZTpy1adc1j7xviWXEvVZl7FYzg
The authenticity of host '10.0.0.253 (10.0.0.253)' can't be established.
ECDSA key fingerprint is SHA256:PtvxQ/Pjk4Yb3LUCCZTpy1adc1j7xviWXEvVZl7FYzg.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.253' (ECDSA) to the list of known hosts.
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /root/.ssh/id_rsa RSA SHA256:XRFFPbgIqbHLFdgGBX7tkJ/x4bdxIIVdt2Pb/KGnPUE
debug1: Will attempt key: /root/.ssh/id_dsa 
debug1: Will attempt key: /root/.ssh/id_ecdsa 
debug1: Will attempt key: /root/.ssh/id_ed25519 
debug1: Will attempt key: /root/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256,webauthn-sk-ecdsa-sha2-nistp256>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: start over, passed a different list publickey,password,keyboard-interactive
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /root/.ssh/id_rsa RSA SHA256:XRFFPbgIqbHLFdgGBX7tkJ/x4bdxIIVdt2Pb/KGnPUE
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Trying private key: /root/.ssh/id_dsa
debug3: no such identity: /root/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ecdsa
debug3: no such identity: /root/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ed25519
debug3: no such identity: /root/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /root/.ssh/id_xmss
debug3: no such identity: /root/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug3: send packet: type 50
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: receive packet: type 60
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password: 
debug3: send packet: type 61
debug3: receive packet: type 60
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 0
debug3: send packet: type 61
debug3: receive packet: type 52
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to 10.0.0.253 ([10.0.0.253]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00 want_reply 0
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 5 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x48
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env LS_COLORS
debug3: Ignored env SSH_CONNECTION
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env HISTCONTROL
debug3: Ignored env GUESTFISH_RESTORE
debug3: Ignored env HOSTNAME
debug3: Ignored env GUESTFISH_INIT
debug3: Ignored env which_declare
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env USER
debug3: Ignored env GUESTFISH_PS1
debug3: Ignored env SELINUX_ROLE_REQUESTED
debug3: Ignored env PWD
debug3: Ignored env HOME
debug3: Ignored env SSH_CLIENT
debug3: Ignored env SELINUX_LEVEL_REQUESTED
debug3: Ignored env SSH_TTY
debug3: Ignored env MAIL
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug1: Sending env XMODIFIERS = @im=ibus
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env SELINUX_USE_CURRENT_RANGE
debug3: Ignored env SHLVL
debug3: Ignored env LOGNAME
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env PATH
debug3: Ignored env GUESTFISH_OUTPUT
debug3: Ignored env HISTSIZE
debug3: Ignored env LESSOPEN
debug3: Ignored env BASH_FUNC_which%%
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Register this system with Red Hat Insights: insights-client --register
Create an account or view all your systems at https://red.ht/insights-dashboard
Last login: Wed Dec  7 08:29:37 2022 from 10.0.0.1
~~~

Comment 5 yatanaka 2022-12-07 00:09:33 UTC

> In case of normal RHEL installation you can enable it via crypto-policy DEFAULT:SHA1. 

And also, SSH works in case of in case of normal RHEL installation and crypto-policy DEFAULT, not DEFAULT:SHA1. 
In this case, ssh-rsa is disabled completely in server side, so ecdsa-sha2-nistp256 is chosen as the host key algorithm.
That's why crypto-policy DEFAULT works.

~~~
[root@utils ~]# sudo update-crypto-policies --set DEFAULT
Setting system policy to DEFAULT
Note: System-wide crypto policies are applied on application start-up.
It is recommended to restart the system for the change of policies
to fully take place.
[root@utils ~]# systemctl restart sshd
[root@utils ~]# 

[root@dell-r730-056 ~]# ssh  -o HostKeyAlgorithms=ssh-rsa,ecdsa-sha2-nistp256 test.0.253  -vvv
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.0.253 originally 10.0.0.253
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug2: resolve_canonicalize: hostname 10.0.0.253 is address
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.0.253 originally 10.0.0.253
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: ssh_connect_direct
debug1: Connecting to 10.0.0.253 [10.0.0.253] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.7
debug1: match: OpenSSH_8.7 pat OpenSSH* compat 0x04000000
debug2: fd 5 setting O_NONBLOCK
debug1: Authenticating to 10.0.0.253:22 as 'test'
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa,ecdsa-sha2-nistp256
debug2: ciphers ctos: aes256-gcm,chacha20-poly1305,aes256-ctr,aes256-cbc,aes128-gcm,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm,chacha20-poly1305,aes256-ctr,aes256-cbc,aes128-gcm,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: compression ctos: none,zlib,zlib
debug2: compression stoc: none,zlib,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 <=================================(*)SSH server doesn't advertise ssh-rsa
debug2: ciphers ctos: aes256-gcm,chacha20-poly1305,aes256-ctr,aes128-gcm,aes128-ctr
debug2: ciphers stoc: aes256-gcm,chacha20-poly1305,aes256-ctr,aes128-gcm,aes128-ctr
debug2: MACs ctos: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm,hmac-sha1-etm,umac-128-etm,hmac-sha2-512-etm,hmac-sha2-256,hmac-sha1,umac-128,hmac-sha2-512
debug2: compression ctos: none,zlib
debug2: compression stoc: none,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:PtvxQ/Pjk4Yb3LUCCZTpy1adc1j7xviWXEvVZl7FYzg
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
The authenticity of host '10.0.0.253 (10.0.0.253)' can't be established.
ECDSA key fingerprint is SHA256:PtvxQ/Pjk4Yb3LUCCZTpy1adc1j7xviWXEvVZl7FYzg.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.253' (ECDSA) to the list of known hosts.
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /root/.ssh/id_rsa RSA SHA256:XRFFPbgIqbHLFdgGBX7tkJ/x4bdxIIVdt2Pb/KGnPUE
debug1: Will attempt key: /root/.ssh/id_dsa 
debug1: Will attempt key: /root/.ssh/id_ecdsa 
debug1: Will attempt key: /root/.ssh/id_ed25519 
debug1: Will attempt key: /root/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256,webauthn-sk-ecdsa-sha2-nistp256>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)


debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)


debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /root/.ssh/id_rsa RSA SHA256:XRFFPbgIqbHLFdgGBX7tkJ/x4bdxIIVdt2Pb/KGnPUE
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /root/.ssh/id_dsa
debug3: no such identity: /root/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ecdsa
debug3: no such identity: /root/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ed25519
debug3: no such identity: /root/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /root/.ssh/id_xmss
debug3: no such identity: /root/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
test.0.253's password: 
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 52
debug1: Authentication succeeded (password).
Authenticated to 10.0.0.253 ([10.0.0.253]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00 want_reply 0
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 5 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x48
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env LS_COLORS
debug3: Ignored env SSH_CONNECTION
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env HISTCONTROL
debug3: Ignored env GUESTFISH_RESTORE
debug3: Ignored env HOSTNAME
debug3: Ignored env GUESTFISH_INIT
debug3: Ignored env which_declare
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env USER
debug3: Ignored env GUESTFISH_PS1
debug3: Ignored env SELINUX_ROLE_REQUESTED
debug3: Ignored env PWD
debug3: Ignored env HOME
debug3: Ignored env SSH_CLIENT
debug3: Ignored env SELINUX_LEVEL_REQUESTED
debug3: Ignored env SSH_TTY
debug3: Ignored env MAIL
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug1: Sending env XMODIFIERS = @im=ibus
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env SELINUX_USE_CURRENT_RANGE
debug3: Ignored env SHLVL
debug3: Ignored env LOGNAME
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env PATH
debug3: Ignored env GUESTFISH_OUTPUT
debug3: Ignored env HISTSIZE
debug3: Ignored env LESSOPEN
debug3: Ignored env BASH_FUNC_which%%
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Register this system with Red Hat Insights: insights-client --register
Create an account or view all your systems at https://red.ht/insights-dashboard
Last login: Wed Dec  7 08:30:12 2022 from 10.0.0.1
[test@utils ~]$ 
~~~

I feel it's strange that the default value of /etc/ssh/sshd_config enables ssh-rsa even though ssh-rsa doesn't work...

Comment 6 Jakub Jelen 2022-12-07 08:53:55 UTC

This is a problem in the openssh does not know if the openssl allows sha1 signatures or not. It can make just the guess from the crypto policies provided configuration files, but the HostkeyAlgorithms on server is derived only from the provided hostkeys (should be filtered with the crypto policies algorithms probably).