Bug 2088298
| Summary: | [Azure] [RHEL 9] [Packer] SSH is failing with error " userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]" | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | anujmaurya | ||||||||||||
| Component: | openssh | Assignee: | Dmitry Belyavskiy <dbelyavs> | ||||||||||||
| Status: | CLOSED NOTABUG | QA Contact: | BaseOS QE Security Team <qe-baseos-security> | ||||||||||||
| Severity: | urgent | Docs Contact: | |||||||||||||
| Priority: | medium | ||||||||||||||
| Version: | 9.0 | CC: | anhvo, campbell.vertesi, cpatterson, eterrell, huzhao, jjelen, manikroy, mathapli, mhavrila, sisatia | ||||||||||||
| Target Milestone: | rc | Keywords: | Triaged | ||||||||||||
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
||||||||||||
| Hardware: | x86_64 | ||||||||||||||
| OS: | Linux | ||||||||||||||
| Whiteboard: | |||||||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||
| Clone Of: | Environment: | ||||||||||||||
| Last Closed: | 2022-06-20 06:45:51 UTC | Type: | Bug | ||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||
| Documentation: | --- | CRM: | |||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||
| Embargoed: | |||||||||||||||
| Attachments: |
|
||||||||||||||
Created attachment 1881186 [details]
/var/log folder
Created attachment 1881198 [details]
ssd -T
Please, provide a debug log from the connection. The ssh-rsa authentication mechanism is using SHA-1 and is no longer allowed in RHEL9 (see release notes) and you are probably using some client that does not support RFC 8332 [1]. If you need to support legacy clients, you can change the system-wide crypto policy on RHEL9 to LEGACY or DEFAULT:SHA1 to allow SHA-1 signatures, but this is not recommended and this change allowing SHA-1 in RHEL9 will not be reverted. See also the following KBAs: * https://access.redhat.com/solutions/6816771 * https://access.redhat.com/solutions/6954602 [1] https://datatracker.ietf.org/doc/html/rfc8332 Seems like permission issue on private key for packer ssh -i C:\vsagent\_work\7\s\azure-arm-pkrvmp5lc30yyi9.pem packer.205.36 -v OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5 debug1: Connecting to 20.212.205.36 [20.212.205.36] port 22. debug1: Connection established. debug1: key_load_public: No such file or directory debug1: identity file C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem type -1 debug1: key_load_public: No such file or directory debug1: identity file C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_for_Windows_7.7 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.7 debug1: match: OpenSSH_8.7 pat OpenSSH* compat 0x04000000 debug1: Authenticating to 20.212.205.36:22 as 'packer' debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: chacha20-poly1305 MAC: <implicit> compression: none debug1: kex: client->server cipher: chacha20-poly1305 MAC: <implicit> compression: none debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key: ecdsa-sha2-nistp256 SHA256:qvb86D3C1BLdGle06yBBSi0tJ/iQxvXwqfmC26WdG+k debug1: Host '20.212.205.36' is known and matches the ECDSA host key. debug1: Found key in C:\\Users\\azlinux/.ssh/known_hosts:12 debug1: rekey after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: rekey after 134217728 blocks debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256,webauthn-sk-ecdsa-sha2-nistp256> debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Trying private key: C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions for 'C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. Load key "C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem": bad permissions debug1: No more authentication methods to try. packer.205.36: Permission denied (publickey,gssapi-keyex,gssapi-with-mic). That sounds like a completely different error than the original one. Yes, I got this error when I tried to log in manually from the cmd prompt. I downloaded the key generated by the packer to test locally. (In reply to anujmaurya from comment #6) > Yes, I got this error when I tried to log in manually from the cmd prompt. I > downloaded the key generated by the packer to test locally. Regarding the new error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions for 'C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. Could you please check the permission of 'C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem'? It should be "-rw-------.", could you please try ssh again after the check/fix? Thanks! # ls -al C:\\vsagent\\_work\\7\\s\\azure-arm-pkrvmp5lc30yyi9.pem hey huzhao, The permission is on windows VM. I was able to log in after granting all access to the current user last time. On linux the permission is 600. let me repro and give you the required info. Can you check the openssh version on target? For the error described on the summary (and by the ssh logs attached) seems like the guest doesn't support ssh-rsa (see pubkeyacceptedalgorithms on the "ssd -T" file attached). You should be able to confirm this by checking on the guest: $ cat /etc/ssh/sshd_config|grep ssh-rsa You should have: PubkeyAcceptedKeyTypes=+ssh-rsa Now, cloud-init already sets this option upon deployment, so I'm not sure why this is failing. https://gitlab.com/redhat/rhel/src/cloud-init/-/blob/rhel-9.0.0/rhel/cloud.cfg#L10 How the deploy was done? (In reply to Eduardo Otubo from comment #9) > You should have: > > PubkeyAcceptedKeyTypes=+ssh-rsa This should not be in any configuration file. The signature algorithms are configured with crypto policies and allowing SHA-1 signatures in OpenSSH will not allow it in the OpenSSL. > Now, cloud-init already sets this option upon deployment, so I'm not sure > why this is failing. > https://gitlab.com/redhat/rhel/src/cloud-init/-/blob/rhel-9.0.0/rhel/cloud. > cfg#L10 These are just key types. The problem is with the signature type using SHA-1. Please, see https://access.redhat.com/articles/6846411 for rhel -8 pubkeyauthentication yes pubkeyacceptedkeytypes ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521-cert-v01,ssh-ed25519-cert-v01,rsa-sha2-512-cert-v01,rsa-sha2-256-cert-v01,ssh-rsa-cert-v01,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa for rhel 9 pubkeyauthentication yes pubkeyacceptedalgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01,sk-ecdsa-sha2-nistp256,sk-ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01,ssh-ed25519,ssh-ed25519-cert-v01,sk-ssh-ed25519,sk-ssh-ed25519-cert-v01,rsa-sha2-256,rsa-sha2-256-cert-v01,rsa-sha2-512,rsa-sha2-512-cert-v01 pubkeyauthoptions none I agree with Jakub, SHA-1 is deprecated, probably some clients are supporting SHA-1. in the above comment ssh-rsa is not present in "sshd -T | grep pub" another team is getting the error "May 23 07:54:04 winvm8755cca3-a sshd[26485]: Unable to negotiate with 52.160.65.37 port 7297: no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth] May 23 07:55:04 winvm8755cca3-a sshd[26491]: Unable to negotiate with 52.160.65.37 port 7298: no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth] " attaching logs and journalctl sshd log Created attachment 1882367 [details]
/var/logs folder
Created attachment 1882368 [details]
JOurnalctl -u sshd
ssh-dss is deprecated by upstream for a long time Nothing from above doesn't look like a bug from our side. Dropping ssh-rsa support was announced for RHEL9, ssh-dss is also deprecated, and currently I don't see anything to do from the developer side. Could you please follow the normal procedures via support in case you still experience problems? Sure, the bug was created while building the RHEL 9 image on the Azure platform. Thanks for letting us know about the changes. Is it possible to add me and my teammates for all the image-related bugs, especially before any major release? It will help us in building the correct RHEL images. Please close the bug. (In reply to Dmitry Belyavskiy from comment #17) > Nothing from above doesn't look like a bug from our side. Dropping ssh-rsa > support was announced for RHEL9, ssh-dss is also deprecated, and currently I > don't see anything to do from the developer side. > > Could you please follow the normal procedures via support in case you still > experience problems? As per this comment I'll CLOSE as NOTABUG. If this is not correct please reopen it. Thanks. |
Created attachment 1881185 [details] journallogs for sshd Description of problem: The ssh is failing with the error " userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]" on creating VM from packer azure-arm builder. Started OpenSSH server daemon. May 19 05:51:39 pkrvmp5lc30yyi9 sshd[19961]: error: kex_exchange_identification: Connection closed by remote host May 19 05:51:39 pkrvmp5lc30yyi9 sshd[19961]: Connection closed by 104.44.111.134 port 26235 May 19 05:51:40 pkrvmp5lc30yyi9 sshd[20293]: userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth] May 19 05:51:40 pkrvmp5lc30yyi9 sshd[20293]: Connection closed by authenticating user packer 104.44.111.134 port 48806 [preauth] May 19 05:51:47 pkrvmp5lc30yyi9 sshd[26460]: error: kex_exchange_identification: Connection closed by remote host May 19 05:51:47 pkrvmp5lc30yyi9 sshd[26460]: Connection closed by 104.44.111.134 port 49699 This is also failing for a lot of SSH SDKs like Certification service is also failing since it is not able to SSH into the VM. Adding journal logs for ssh, logs for /var/log and the sshd -T o/p Version-Release number of selected component (if applicable): 9.0 How reproducible: always Steps to Reproduce: 1. CReate RHEL 9 VM with the azure-arm builder from Packer. 2. Provisioning VM with cloud-init 3. Packer is failing to ssh, failing in preauth Actual results: packer is not able to ssh into the provisioned RHEL 9 VM Expected results: Packer should be able to ssh into the VM Additional info: