Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2235037

Summary: [17.0 to 17.1 Upgrade] [RHEL 9.0 to 9.2] SSH key will not work with in RHEL 9.2 due to they being less than 2048
Product: Red Hat OpenStack Reporter: chrisbro <chrisbro>
Component: tripleo-ansibleAssignee: Mikolaj Ciecierski <mciecier>
Status: CLOSED ERRATA QA Contact: Archana Singh <arcsingh>
Severity: low Docs Contact:
Priority: low    
Version: 17.1 (Wallaby)CC: arcsingh, bshephar, ccamacho, jamsmith, jpretori, mburns, mciecier, sgolovat
Target Milestone: z2Keywords: Triaged
Target Release: 17.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tripleo-ansible-3.3.1-17.1.20230921160833.d7d4f55.el9ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-01-16 14:30:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description chrisbro@redhat.com 2023-08-26 05:24:44 UTC
Description of problem:
Upgraded director node on version 17.1 from 17.0 will not connect to overcloud nodes using the `id_rsa.pub` and `id_rsa_tripleo.pub` keys

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.1/html-single/performing_a_minor_update_of_red_hat_openstack_platform/index#assembly_updating-the-overcloud_keeping-updated


Version-Release number of selected component (if applicable):
Red Hat OpenStack Platform release 17.1.0 (Wallaby)

How reproducible:
Upgrade director node to 17.1 from 17.0 and try to ssh to a overcloud node using the aforementioned keys


Steps to Reproduce:
1.Upgrade to 17.1 from 17.0 
2. ssh to one of the overcloud nodes. 
3.

Actual results:

```
(undercloud) [stack@director ~]$ ssh heat-admin.99.45                                                                                                                                                                             
load pubkey "/home/stack/.ssh/id_rsa": Invalid key length   
```

And during the update of the overcloud nodes at the first upgrade command that connects to the overcloud nodes 
  `openstack overcloud external-update run --stack <stack_name> --tags ovn`
```
PLAY [External update step 1] **************************************************                                                                                                                                                        	 
2023-08-26 14:11:37.826774 | 525400e9-eec8-171a-58c9-00000000012e |   	TASK | Force pull image in case image name doesn't change.                                                                                                    	 
2023-08-26 14:11:41.998325 | 525400e9-eec8-171a-58c9-00000000012e |  	FATAL | Force pull image in case image name doesn't change. | undercloud | item=overcloud-compute-0 | error={"ansible_loop_var": "item", "item": "overcloud-compute-
0", "msg": "Failed to connect to the host via ssh: load pubkey \"/home/stack/.ssh/id_rsa\": Invalid key length\r\nWarning: Permanently added '192.168.99.21' (ED25519) to the list of known hosts.\r\ntripleo-admin.99.21: Permission
 denied (publickey,password).", "unreachable": true}                                                                                                                                                                                    	 
2023-08-26 14:11:41.999642 | 525400e9-eec8-171a-58c9-00000000012e | 	TIMING | Force pull image in case image name doesn't change. | undercloud | 0:00:08.893209 | 4.17s                                                              	 
2023-08-26 14:11:46.136730 | 525400e9-eec8-171a-58c9-00000000012e |  	FATAL | Force pull image in case image name doesn't change. | undercloud | item=overcloud-compute-1 | error={"ansible_loop_var": "item", "item": "overcloud-compute-
1", "msg": "Failed to connect to the host via ssh: load pubkey \"/home/stack/.ssh/id_rsa\": Invalid key length\r\nWarning: Permanently added '192.168.99.38' (ED25519) to the list of known hosts.\r\ntripleo-admin.99.38: Permission
 denied (publickey,password).", "unreachable": true}                                                                                                                                                                                    	 
2023-08-26 14:11:46.137402 | 525400e9-eec8-171a-58c9-00000000012e | 	TIMING | Force pull image in case image name doesn't change. | undercloud | 0:00:13.030985 | 8.31s                                                              	 
2023-08-26 14:11:50.379641 | 525400e9-eec8-171a-58c9-00000000012e |  	FATAL | Force pull image in case image name doesn't change. | undercloud | item=overcloud-controller-0 | error={"ansible_loop_var": "item", "item": "overcloud-contr
oller-0", "msg": "Failed to connect to the host via ssh: load pubkey \"/home/stack/.ssh/id_rsa\": Invalid key length\r\nWarning: Permanently added '192.168.99.45' (ED25519) to the list of known hosts.\r\ntripleo-admin.99.45: Perm
ission denied (publickey,password).", "unreachable": true}                                                                                                                                                                              	 
2023-08-26 14:11:50.380326 | 525400e9-eec8-171a-58c9-00000000012e | 	TIMING | Force pull image in case image name doesn't change. | undercloud | 0:00:17.273909 | 12.55s                                                             	 
2023-08-26 14:11:54.567310 | 525400e9-eec8-171a-58c9-00000000012e |  	FATAL | Force pull image in case image name doesn't change. | undercloud | item=overcloud-controller-1 | error={"ansible_loop_var": "item", "item": "overcloud-contr
oller-1", "msg": "Failed to connect to the host via ssh: load pubkey \"/home/stack/.ssh/id_rsa\": Invalid key length\r\nWarning: Permanently added '192.168.99.55' (ED25519) to the list of known hosts.\r\ntripleo-admin.99.55: Perm
ission denied (publickey,password).", "unreachable": true}                                                                                                                                                                              	 
2023-08-26 14:11:54.567814 | 525400e9-eec8-171a-58c9-00000000012e | 	TIMING | Force pull image in case image name doesn't change. | undercloud | 0:00:21.461398 | 16.74s                                                             	 
2023-08-26 14:11:58.735350 | 525400e9-eec8-171a-58c9-00000000012e |  	FATAL | Force pull image in case image name doesn't change. | undercloud | item=overcloud-controller-2 | error={"ansible_loop_var": "item", "item": "overcloud-contr
oller-2", "msg": "Failed to connect to the host via ssh: load pubkey \"/home/stack/.ssh/id_rsa\": Invalid key length\r\nWarning: Permanently added '192.168.99.40' (ED25519) to the list of known hosts.\r\ntripleo-admin.99.40: Perm
ission denied (publickey,password).", "unreachable": true
```

Expected results:


Additional info:

A workaround to this issue would be below to get the upgrade completed. 
  https://access.redhat.com/solutions/6973518

Comment 2 Brendan Shephard 2023-08-26 06:59:02 UTC
Thanks for identifying this issue and raising the BZ Chris++

So, this was raised many moons ago:
https://bugzilla.redhat.com/show_bug.cgi?id=2151002#c13

A playbook was added to rotate said keys:
https://review.opendev.org/c/openstack/tripleo-ansible/+/875886

But do we call that playbook from anywhere to actually address this during the update?

Added in the Upgrades team to take a look at this BZ

Comment 18 errata-xmlrpc 2024-01-16 14:30:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 17.1.2 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:0209