Bug 2058894

Summary:	Server fingerprints not managed properly
Product:	Red Hat Satellite	Reporter:	Gary Scarborough <gscarbor>
Component:	Remote Execution	Assignee:	Adam Ruzicka <aruzicka>
Status:	CLOSED ERRATA	QA Contact:	Peter Ondrejka <pondrejk>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.9.7	CC:	aruzicka, ehelms, lstejska, mkalyat, rdesouza
Target Milestone:	6.11.0	Keywords:	Regression, Triaged, WorkAround
Target Release:	Unused
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	tfm-rubygem-foreman_remote_execution-5.0.5	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-07-05 14:34:01 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Gary Scarborough 2022-02-26 19:09:24 UTC

Description of problem:

This is a regression of bug 15005842 which was supposed to be fixed in 6.8.

I have tested this in Sat 6.8.6, 6.9.7 and 6.10.2.  So I dont think this WAS actually fixed.

If the host fingerprint changes on a host (like its rebuilt) satellite does not gracefully manage the change.  You just end up with the error:

Error initializing command: Net::SSH::HostKeyMismatch - fingerprint df:52:3f:01:f6:7b:85:62:fa:26:ac:aa:56:a1:65:97 does not match for "node761.local.net,192.168.1.21"


There is a KCS: https://access.redhat.com/solutions/3708891 with a workaround.  



Version-Release number of selected component (if applicable):

Sat 6.8.6, 6.9.7, 6.10.2


How reproducible:

Always

Steps to Reproduce:
1. Register a host to satellite and setup ssh keys for REX.

2. Change the host fingerprint keys (rm -f /etc/ssh/*key ; systemctl sshd restart)

3. Run a new job to that host, you will hit the error.



Actual results:

You get the error:  Error initializing command: Net::SSH::HostKeyMismatch - fingerprint df:52:3f:01:f6:7b:85:62:fa:26:ac:aa:56:a1:65:97 does not match for "node761.local.net,192.168.1.21"


Expected results:

Satellite should remove the ssh key for this host when the registration to removed or overwritten.

Comment 1 Gary Scarborough 2022-02-26 19:14:53 UTC

Typo in the post above, the regression is of bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1505842

Comment 2 Gary Scarborough 2022-02-26 20:10:21 UTC

CLARIFICATION:

So on further testing, the known host key is removed if you delete the host from satellite.

However, if you re-register the host with an AK and from the host without deleting the old host first, the known host entry is not removed.

So I guess this is an RFE to enhance the fix to include registrations.  Basically if the uuid for a host is changing, the known host entry for it in foreman should be removed.

Comment 3 Adam Ruzicka 2022-02-28 07:42:20 UTC

I'm afraid I'm not following completely. From what I can tell the steps are:

1) Register a host somehow
2) Mess with its keys
3) Register it again through a different mechanism

Why should we expect that the host has different identity in 1 and 3?

Comment 4 Brad Buckingham 2022-02-28 14:08:13 UTC

Adding needinfo based upon comment 3.

Comment 5 Gary Scarborough 2022-03-01 14:14:16 UTC

Sorry for the confusion.  The keys will change if the host is re-built for some reason.  Either system failure or they build a new server instead of the OS upgrade, converting a RHEL 6 box to RHEL 8.  

Deleting the host information will remove the host from the known hosts file.

Re-registering it (which seems to happen a lot) does not remove the key.

Comment 6 Bryan Kearney 2022-03-14 16:05:51 UTC

Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/34574 has been resolved.

Comment 7 Peter Ondrejka 2022-04-19 12:16:38 UTC

Verified on Satellite 6.11 snap 16, entries from the host_proxy_invocations table are successfully deleted on host re-registration

Comment 10 errata-xmlrpc 2022-07-05 14:34:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5498