Bug 2175821

Summary:	Disable use of the SSH agent for the rescue test
Product:	Red Hat OpenStack	Reporter:	Martin Kopec <mkopec>
Component:	python-ironic-tests-tempest	Assignee:	Lukas Piwowarski <lpiwowar>
Status:	CLOSED ERRATA	QA Contact:	James E. LaBarre <jlabarre>
Severity:	medium	Docs Contact:
Priority:	high
Version:	17.1 (Wallaby)	CC:	eshames, lpiwowar, pweeks, sbaker
Target Milestone:	ga	Keywords:	Triaged
Target Release:	17.1	Flags:	lpiwowar: needinfo+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-ironic-tests-tempest-2.6.0-1.20230309130906.4684f91.el9ost	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-08-16 01:13:59 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2166951
Bug Blocks:

Description Martin Kopec 2023-03-06 15:43:58 UTC

Description of problem:

Ironic tempest plugin rescue scenario test fails when trying to ssh to the ironic node.

While debugging the ``rescue`` test functionality with ironic's
tempest plugin, we discovered that if the environment suggests the
agent is available, then we may enter a situation where the test
can fail because paramiko prefers ssh over password authentication.

This is important, because for rescue functionality in particular,
it is password authentication based without the use of SSH keys,
as a temporary password is generated by the services and provided
to the user requesting to rescue the instance/node.

Instead of trying to make an assumption that password being present
means we should just disable the agent, explicitly allow the caller
to specify it.

Version-Release number of selected component (if applicable):


How reproducible:
Every time

Steps to Reproduce:
1. run ironic tempest rescue test via jenkins
2.
3.

Actual results:
ssh with new user and password fails

Expected results:
ssh succeeds and test passes

Additional info:
using a workaround with "unset SSH_AUTH_SOCK" just before the test run does solve the issue, but it is unfeasible to use this for just this test, it would impact all tempest tests for all teams.

test case code, use rescue mode: https://github.com/openstack/ironic-tempest-plugin/blob/cda96d5ca3f84a5b883fee521bbabbe1c283c2a0/ironic_tempest_plugin/tests/scenario/test_baremetal_basic_ops.py#L258

upstream fixes:

https://review.opendev.org/c/openstack/tempest/+/872566

https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/872567

Comment 9 James E. LaBarre 2023-07-18 17:31:42 UTC

SSH connection adapts to preferred configuration, establishes SSH connection as expected
=====================================================

2023-07-13 18:22:44.525 220837 INFO tempest.lib.common.ssh [-] Creating ssh connection to '192.168.24.88:22' as 'cloud-user' with public key authentication

2023-07-13 18:22:51.638 220837 WARNING tempest.lib.common.ssh [-] Failed to establish authenticated ssh connection to cloud-user.24.88 ([Errno None] Unable to connect to port 22 on 192.168.24.88). Number attempts: 1. Retry after 2 seconds.: paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.24.88

2023-07-13 18:22:54.141 220837 WARNING tempest.lib.common.ssh [-] Failed to establish authenticated ssh connection to cloud-user.24.88 ([Errno None] Unable to connect to port 22 on 192.168.24.88). Number attempts: 2. Retry after 3 seconds.: paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.24.88

2023-07-13 18:22:57.656 220837 INFO paramiko.transport [-] Connected (version 2.0, client OpenSSH_8.7)

2023-07-13 18:22:57.748 220837 INFO paramiko.transport [-] Authentication (publickey) successful!

2023-07-13 18:22:57.749 220837 INFO tempest.lib.common.ssh [-] ssh connection to cloud-user.24.88 successfully created

Comment 15 errata-xmlrpc 2023-08-16 01:13:59 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577