Bug 1394744
| Summary: | TestSecurityGroupsBasicOps.test_port_security_disable_security_group test gets halted by giving request timed out. | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Chandan Kumar <chkumar> |
| Component: | openstack-ironic-inspector | Assignee: | Dmitry Tantsur <dtantsur> |
| Status: | CLOSED ERRATA | QA Contact: | Raviv Bar-Tal <rbartal> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 10.0 (Newton) | CC: | afazekas, apevec, dmellado, dtantsur, jschluet, lhh, mburns, mmagr, slinaber |
| Target Milestone: | rc | Keywords: | AutomationBlocker, Triaged |
| Target Release: | 10.0 (Newton) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-ironic-inspector-4.2.0-3.el7ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-12-14 16:32:20 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This issue prevents us to run the shipped tempest on the undercloud node, as we normally do. > It appears that there is some issue with the ironic-inspector tempest plugin.
I don't see any signs of it, sorry.
This is a known issue with SSH code in tempest, mtreinish has tried to fix it previously, but his attempt broke other people. I'm not entirely sure why it exhibit itself when inspector plugin is enabled, but I don't see how we can help.
For the record: we had similar problems in upstream CI since https://review.openstack.org/#/c/358610/ was merged. Try reverting it to see if it helps. Hey dmitry. What makes me wonder is *why* this shows only up when ironic-inspector plugin is enabled. Is there any code within the plugin that would touch any ssh related setting? We don't seem to. Actually our plugin [1] is pretty simple, I've no idea in what way it can cause configuration changes. I also don't think there should be any configuration in Tempest breaking SSH. But FWIW tempest is black magic for me and most of upstream contributors, so we *might* be missing something. [1] https://github.com/openstack/ironic-inspector/tree/master/ironic_inspector/test/inspector_tempest_plugin Ironic-Inspector monkey-patching eventlet on the path to the tempest_plugin is the source of this issue. https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/test/__init__.py Tempest doesn't support the usage of eventlet and if the eventlet monkey patch happens late (plugin load time) it causes this kind of issues. Removing the eventlet import affecting the ironic-inspector tempest plugin solves the issue affecting another tests. Reassigning the bug to ironic-inspector in order to track the packaging. openstack-ironic-inspector-4.2.0-3.el7ost has the remove eventlet monkeypatch from test openstack-tempest-13.0.0-5.bafe630git.el7ost has the "Delete duplicated dvr tests " patch in it Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html |
Description of problem: TestSecurityGroupsBasicOps.test_port_security_disable_security_group tempest test gets hautled on undercloud by giving a Request timed out following by Details: Command: 'set -eu -o pipefail; PATH=$PATH:/sbin; ping -c1 -w1 -s56 10.100.0.9' executed on host '10.0.0.216'. Version-Release number of selected component (if applicable): Ironic-inspector: openstack-ironic-inspector-4.2.0-2.el7ost.noarch python-ironic-inspector-client-1.9.0-1.el7ost.noarch Tempest: puppet-tempest-9.4.0-1.el7ost.noarch openstack-tempest-13.0.0-4.bafe630git.el7ost.noarch python-tempest-13.0.0-4.bafe630git.el7ost.noarch Steps to Reproduce: [1.] Deploy OpenStack using RHOP-10 director on RHEL-7.3 [2.] Install openstack-tempest-13.0.0-4.bafe630git.el7ost.noarch on undercloud [3.] configure tempest then try to run tempest list-plugins, make sure ironic-inspector is visible [4.] run ostestr -r 'TestSecurityGroupsBasicOps.test_port_security_disable_security_group' you will get the following traceback: [root@undercloud-0 ~]# ( cd tempest-dir ; ostestr -r 'TestSecurityGroupsBasicOps.test_port_security_disable_security_group' ) running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \ OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \ OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \ OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \ ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \ OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \ OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \ OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \ ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --load-list /tmp/tmp7ShEYC {0} tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group [335.122332s] ... FAILED ============================== Failed 1 tests - output below: ============================== tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group[compute,id-7c811dcc-263b-49a3-92d2-1b4d8405f50c,network] ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Captured traceback: ~~~~~~~~~~~~~~~~~~~ Traceback (most recent call last): File "/root/tempest-dir/tempest/test.py", line 152, in wrapper return func(*func_args, **func_kwargs) File "/root/tempest-dir/tempest/test.py", line 107, in wrapper return f(self, *func_args, **func_kwargs) File "/root/tempest-dir/tempest/scenario/test_security_groups_basic_ops.py", line 614, in test_port_security_disable_security_group should_succeed=False) File "/root/tempest-dir/tempest/scenario/test_security_groups_basic_ops.py", line 377, in _check_connectivity should_succeed), msg) File "/root/tempest-dir/tempest/scenario/manager.py", line 926, in _check_remote_connectivity 1) File "/root/tempest-dir/tempest/lib/common/utils/test_utils.py", line 103, in call_until_true if func(): File "/root/tempest-dir/tempest/scenario/manager.py", line 917, in ping_remote source.ping_host(dest, nic=nic) File "/root/tempest-dir/tempest/common/utils/linux/remote_client.py", line 141, in ping_host return self.exec_command(cmd) File "/root/tempest-dir/tempest/common/utils/linux/remote_client.py", line 36, in wrapper return function(self, *args, **kwargs) File "/root/tempest-dir/tempest/common/utils/linux/remote_client.py", line 92, in exec_command return self.ssh_client.exec_command(cmd) File "/root/tempest-dir/tempest/lib/common/ssh.py", line 140, in exec_command cmd, self.host)) tempest.lib.exceptions.TimeoutException: Request timed out Details: Command: 'set -eu -o pipefail; PATH=$PATH:/sbin; ping -c1 -w1 -s56 10.100.0.9' executed on host '10.0.0.216'. [5.] On disabling ironic-inspector tempest plugin from /usr/lib/python2.7/site-packages/ironic_inspector-4.2.0-py2.7.egg-info/entry_points.txt by commenting #ironic_inspector_tests = ironic_inspector.test.inspector_tempest_plugin.plugin:InspectorTempestPlugin, this problem just vanished. It appears that there is some issue with the ironic-inspector tempest plugin.