Bug 1689671 - Undercloud: neutron containers healthcheck failed
Summary: Undercloud: neutron containers healthcheck failed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 15.0 (Stein)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: beta
: 15.0 (Stein)
Assignee: Cédric Jeanneret
QA Contact: Alexander Chuzhoy
URL:
Whiteboard:
: 1693203 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-17 13:38 UTC by Sasha Smolyak
Modified: 2019-09-26 10:48 UTC (History)
9 users (show)

Fixed In Version: openstack-tripleo-common-10.6.1-0.20190328200349.45cd562.el8ost.noarch.rpm
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-21 11:20:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 648027 0 'None' MERGED Silent file descriptor checks 2020-12-16 01:54:09 UTC
Red Hat Product Errata RHEA-2019:2811 0 None None None 2019-09-21 11:21:01 UTC

Internal Links: 1693196

Description Sasha Smolyak 2019-03-17 13:38:40 UTC
Description of problem:
When running neutron containers healthcheck on undercloud, it fails:

[stack@undercloud-0 ~]$ systemctl status tripleo_neutron_ovs_agent.service
● tripleo_neutron_ovs_agent.service - neutron_ovs_agent container
   Loaded: loaded (/etc/systemd/system/tripleo_neutron_ovs_agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-03-17 08:59:13 UTC; 4h 29min ago
 Main PID: 51529 (podman)
    Tasks: 17 (limit: 26213)
   Memory: 28.3M
   CGroup: /system.slice/tripleo_neutron_ovs_agent.service
           └─51529 /usr/bin/podman start -a neutron_ovs_agent
[stack@undercloud-0 ~]$ systemctl status tripleo_neutron_ovs_agent_healthcheck.service
● tripleo_neutron_ovs_agent_healthcheck.service - neutron_ovs_agent healthcheck
   Loaded: loaded (/etc/systemd/system/tripleo_neutron_ovs_agent_healthcheck.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sun 2019-03-17 13:28:41 UTC; 18s ago
  Process: 358657 ExecStart=/usr/bin/podman exec neutron_ovs_agent /openstack/healthcheck (code=exited, status=1/FAILURE)
 Main PID: 358657 (code=exited, status=1/FAILURE)

True for following containers:
neutron_ovs_agent
neutron_l3_agent
neutron_dhcp

Same time neutron_api healthcheck status is SUCCESS

Version-Release number of selected component (if applicable):
puppet-neutron-14.3.0-0.20190311073237.5fdbb26.el8ost.noarch
python3-neutron-lbaas-13.1.0-0.20190301124315.2552fb1.el8ost.noarch
openstack-neutron-ml2-14.0.0-0.20190308181536.e636b27.el8ost.noarch
python3-neutron-lib-1.25.0-0.20190309062008.fc2a810.el8ost.noarch
openstack-neutron-common-14.0.0-0.20190308181536.e636b27.el8ost.noarch
openstack-neutron-lbaas-13.1.0-0.20190301124315.2552fb1.el8ost.noarch
python3-neutronclient-6.12.0-0.20190311151844.680b417.el8ost.noarch
python3-neutron-14.0.0-0.20190308181536.e636b27.el8ost.noarch
openstack-neutron-14.0.0-0.20190308181536.e636b27.el8ost.noarch


How reproducible:


Steps to Reproduce:
1. Deploy undercloud
2. Run healthcheck for neutron:
systemctl status tripleo_neutron_ovs_agent_healthcheck.service

Actual results:
Failure

Expected results:
Success

Additional info:

Comment 1 Brent Eagles 2019-03-19 13:43:37 UTC
The healthcheck_port test is failing because the services are showing up in the 'ss' output as "/usr/bin/python" instead of the service name.

Comment 2 Cédric Jeanneret 2019-03-27 10:40:42 UTC
Hello!

I think this patch should do: https://review.openstack.org/648027

Using "pgrep -f" gives the PIDs matching the process "in the line", and then we can match each PID in "ss" directly. Might looks ugly, but I couldn't find any better way. Apparently something changed "somewhere" and modified ss output, preventing a correct matching of the processes.

In addition, neutron probably isn't the only one affected with that.

Cheers,

C.

Comment 3 Cédric Jeanneret 2019-03-27 12:29:14 UTC
*** Bug 1693203 has been marked as a duplicate of this bug. ***

Comment 10 errata-xmlrpc 2019-09-21 11:20:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811


Note You need to log in before you can comment on or make changes to this bug.