Bug 1874741

Summary: non existing (orphan) tap devices still exists on compute nodes after upgrade to OSP13 z11
Product: Red Hat OpenStack Reporter: Shailesh Chhabdiya <schhabdi>
Component: openstack-tripleo-heat-templatesAssignee: Jakub Libosvar <jlibosva>
Status: CLOSED ERRATA QA Contact: Roman Safronov <rsafrono>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: bcafarel, beagles, chrisw, dalvarez, ekuris, jlibosva, jschluet, jthomas, knoha, majopela, mburns, mowens, mvalsecc, nalmond, nyechiel, ragiman, scohen, srevivo, tfreger
Target Milestone: z16Keywords: Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-8.4.1-81.el7ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 1591067 Environment:
Last Closed: 2021-06-16 10:58:53 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1591067    
Bug Blocks:    

Description Shailesh Chhabdiya 2020-09-02 06:58:45 UTC
+++ This bug was initially created as a clone of Bug #1591067 +++

Description of problem:
There are orphan tap devices entries still present in ovs-vsctl show output

Version-Release number of selected component (if applicable):
RHOSP13z11

Actual results:
tap devices still exists on compute nodes after upgrade to OSP13 z11

Expected results:
Tap devices should be clean after deleting the VM. 

Additional Info:

We can see tap devices still present in error state on compute node 
ansible -i inventory  cmp-prod  -m shell -a 'ovs-vsctl show | grep error')  : 

node-x.x.x.x | SUCCESS | rc=0 >>
                error: "could not open network device tap9538318e-dd (No such device)"
                error: "could not open network device tap6840d303-27 (No such device)"
                error: "could not open network device tapb53097fe-50 (No such device)"
                error: "could not open network device tapb6371131-06 (No such device)"
                error: "could not open network device tap2bbfd7bd-2a (No such device)"
                error: "could not open network device tap5fd23df6-57 (No such device)"

node-x.x.x.x | SUCCESS | rc=0 >>
                error: "could not open network device tap1d638559-89 (No such device)"
                error: "could not open network device tapa845e7e0-14 (No such device)"
                error: "could not open network device tap0b82eac0-6f (No such device)"
                error: "could not open network device tap3fa2706e-5f (No such device)"
                error: "could not open network device tap56f08b45-16 (No such device)"
                error: "could not open network device tapc1d9359f-72 (No such device)"
                error: "could not open network device tap848a098c-f6 (No such device)"
                error: "could not open network device tap09d5407b-47 (No such device)"
                error: "could not open network device tap387e7859-20 (No such device)"
                error: "could not open network device tap63f9a7f6-48 (No such device)"
                error: "could not open network device tap22ce12dc-c4 (No such device)"
                error: "could not open network device tap2781969c-fb (No such device)"
                error: "could not open network device tap074a969f-fa (No such device)"

...
...
...



RPMs details are from compute node:
 
openstack-neutron-12.1.1-6.el7ost.noarch                    Sun Jul 12 02:33:22 2020
openstack-neutron-common-12.1.1-6.el7ost.noarch             Sun Jul 12 02:32:28 2020
openstack-neutron-l2gw-agent-12.0.2-0.20190420004620.270972f.el7ost.noarch Sun Jul 12 02:34:15 2020
openstack-neutron-lbaas-12.0.1-0.20190803015156.b86fcef.el7ost.noarch Sun Jul 12 02:33:46 2020
openstack-neutron-lbaas-ui-4.0.1-0.20190723082436.ccf8621.el7ost.noarch Sun Jul 12 02:33:46 2020
openstack-neutron-linuxbridge-12.1.1-6.el7ost.noarch        Sun Jul 12 02:34:14 2020
openstack-neutron-metering-agent-12.1.1-6.el7ost.noarch     Sun Jul 12 02:34:15 2020
openstack-neutron-ml2-12.1.1-6.el7ost.noarch                Sun Jul 12 02:32:30 2020
openstack-neutron-openvswitch-12.1.1-6.el7ost.noarch        Sun Jul 12 02:34:14 2020
openstack-neutron-sriov-nic-agent-12.1.1-6.el7ost.noarch    Sun Jul 12 02:34:15 2020
puppet-neutron-12.4.1-10.el7ost.noarch                      Sun Jul 12 02:33:18 2020
python2-neutronclient-6.7.0-1.el7ost.noarch                 Mon May 20 05:38:59 2019
python2-neutron-lib-1.13.0-2.el7ost.noarch                  Sun Jul 12 02:32:17 2020
python-neutron-12.1.1-6.el7ost.noarch                       Sun Jul 12 02:32:20 2020
python-neutron-lbaas-12.0.1-0.20190803015156.b86fcef.el7ost.noarch Sun Jul 12 02:32:20 2020


The setup is upgrade from RHOSP13z5 to RHOSP13z11

To clean this tap devices manual intervention is carried out
~~~
for i in $(ovs-vsctl --columns name,error list interface | grep "could not" -B2 | grep name| awk '{print $3}' | sed 's/"//g'); do ovs-vsctl del-port $i; done
~~~

Comment 17 Roman Safronov 2021-04-14 15:40:33 UTC
Verified after upgrade to puddle 2021-04-09.1 (openstack-tripleo-heat-templates-8.4.1-83.el7ost.noarch) that neutron-cleanup.service is available on all nodes.
It can be seen in journalctl that neutron-cleanup.service was executed after reboot and deleted tap ports successfully.
Verified that neutron-cleanup.service is executed after normal reboot, power off and kernel panic.

Comment 23 errata-xmlrpc 2021-06-16 10:58:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 13.0 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2385