Bug 2028274

Summary: DOWN subports are not cleaned up
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: Maysa Macedo <mdemaced>
Networking sub component: kuryr QA Contact: Itay Matza <imatza>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: arghosh, imatza, itbrown, rlobillo
Version: 4.9Keywords: Triaged
Target Milestone: ---   
Target Release: 4.8.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-01-05 16:11:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2023731    
Bug Blocks:    

Comment 3 Itay Matza 2021-12-16 14:47:43 UTC
Verified in OCP 4.8.0-0.nightly-2021-12-13-234359 with Kuryr on top of RHOS-16.1-RHEL-8-20210903.n.0.


Verification steps:

- Confirm kuryr pods are up and running:

	$ oc get pods -n openshift-kuryr                                                                                                                                         
	kuryr-cni-2j7fv                    1/1     Running   0          3h32m
	kuryr-cni-9b8mp                    1/1     Running   0          3h33m
	kuryr-cni-9bljk                    1/1     Running   1          3h47m
	kuryr-cni-dhz44                    1/1     Running   0          3h47m
	kuryr-cni-fchbs                    1/1     Running   0          3h34m
	kuryr-cni-k8mzt                    1/1     Running   0          3h47m
	kuryr-controller-95bcf6b69-qsslp   1/1     Running   1          3h47m


- pick one subport:

	$ openstack port list --device-owner "trunk:subport" -f value | tail -1
	ffca7f6e-246e-40ec-97a0-0f6e4d43a55f  fa:16:3e:8e:cb:d4 [{'subnet_id': '5fc930fd-497a-437a-acb3-76ef5562759a', 'ip_address': '10.128.20.220'}] ACTIVE


- Connect to the database and move the port to DOWN status:
	MariaDB [ovs_neutron]> update ports set status='DOWN' where id='ffca7f6e-246e-40ec-97a0-0f6e4d43a55f';
	Query OK, 1 row affected (0.002 sec)
	Rows matched: 1  Changed: 1  Warnings: 0


- Confirm that the port appears as DOWN:

	$ openstack port list --device-owner "trunk:subport" -f value | tail -1
	ffca7f6e-246e-40ec-97a0-0f6e4d43a55f  fa:16:3e:8e:cb:d4 [{'subnet_id': '5fc930fd-497a-437a-acb3-76ef5562759a', 'ip_address': '10.128.20.220'}] DOWN
	
	
- Restart kuryr-controller:

	$ oc delete pod -n openshift-kuryr -l app=kuryr-controller
	pod "kuryr-controller-95bcf6b69-qsslp" deleted
	
	
- Wait until the pod is up again:

	$ oc get pods -n openshift-kuryr
	NAME                                READY   STATUS    RESTARTS   AGE
	kuryr-cni-2j7fv                    1/1     Running   0          3h37m
	kuryr-cni-9b8mp                    1/1     Running   0          3h37m
	kuryr-cni-9bljk                    1/1     Running   1          3h51m
	kuryr-cni-dhz44                    1/1     Running   0          3h51m
	kuryr-cni-fchbs                    1/1     Running   0          3h38m
	kuryr-cni-k8mzt                    1/1     Running   0          3h51m
	kuryr-controller-95bcf6b69-g4kg5   0/1     Running   0          35s


- After ~20 minutes, the port disappeared, and there isn't a port on a DOWN status
	$ date; openstack port list --device-owner "trunk:subport" -f value | grep DOWN
	Thu Dec 16 08:54:07 EST 2021
	ffca7f6e-246e-40ec-97a0-0f6e4d43a55f  fa:16:3e:8e:cb:d4 [{'subnet_id': '5fc930fd-497a-437a-acb3-76ef5562759a', 'ip_address': '10.128.20.220'}] DOWN

	$ openstack port show ffca7f6e-246e-40ec-97a0-0f6e4d43a55f
	+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
	| Field                   | Value                                                                                                                                                                 |
	+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
	| admin_state_up          | UP                                                                                                                                                                    |
	| allowed_address_pairs   |                                                                                                                                                                       |
	| binding_host_id         | None                                                                                                                                                                  |
	| binding_profile         | None                                                                                                                                                                  |
	| binding_vif_details     | None                                                                                                                                                                  |
	| binding_vif_type        | None                                                                                                                                                                  |
	| binding_vnic_type       | normal                                                                                                                                                                |
	| created_at              | 2021-12-16T10:05:10Z                                                                                                                                                  |
	| data_plane_status       | None                                                                                                                                                                  |
	| description             |                                                                                                                                                                       |
	| device_id               |                                                                                                                                                                       |
	| device_owner            | trunk:subport                                                                                                                                                         |
	| dns_assignment          | fqdn='host-10-128-20-220.shiftstack.com.', hostname='host-10-128-20-220', ip_address='10.128.20.220'                                                                  |
	| dns_domain              |                                                                                                                                                                       |
	| dns_name                |                                                                                                                                                                       |
	| extra_dhcp_opts         |                                                                                                                                                                       |
	| fixed_ips               | ip_address='10.128.20.220', subnet_id='5fc930fd-497a-437a-acb3-76ef5562759a'                                                                                          |
	| id                      | ffca7f6e-246e-40ec-97a0-0f6e4d43a55f                                                                                                                                  |
	| location                | cloud='', project.domain_id=, project.domain_name='Default', project.id='41e028d8665f4db3a2174aee71b2d8b3', project.name='shiftstack', region_name='regionOne', zone= |
	| mac_address             | fa:16:3e:8e:cb:d4                                                                                                                                                     |
	| name                    |                                                                                                                                                                       |
	| network_id              | c8bdcf74-3125-468c-b8cc-9b5eeee4ede5                                                                                                                                  |
	| port_security_enabled   | True                                                                                                                                                                  |
	| project_id              | 41e028d8665f4db3a2174aee71b2d8b3                                                                                                                                      |
	| propagate_uplink_status | None                                                                                                                                                                  |
	| qos_policy_id           | None                                                                                                                                                                  |
	| resource_request        | None                                                                                                                                                                  |
	| revision_number         | 5                                                                                                                                                                     |
	| security_group_ids      | 95e8f72b-da33-4112-a652-f28dad3adbfe                                                                                                                                  |
	| status                  | DOWN                                                                                                                                                                  |
	| tags                    | openshiftClusterID=ostest-6r2vj                                                                                                                                       |
	| trunk_details           | None                                                                                                                                                                  |
	| updated_at              | 2021-12-16T10:05:15Z                                                                                                                                                  |
	+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+


	$ date; openstack port list --device-owner "trunk:subport" -f value | grep DOWN
	Thu Dec 16 09:14:17 EST 2021

	$ openstack port show ffca7f6e-246e-40ec-97a0-0f6e4d43a55f
	No Port found for ffca7f6e-246e-40ec-97a0-0f6e4d43a55f

Comment 6 errata-xmlrpc 2022-01-05 16:11:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.25 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:5209

Comment 7 Maysa Macedo 2022-05-30 18:46:14 UTC
*** Bug 2076169 has been marked as a duplicate of this bug. ***