Bug 1894194 - KuryrPorts leftovers from 4.6 GA need to be deleted
Summary: KuryrPorts leftovers from 4.6 GA need to be deleted
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.7
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.7.0
Assignee: Michał Dulko
QA Contact: Itzik Brown
URL:
Whiteboard:
Depends On:
Blocks: 1894408
TreeView+ depends on / blocked
 
Reported: 2020-11-03 17:48 UTC by Michał Dulko
Modified: 2021-03-24 09:50 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Due to https://bugzilla.redhat.com/show_bug.cgi?id=1886871 unnecessary Neutron ports were created for hostNetworking pods. Consequence: Deployments installed with that bug existing would keep a number of unnecessary Neutron ports. Fix: On startup Kuryr makes sure to delete any unnecessary ports created for hostNetworking pods. Result: After upgrading to a version with this bug fix the ports should get removed.
Clone Of:
Environment:
Last Closed: 2021-02-24 15:30:39 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift kuryr-kubernetes pull 393 0 None closed Bug 1894194: Delete ports created for host networking pods 2021-01-08 09:01:36 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:30:44 UTC

Description Michał Dulko 2020-11-03 17:48:12 UTC
Description of problem:
Anybody installing 4.6 GA with Kuryr will get KuryrPorts and Neutron ports created for host-networking pods due to bug https://bugzilla.redhat.com/show_bug.cgi?id=1886871, which is totally unnecessary and only uses the quota. This bug is about making sure an upgrade to 4.7 or 4.6.z deletes the KuryrPorts, at least allowing them to be used for other pods in the namespace.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Install 4.6 GA.
2. Upgrade kuryr pods to 4.7 or 4.6.z with the fix, wait a bit for Kuryr to finish deleting.
3. Check number of KuryrPorts created for host-networking pods using method from https://bugzilla.redhat.com/show_bug.cgi?id=1886871.

Actual results:
There will be KuryrPorts (probably 64).

Expected results:
The KuryrPorts should be gone.

Additional info:
This will not delete Neutron ports, we'll address that in a future change that'll configure port pools to actually delete Neutron ports.

Comment 3 rlobillo 2020-11-11 16:32:08 UTC
Verified on OCP4.7.0-0.nightly-2020-11-11-055454 over OSP16.1 with OVN-Octavia (RHOS-16.1-RHEL-8-20201021.n.0)

Upgrade from OCP4.6.0 experiencing the issue to the latest 4.7 nightly results on the deletion of the unnecessary kuryr ports.

# Issue replicated on OCP4.6.0:

	(overcloud) [stack@undercloud-0 ~]$ oc get clusterversion
	NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
	version   4.6.0     True        False         18h     Cluster version is 4.6.0
	(overcloud) [stack@undercloud-0 ~]$ cat core_puddle_version 
	RHOS-16.1-RHEL-8-20201021.n.0
	(overcloud) [stack@undercloud-0 ~]$ oc get nodes
	NAME                          STATUS   ROLES    AGE   VERSION
	ostest-dhsb6-master-0         Ready    master   19h   v1.19.0+d59ce34
	ostest-dhsb6-master-1         Ready    master   19h   v1.19.0+d59ce34
	ostest-dhsb6-master-2         Ready    master   19h   v1.19.0+d59ce34
	ostest-dhsb6-worker-0-h9hpn   Ready    worker   18h   v1.19.0+d59ce34
	ostest-dhsb6-worker-0-q8qmf   Ready    worker   19h   v1.19.0+d59ce34
	ostest-dhsb6-worker-0-v6f9f   Ready    worker   18h   v1.19.0+d59ce34
	(overcloud) [stack@undercloud-0 ~]$ oc get pods -n openshift-kuryr
	NAME                               READY   STATUS    RESTARTS   AGE
	kuryr-cni-bmrrh                    1/1     Running   0          19h
	kuryr-cni-fqz44                    1/1     Running   1          19h
	kuryr-cni-k4v64                    1/1     Running   2          18h
	kuryr-cni-l77np                    1/1     Running   0          19h
	kuryr-cni-n7mmr                    1/1     Running   0          18h
	kuryr-cni-vj75s                    1/1     Running   0          19h
	kuryr-controller-77bc4f8cf-mcgn9   1/1     Running   1          19h

	(overcloud) [stack@undercloud-0 ~]$  oc get pods -A -o wide | grep 10.196 | awk '{print $2}' > host-networking-pods
	(overcloud) [stack@undercloud-0 ~]$ oc get kp -A | grep -f host-networking-pods | wc -l
	64


# Performing the upgrade to latest 4.7 nightly:


	(overcloud) [stack@undercloud-0 ~]$ oc adm upgrade --to-image="registry.svc.ci.openshift.org/ocp/release:4.7.0-0.nightly-2020-11-11-055454" --allow-explicit-upgrade --force=true
	warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead
	warning: The requested upgrade image is not one of the available updates.  You have used --allow-explicit-upgrade to the update to proceed anyway
	warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures.
	Updating to release image registry.svc.ci.openshift.org/ocp/release:4.7.0-0.nightly-2020-11-11-055454
	(overcloud) [stack@undercloud-0 ~]$ oc get clusterversion
	NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
	version   4.6.0     True        True          109s    Working towards 4.7.0-0.nightly-2020-11-11-055454: 11% complete

# Once openshift-kuryr pods are upgraded, unnecesary kuryr-ports are removed:

	(shiftstack) [stack@undercloud-0 ~]$ oc get clusterversion
	NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
	version   4.6.0     True        True          58m     Working towards 4.7.0-0.nightly-2020-11-11-055454: 84% complete
	(shiftstack) [stack@undercloud-0 ~]$  oc get pods -A -o wide | grep 10.196 | awk '{print $2}' > host-networking-pods.after_upgrade
	(shiftstack) [stack@undercloud-0 ~]$ wc -l host-networking-pods host-networking-pods.after_upgrade 
	  81 host-networking-pods
	  81 host-networking-pods.after_upgrade
	 162 total
	(shiftstack) [stack@undercloud-0 ~]$ oc get kp -A | grep -f host-networking-pods | wc -l
	0
	(shiftstack) [stack@undercloud-0 ~]$ oc get kp -A | grep -f host-networking-pods.after_upgrade | wc -l
	0
	(shiftstack) [stack@undercloud-0 ~]$ oc get pods -n openshift-kuryr
	NAME                                READY   STATUS    RESTARTS   AGE
	kuryr-cni-28vzw                     1/1     Running   0          22m
	kuryr-cni-6dklb                     1/1     Running   0          24m
	kuryr-cni-6qnxf                     1/1     Running   0          23m
	kuryr-cni-7wddz                     1/1     Running   0          26m
	kuryr-cni-hdmxc                     1/1     Running   0          19m
	kuryr-cni-rcvbn                     1/1     Running   0          21m
	kuryr-controller-79f8cc85f8-w7qlx   1/1     Running   1          28m
	(shiftstack) [stack@undercloud-0 ~]$

Comment 6 errata-xmlrpc 2021-02-24 15:30:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.