Bug 2074471
| Summary: | Cannot delete namespace with a LB type svc and Kuryr when ExternalCloudProvider is enabled | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Jon Uriarte <juriarte> |
| Component: | Networking | Assignee: | Maysa Macedo <mdemaced> |
| Networking sub component: | kuryr | QA Contact: | Jon Uriarte <juriarte> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | high | ||
| Priority: | urgent | CC: | emacchi, mdulko, rlobillo |
| Version: | 4.11 | Keywords: | Triaged |
| Target Milestone: | --- | ||
| Target Release: | 4.11.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-08-10 11:06:14 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Verified in 4.11.0-0.nightly-2022-05-05-015322 on top of OSP 16.1.7.
Steps:
1. Install 4.11 with ExternalCloudProvider
$ openshift-install create manifests --log-level=debug --dir=/home/stack/ostest/
$ cd ostest/
$ cat <<EOF >manifests/manifest_feature_gate.yaml
apiVersion: config.openshift.io/v1
kind: FeatureGate
metadata:
annotations:
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
release.openshift.io/create-only: "true"
name: cluster
spec:
customNoUpgrade:
enabled:
- ExternalCloudProvider
featureSet: CustomNoUpgrade
EOF
$ openshift-install create cluster --log-level=debug --dir=/home/stack/ostest/
2. Create the loadbalancer type svc with below manifest:
cat <<EOF | oc apply -f -
---
apiVersion: project.openshift.io/v1
kind: Project
metadata:
name: lb-test-ns
labels:
kubernetes.io/metadata.name: lb-test-ns
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: lb-test-dep
namespace: lb-test-ns
labels:
app: lb-test-dep
spec:
replicas: 2
selector:
matchLabels:
app: lb-test-dep
template:
metadata:
labels:
app: lb-test-dep
spec:
containers:
- image: quay.io/kuryr/demo
name: demo
---
apiVersion: v1
kind: Service
metadata:
name: lb-test-svc
namespace: lb-test-ns
labels:
app: lb-test-dep
spec:
ports:
- port: 80
targetPort: 8080
selector:
app: lb-test-dep
type: LoadBalancer
EOF
3. Check LB, pod and svc creation
$ oc describe svc -n lb-test-ns
Name: lb-test-svc
Namespace: lb-test-ns
Labels: app=lb-test-dep
Annotations: <none>
Selector: app=lb-test-dep
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 172.30.227.8
IPs: 172.30.227.8
LoadBalancer Ingress: 10.46.22.247
Port: <unset> 80/TCP
TargetPort: 8080/TCP
NodePort: <unset> 30428/TCP
Endpoints: 10.128.83.107:8080,10.128.83.44:8080
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal KuryrServiceSkipped 2m55s kuryr-controller Skipping Service lb-test-ns/lb-test-svc without Endpoints
Normal KuryrEnsureLB 2m29s kuryr-controller Provisioning a load balancer
Normal KuryrEnsuredLB 2m21s kuryr-controller Load balancer provisioned
Normal KuryrEnsureFIP 2m19s kuryr-controller Associating floating IP to the load balancer
Normal KuryrEnsuredLB 2m5s kuryr-controller Load balancer provisioned
Normal EnsuringLoadBalancer 2m57s service-controller Ensuring load balancer
4. Check service connectivity
$ curl 10.46.22.247
lb-test-dep-68d6754b4d-6jr97: HELLO! I AM ALIVE!!!
$ curl 10.46.22.247
lb-test-dep-68d6754b4d-w64rk: HELLO! I AM ALIVE!!!
5. Delete the svc and check the svc and LB are deleted
$ oc -n lb-test-ns delete svc lb-test-svc
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069 |
Description of problem: Namespace deletion remains in terminating status when it holds a LoadBalancer type svc, which is not deleted either. $ oc get project NAME DISPLAY NAME STATUS [...] lb-test-ns Terminating $ oc get svc -n lb-test-ns lb-test-svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE lb-test-svc LoadBalancer 172.30.238.135 10.46.22.247 80:32469/TCP 24m Version-Release number of selected component (if applicable): OCP 4.11.0-0.nightly-2022-04-08-205307 OSP 16.1.7 How reproducible: always Steps to Reproduce: 1. Install 4.11 with ExternalCloudProvider $ openshift-install create manifests --log-level=debug --dir=/home/stack/ostest/ $ cd ostest/ $ cat <<EOF >manifests/manifest_feature_gate.yaml apiVersion: config.openshift.io/v1 kind: FeatureGate metadata: annotations: include.release.openshift.io/self-managed-high-availability: "true" include.release.openshift.io/single-node-developer: "true" release.openshift.io/create-only: "true" name: cluster spec: customNoUpgrade: enabled: - ExternalCloudProvider featureSet: CustomNoUpgrade EOF $ openshift-install create cluster --log-level=debug --dir=/home/stack/ostest/ 2. Create a ns, deployment and LB type svc $ cat lb_test_resources.yaml --- apiVersion: project.openshift.io/v1 kind: Project metadata: name: lb-test-ns labels: kubernetes.io/metadata.name: lb-test-ns --- apiVersion: apps/v1 kind: Deployment metadata: name: lb-test-dep namespace: lb-test-ns labels: app: lb-test-dep spec: replicas: 2 selector: matchLabels: app: lb-test-dep template: metadata: labels: app: lb-test-dep spec: containers: - image: quay.io/kuryr/demo name: demo --- apiVersion: v1 kind: Service metadata: name: lb-test-svc namespace: lb-test-ns labels: app: lb-test-dep spec: ports: - port: 80 targetPort: 8080 selector: app: lb-test-dep type: LoadBalancer $ oc apply -f lb_test_resources.yaml 3. Wait until the svc is assigned a fip: $ oc get svc -n lb-test-ns lb-test-svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE lb-test-svc LoadBalancer 172.30.238.135 10.46.22.247 80:32469/TCP 3 4. Check the connectivity to the svc: $ curl 10.46.22.247 lb-test-dep-68d6754b4d-b9lmp: HELLO! I AM ALIVE!!! $ curl 10.46.22.247 lb-test-dep-68d6754b4d-lxpdg: HELLO! I AM ALIVE!!! 5. Delete the ns: $ oc delete project lb-test-ns project.project.openshift.io "lb-test-ns" deleted 6. Check the ns is deleted Actual results: the namespace (and the LB type svc) are not deleted $ oc get project NAME DISPLAY NAME STATUS [...] lb-test-ns Terminating Expected results: Namespace (and LB type svc) deleted Additional info: $ oc get svc -n lb-test-ns lb-test-svc -o yaml apiVersion: v1 kind: Service metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"lb-test-dep"},"name":"lb-test-svc","namespace":"lb-test-ns"},"spec":{"ports":[{"port":80,"targetPort":8080}],"selector":{"app":"lb-test-dep"},"type":"LoadBalancer"}} creationTimestamp: "2022-04-12T09:00:41Z" deletionGracePeriodSeconds: 0 deletionTimestamp: "2022-04-12T09:02:36Z" finalizers: - service.kubernetes.io/load-balancer-cleanup labels: app: lb-test-dep name: lb-test-svc namespace: lb-test-ns resourceVersion: "64754" uid: 01559d25-9e4a-43c8-9e81-9343f7eaa500 spec: allocateLoadBalancerNodePorts: true clusterIP: 172.30.238.135 clusterIPs: - 172.30.238.135 externalTrafficPolicy: Cluster internalTrafficPolicy: Cluster ipFamilies: - IPv4 ipFamilyPolicy: SingleStack ports: - nodePort: 32469 port: 80 protocol: TCP targetPort: 8080 selector: app: lb-test-dep sessionAffinity: None type: LoadBalancer status: loadBalancer: ingress: - ip: 10.46.22.247 $ oc describe svc -n lb-test-ns lb-test-svc Name: lb-test-svc Namespace: lb-test-ns Labels: app=lb-test-dep Annotations: <none> Selector: app=lb-test-dep Type: LoadBalancer IP Family Policy: SingleStack IP Families: IPv4 IP: 172.30.238.135 IPs: 172.30.238.135 LoadBalancer Ingress: 10.46.22.247 Port: <unset> 80/TCP TargetPort: 8080/TCP NodePort: <unset> 32469/TCP Endpoints: <none> Session Affinity: None External Traffic Policy: Cluster Events: <none>