Bug 2039377 - [osp][octavia lb] cannot create LoadBalancer type svcs
Summary: [osp][octavia lb] cannot create LoadBalancer type svcs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Compute
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.z
Assignee: Martin André
QA Contact: Jon Uriarte
URL:
Whiteboard:
Depends On: 2039373
Blocks: 2054669
TreeView+ depends on / blocked
 
Reported: 2022-01-11 15:54 UTC by Martin André
Modified: 2022-02-16 06:52 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: There's a race condition between OpenStack credentials secret creation and kube-controller-manager startup. Consequence: If it happens OpenStack cloud-provider will not get configured with OpenStack credentials, effectively breaking support for creating Octavia load balancers for LoadBalancer services. Fix: Retry fetching the OpenStack credentials secret until it succeeds during the kube-controller-manager started. Result: The OpenStack cloud-provider is now consistently initialized upon kube-controller-manager startup.
Clone Of: 2039373
Environment:
Last Closed: 2022-02-16 06:51:42 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift kubernetes pull 1108 0 None open Bug 2039377: UPSTREAM: 89885: SQUASH: Retry fetching clouds.conf 2022-02-04 12:34:23 UTC
Red Hat Product Errata RHBA-2022:0484 0 None None None 2022-02-16 06:52:03 UTC

Comment 1 ShiftStack Bugwatcher 2022-01-12 07:03:52 UTC
Removing the Triaged keyword because:
* the QE automation assessment (flag qe_test_coverage) is missing

Comment 6 Jon Uriarte 2022-02-08 10:07:57 UTC
Verified on 4.8.0-0.nightly-2022-02-07-035953 on top of OSP 16.1.7.

Tried out 4 times a fresh OCP installation and LB creation worked after it all the times.


Procedure:
1. Install OCP

2. Add Octavia LB support
$ oc get cm cloud-provider-config -n openshift-config  -o yaml
...
    [LoadBalancer]
    use-octavia = True

3. Create a project, deployment and a LB type svc
cat <<EOF | oc apply -f -
---
apiVersion: project.openshift.io/v1
kind: Project
metadata:
  name: lb-test-ns
  labels:
    kubernetes.io/metadata.name: lb-test-ns
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: lb-test-dep
  namespace: lb-test-ns
  labels:
    app: lb-test-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: lb-test-dep
  template:
    metadata:
      labels:
        app: lb-test-dep
    spec:
      containers:
      - image: quay.io/kuryr/demo
        name: demo
---
apiVersion: v1
kind: Service
metadata:
  name: lb-test-svc
  namespace: lb-test-ns
  labels:
    app: lb-test-dep
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: lb-test-dep
  type: LoadBalancer
EOF

$ oc -n lb-test-ns get pods
NAME                          READY   STATUS    RESTARTS   AGE
lb-test-dep-8476c494b-9g86v   1/1     Running   0          14m
lb-test-dep-8476c494b-hx2xn   1/1     Running   0          14m

4. Check the LB creation in OSP
$ openstack loadbalancer list
+--------------------------------------+----------------------------------+----------------------------------+-------------+---------------------+----------+
| id                                   | name                             | project_id                       | vip_address | provisioning_status | provider |
+--------------------------------------+----------------------------------+----------------------------------+-------------+---------------------+----------+
| 8872ee9c-f91f-4b3a-a635-6d646b85cb7d | a4ea2c607c5dd4df09a4b703c7e3b2ff | a0062929e9ae42b1b75c6f1b2fbdbd2f | 10.196.2.86 | ACTIVE              | amphora  |
+--------------------------------------+----------------------------------+----------------------------------+-------------+---------------------+----------+

5. Check the svc has been assigned the FIP
$ oc -n lb-test-ns get svc
NAME          TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
lb-test-svc   LoadBalancer   172.30.20.94    10.46.22.245   80:31487/TCP   15m

6. Test the svc
$ curl 10.46.22.245
lb-test-dep-8476c494b-hx2xn: HELLO! I AM ALIVE!!!

$ curl 10.46.22.245
lb-test-dep-8476c494b-9g86v: HELLO! I AM ALIVE!!!

Comment 9 errata-xmlrpc 2022-02-16 06:51:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.31 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0484


Note You need to log in before you can comment on or make changes to this bug.