Bug 1867520

Summary: Unable to upgrade from Load Balancer annotations to Load Balancer CRD
Product: OpenShift Container Platform Reporter: Maysa Macedo <mdemaced>
Component: NetworkingAssignee: Maysa Macedo <mdemaced>
Networking sub component: kuryr QA Contact: GenadiC <gcheresh>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: rlobillo
Version: 4.6   
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:26:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Maysa Macedo 2020-08-10 09:29:51 UTC
Description of problem:

With the support of Load Balancer CRD added on ocp 4.6 to allow simpler management of OpenStack resources created for a Service, we must ensure the upgrade path from lb annotations on Service and Enpoints to lb CRD is working as expected.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 3 rlobillo 2020-09-07 10:29:09 UTC
Verfied on 4.6.0-0.nightly-2020-09-05-015624 over RHOS-16.1-RHEL-8-20200821.n.0.


1. Install OCP4.5.8:

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.8     True        False         69s     Cluster version is 4.5.8

2. Create project and pods with service:

$ oc new-project test
Now using project "test" on server "https://api.ostest.shiftstack.com:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app rails-postgresql-example

to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:

    kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4

$ oc --image=kuryr/demo run demo
pod/demo created
$ oc --image=kuryr/demo run demo-caller
pod/demo-caller created
$ oc expose pod/demo --port 80 --target-port 8080
service/demo exposed
$ oc get all
NAME              READY   STATUS    RESTARTS   AGE
pod/demo          1/1     Running   0          28s
pod/demo-caller   1/1     Running   0          17s

NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/demo   ClusterIP   172.30.161.244   <none>        80/TCP    10s


$ oc rsh pod/demo curl 172.30.161.244
demo: HELLO! I AM ALIVE!!!



$ . overcloudrc && openstack loadbalancer show test/demo
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| admin_state_up      | True                                 |
| created_at          | 2020-09-07T08:39:18                  |
| description         |                                      |
| flavor_id           | None                                 |
| id                  | e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1 |
| listeners           | f53bcbb2-f866-4559-868a-2fc43a2ef289 |
| name                | test/demo                            |
| operating_status    | ONLINE                               |
| pools               | 96a06be9-4fe1-4c51-92ae-f17460b28d95 |
| project_id          | 92c48c7c788145db89c97f2554aefa4f     |
| provider            | ovn                                  |
| provisioning_status | ACTIVE                               |
| updated_at          | 2020-09-07T08:39:39                  |
| vip_address         | 172.30.161.244                       |
| vip_network_id      | 751c94d9-ff7c-43a0-9672-e92f89c363b6 |
| vip_port_id         | 97f99ce6-fe86-4cf0-afe3-d640ac3b0561 |
| vip_qos_policy_id   | None                                 |
| vip_subnet_id       | d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9 |
+---------------------+--------------------------------------+




$ oc get svc -o json | jq ".items[0].metadata.annotations"
{
  "openstack.org/kuryr-lbaas-spec": "{\"versioned_object.data\": {\"ip\": \"172.30.161.244\", \"lb_ip\": null, \"ports\": [{\"versioned_object.data\": {\"name\": null, \"port\": 80, \"protocol\": \"TCP\", \"targetPort\": \"8080\"}, \"versioned_object.name\": \"LBaaSPortSpec\", \"versioned_object.namespace\": \"kuryr_kubernetes\", \"versioned_object.version\": \"1.1\"}], \"project_id\": \"92c48c7c788145db89c97f2554aefa4f\", \"security_groups_ids\": [\"d61b6bbb-7e01-4c3e-ac24-1a588b289524\"], \"subnet_id\": \"d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9\", \"type\": \"ClusterIP\"}, \"versioned_object.name\": \"LBaaSServiceSpec\", \"versioned_object.namespace\": \"kuryr_kubernetes\", \"versioned_object.version\": \"1.0\"}"
}



$ oc get pods -n openshift-kuryr -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP             NODE                        NOMINATED NODE   READINESS GATES
kuryr-cni-6sfjr                     1/1     Running   0          39m   10.196.3.94    ostest-xcbsx-master-0       <none>           <none>
kuryr-cni-g5jk6                     1/1     Running   0          39m   10.196.0.33    ostest-xcbsx-master-2       <none>           <none>
kuryr-cni-kp8cr                     1/1     Running   1          19m   10.196.1.58    ostest-xcbsx-worker-cb5kt   <none>           <none>
kuryr-cni-qgc7s                     1/1     Running   1          19m   10.196.2.90    ostest-xcbsx-worker-8sc92   <none>           <none>
kuryr-cni-tnqbg                     1/1     Running   0          19m   10.196.2.16    ostest-xcbsx-worker-ctm65   <none>           <none>
kuryr-cni-tsj6x                     1/1     Running   0          39m   10.196.3.204   ostest-xcbsx-master-1       <none>           <none>
kuryr-controller-6f999575cd-qsxdz   1/1     Running   2          39m   10.196.3.94    ostest-xcbsx-master-0       <none>           <none>



3. Perform upgrade to 4.6.0-0.nightly-2020-09-05-015624:

(overcloud) [stack@undercloud-0 ~]$ oc adm upgrade --to-image="registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-05-015624"  --allow-explicit-upgrade --force=true
warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead
warning: The requested upgrade image is not one of the available updates.  You have used --allow-explicit-upgrade to the update to preceed anyway
warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures.
Updating to release image registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-05-015624

(overcloud) [stack@undercloud-0 ~]$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.8     True        True          9s      Working towards registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-05-015624: downloading update

(overcloud) [stack@undercloud-0 ~]$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.8     True        True          97s     Working towards 4.6.0-0.nightly-2020-09-05-015624: 11% complete

(overcloud) [stack@undercloud-0 ~]$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.8     True        True          50m     Working towards 4.6.0-0.nightly-2020-09-05-015624: 79% complete

(overcloud) [stack@undercloud-0 ~]$ oc get svc -o json | jq ".items[0].metadata.annotations"
null
(overcloud) [stack@undercloud-0 ~]$ oc get klb -o json
{
    "apiVersion": "v1",
    "items": [
        {
            "apiVersion": "openstack.org/v1",
            "kind": "KuryrLoadBalancer",
            "metadata": {
                "creationTimestamp": "2020-09-07T09:24:53Z",
                "finalizers": [
                    "kuryr.openstack.org/kuryrloadbalancer-finalizers"
                ],
                "generation": 3,
                "managedFields": [
                    {
                        "apiVersion": "openstack.org/v1",
                        "fieldsType": "FieldsV1",
                        "fieldsV1": {
                            "f:metadata": {
                                "f:finalizers": {
                                    ".": {},
                                    "v:\"kuryr.openstack.org/kuryrloadbalancer-finalizers\"": {}
                                }
                            },
                            "f:spec": {
                                ".": {},
                                "f:endpointSlices": {},
                                "f:ip": {},
                                "f:ports": {},
                                "f:project_id": {},
                                "f:security_groups_ids": {},
                                "f:subnet_id": {},
                                "f:type": {}
                            },
                            "f:status": {
                                ".": {},
                                "f:listeners": {},
                                "f:loadbalancer": {
                                    ".": {},
                                    "f:id": {},
                                    "f:ip": {},
                                    "f:name": {},
                                    "f:port_id": {},
                                    "f:project_id": {},
                                    "f:provider": {},
                                    "f:security_groups": {},
                                    "f:subnet_id": {}
                                },
                                "f:members": {},
                                "f:pools": {}
                            }
                        },
                        "manager": "python-requests",
                        "operation": "Update",
                        "time": "2020-09-07T09:25:01Z"
                    }
                ],
                "name": "demo",
                "namespace": "test",
                "resourceVersion": "90690",
                "selfLink": "/apis/openstack.org/v1/namespaces/test/kuryrloadbalancers/demo",
                "uid": "310fd15b-1527-4c92-b95a-75a0bd668451"
            },
            "spec": {
                "endpointSlices": [
                    {
                        "endpoints": [
                            {
                                "addresses": [
                                    "10.128.115.191"
                                ],
                                "conditions": {
                                    "ready": true
                                },
                                "targetRef": {
                                    "kind": "Pod",
                                    "name": "demo",
                                    "namespace": "test",
                                    "resourceVersion": "50977",
                                    "uid": "ec893c71-306f-4893-9bd2-052198ebc267"
                                }
                            }
                        ],
                        "ports": [
                            {
                                "port": 8080,
                                "protocol": "TCP"
                            }
                        ]
                    }
                ],
                "ip": "172.30.161.244",
                "ports": [
                    {
                        "port": 80,
                        "protocol": "TCP",
                        "targetPort": "8080"
                    }
                ],
                "project_id": "92c48c7c788145db89c97f2554aefa4f",
                "security_groups_ids": [
                    "d61b6bbb-7e01-4c3e-ac24-1a588b289524"
                ],
                "subnet_id": "d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9",
                "type": "ClusterIP"
            },
            "status": {
                "listeners": [
                    {
                        "id": "f53bcbb2-f866-4559-868a-2fc43a2ef289",
                        "loadbalancer_id": "e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1",
                        "name": "test/demo:TCP:80",
                        "port": 80,
                        "project_id": "92c48c7c788145db89c97f2554aefa4f",
                        "protocol": "TCP"
                    }
                ],
                "loadbalancer": {
                    "id": "e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1",
                    "ip": "172.30.161.244",
                    "name": "test/demo",
                    "port_id": "97f99ce6-fe86-4cf0-afe3-d640ac3b0561",
                    "project_id": "92c48c7c788145db89c97f2554aefa4f",
                    "provider": "ovn",
                    "security_groups": [
                        "d61b6bbb-7e01-4c3e-ac24-1a588b289524"
                    ],
                    "subnet_id": "d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9"
                },
                "members": [
                    {
                        "id": "807e585a-7944-4393-b395-0531f49c02d4",
                        "ip": "10.128.115.191",
                        "name": "test/demo:8080",
                        "pool_id": "96a06be9-4fe1-4c51-92ae-f17460b28d95",
                        "port": 8080,
                        "project_id": "92c48c7c788145db89c97f2554aefa4f",
                        "subnet_id": "34cb72a8-128a-4b82-af02-2b1d4d0ff2a0"
                    }
                ],
                "pools": [
                    {
                        "id": "96a06be9-4fe1-4c51-92ae-f17460b28d95",
                        "listener_id": "f53bcbb2-f866-4559-868a-2fc43a2ef289",
                        "loadbalancer_id": "e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1",
                        "name": "test/demo:TCP:80",
                        "project_id": "92c48c7c788145db89c97f2554aefa4f",
                        "protocol": "TCP"
                    }
                ]
            }
        }
    ],
    "kind": "List",
    "metadata": {
        "resourceVersion": "",
        "selfLink": ""
    }
}
(overcloud) [stack@undercloud-0 ~]$ oc rsh pod/demo curl 172.30.161.244
demo: HELLO! I AM ALIVE!!!

So after upgrade the annotations are moved to the kuryr loadbalancer CRD keeping the functionality working as expected.

Comment 5 errata-xmlrpc 2020-10-27 16:26:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196