Description of problem: With the support of Load Balancer CRD added on ocp 4.6 to allow simpler management of OpenStack resources created for a Service, we must ensure the upgrade path from lb annotations on Service and Enpoints to lb CRD is working as expected. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Verfied on 4.6.0-0.nightly-2020-09-05-015624 over RHOS-16.1-RHEL-8-20200821.n.0. 1. Install OCP4.5.8: $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.8 True False 69s Cluster version is 4.5.8 2. Create project and pods with service: $ oc new-project test Now using project "test" on server "https://api.ostest.shiftstack.com:6443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app rails-postgresql-example to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4 $ oc --image=kuryr/demo run demo pod/demo created $ oc --image=kuryr/demo run demo-caller pod/demo-caller created $ oc expose pod/demo --port 80 --target-port 8080 service/demo exposed $ oc get all NAME READY STATUS RESTARTS AGE pod/demo 1/1 Running 0 28s pod/demo-caller 1/1 Running 0 17s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/demo ClusterIP 172.30.161.244 <none> 80/TCP 10s $ oc rsh pod/demo curl 172.30.161.244 demo: HELLO! I AM ALIVE!!! $ . overcloudrc && openstack loadbalancer show test/demo +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | admin_state_up | True | | created_at | 2020-09-07T08:39:18 | | description | | | flavor_id | None | | id | e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1 | | listeners | f53bcbb2-f866-4559-868a-2fc43a2ef289 | | name | test/demo | | operating_status | ONLINE | | pools | 96a06be9-4fe1-4c51-92ae-f17460b28d95 | | project_id | 92c48c7c788145db89c97f2554aefa4f | | provider | ovn | | provisioning_status | ACTIVE | | updated_at | 2020-09-07T08:39:39 | | vip_address | 172.30.161.244 | | vip_network_id | 751c94d9-ff7c-43a0-9672-e92f89c363b6 | | vip_port_id | 97f99ce6-fe86-4cf0-afe3-d640ac3b0561 | | vip_qos_policy_id | None | | vip_subnet_id | d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9 | +---------------------+--------------------------------------+ $ oc get svc -o json | jq ".items[0].metadata.annotations" { "openstack.org/kuryr-lbaas-spec": "{\"versioned_object.data\": {\"ip\": \"172.30.161.244\", \"lb_ip\": null, \"ports\": [{\"versioned_object.data\": {\"name\": null, \"port\": 80, \"protocol\": \"TCP\", \"targetPort\": \"8080\"}, \"versioned_object.name\": \"LBaaSPortSpec\", \"versioned_object.namespace\": \"kuryr_kubernetes\", \"versioned_object.version\": \"1.1\"}], \"project_id\": \"92c48c7c788145db89c97f2554aefa4f\", \"security_groups_ids\": [\"d61b6bbb-7e01-4c3e-ac24-1a588b289524\"], \"subnet_id\": \"d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9\", \"type\": \"ClusterIP\"}, \"versioned_object.name\": \"LBaaSServiceSpec\", \"versioned_object.namespace\": \"kuryr_kubernetes\", \"versioned_object.version\": \"1.0\"}" } $ oc get pods -n openshift-kuryr -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kuryr-cni-6sfjr 1/1 Running 0 39m 10.196.3.94 ostest-xcbsx-master-0 <none> <none> kuryr-cni-g5jk6 1/1 Running 0 39m 10.196.0.33 ostest-xcbsx-master-2 <none> <none> kuryr-cni-kp8cr 1/1 Running 1 19m 10.196.1.58 ostest-xcbsx-worker-cb5kt <none> <none> kuryr-cni-qgc7s 1/1 Running 1 19m 10.196.2.90 ostest-xcbsx-worker-8sc92 <none> <none> kuryr-cni-tnqbg 1/1 Running 0 19m 10.196.2.16 ostest-xcbsx-worker-ctm65 <none> <none> kuryr-cni-tsj6x 1/1 Running 0 39m 10.196.3.204 ostest-xcbsx-master-1 <none> <none> kuryr-controller-6f999575cd-qsxdz 1/1 Running 2 39m 10.196.3.94 ostest-xcbsx-master-0 <none> <none> 3. Perform upgrade to 4.6.0-0.nightly-2020-09-05-015624: (overcloud) [stack@undercloud-0 ~]$ oc adm upgrade --to-image="registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-05-015624" --allow-explicit-upgrade --force=true warning: Using by-tag pull specs is dangerous, and while we still allow it in combination with --force for backward compatibility, it would be much safer to pass a by-digest pull spec instead warning: The requested upgrade image is not one of the available updates. You have used --allow-explicit-upgrade to the update to preceed anyway warning: --force overrides cluster verification of your supplied release image and waives any update precondition failures. Updating to release image registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-05-015624 (overcloud) [stack@undercloud-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.8 True True 9s Working towards registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-09-05-015624: downloading update (overcloud) [stack@undercloud-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.8 True True 97s Working towards 4.6.0-0.nightly-2020-09-05-015624: 11% complete (overcloud) [stack@undercloud-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.8 True True 50m Working towards 4.6.0-0.nightly-2020-09-05-015624: 79% complete (overcloud) [stack@undercloud-0 ~]$ oc get svc -o json | jq ".items[0].metadata.annotations" null (overcloud) [stack@undercloud-0 ~]$ oc get klb -o json { "apiVersion": "v1", "items": [ { "apiVersion": "openstack.org/v1", "kind": "KuryrLoadBalancer", "metadata": { "creationTimestamp": "2020-09-07T09:24:53Z", "finalizers": [ "kuryr.openstack.org/kuryrloadbalancer-finalizers" ], "generation": 3, "managedFields": [ { "apiVersion": "openstack.org/v1", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"kuryr.openstack.org/kuryrloadbalancer-finalizers\"": {} } }, "f:spec": { ".": {}, "f:endpointSlices": {}, "f:ip": {}, "f:ports": {}, "f:project_id": {}, "f:security_groups_ids": {}, "f:subnet_id": {}, "f:type": {} }, "f:status": { ".": {}, "f:listeners": {}, "f:loadbalancer": { ".": {}, "f:id": {}, "f:ip": {}, "f:name": {}, "f:port_id": {}, "f:project_id": {}, "f:provider": {}, "f:security_groups": {}, "f:subnet_id": {} }, "f:members": {}, "f:pools": {} } }, "manager": "python-requests", "operation": "Update", "time": "2020-09-07T09:25:01Z" } ], "name": "demo", "namespace": "test", "resourceVersion": "90690", "selfLink": "/apis/openstack.org/v1/namespaces/test/kuryrloadbalancers/demo", "uid": "310fd15b-1527-4c92-b95a-75a0bd668451" }, "spec": { "endpointSlices": [ { "endpoints": [ { "addresses": [ "10.128.115.191" ], "conditions": { "ready": true }, "targetRef": { "kind": "Pod", "name": "demo", "namespace": "test", "resourceVersion": "50977", "uid": "ec893c71-306f-4893-9bd2-052198ebc267" } } ], "ports": [ { "port": 8080, "protocol": "TCP" } ] } ], "ip": "172.30.161.244", "ports": [ { "port": 80, "protocol": "TCP", "targetPort": "8080" } ], "project_id": "92c48c7c788145db89c97f2554aefa4f", "security_groups_ids": [ "d61b6bbb-7e01-4c3e-ac24-1a588b289524" ], "subnet_id": "d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9", "type": "ClusterIP" }, "status": { "listeners": [ { "id": "f53bcbb2-f866-4559-868a-2fc43a2ef289", "loadbalancer_id": "e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1", "name": "test/demo:TCP:80", "port": 80, "project_id": "92c48c7c788145db89c97f2554aefa4f", "protocol": "TCP" } ], "loadbalancer": { "id": "e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1", "ip": "172.30.161.244", "name": "test/demo", "port_id": "97f99ce6-fe86-4cf0-afe3-d640ac3b0561", "project_id": "92c48c7c788145db89c97f2554aefa4f", "provider": "ovn", "security_groups": [ "d61b6bbb-7e01-4c3e-ac24-1a588b289524" ], "subnet_id": "d68acf4f-0ba5-4e09-a4c5-3da8ac1cd0d9" }, "members": [ { "id": "807e585a-7944-4393-b395-0531f49c02d4", "ip": "10.128.115.191", "name": "test/demo:8080", "pool_id": "96a06be9-4fe1-4c51-92ae-f17460b28d95", "port": 8080, "project_id": "92c48c7c788145db89c97f2554aefa4f", "subnet_id": "34cb72a8-128a-4b82-af02-2b1d4d0ff2a0" } ], "pools": [ { "id": "96a06be9-4fe1-4c51-92ae-f17460b28d95", "listener_id": "f53bcbb2-f866-4559-868a-2fc43a2ef289", "loadbalancer_id": "e24d8a38-6100-41ae-a8e2-0cbdd4a80bf1", "name": "test/demo:TCP:80", "project_id": "92c48c7c788145db89c97f2554aefa4f", "protocol": "TCP" } ] } } ], "kind": "List", "metadata": { "resourceVersion": "", "selfLink": "" } } (overcloud) [stack@undercloud-0 ~]$ oc rsh pod/demo curl 172.30.161.244 demo: HELLO! I AM ALIVE!!! So after upgrade the annotations are moved to the kuryr loadbalancer CRD keeping the functionality working as expected.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196