Bug 1690333

Summary: the status.conditions of ingress operator is not updated after upgrade
Product: OpenShift Container Platform Reporter: Hongan Li <hongli>
Component: NetworkingAssignee: Daneyon Hansen <dhansen>
Networking sub component: router QA Contact: Hongan Li <hongli>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: aos-bugs, dhansen
Version: 4.1.0   
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:46:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hongan Li 2019-03-19 10:13:59 UTC
Description of problem:
the status.conditions of ingress operator is not updated after upgrade

Version-Release number of selected component (if applicable):
4.0.0-0.nightly-2019-03-18-200009

How reproducible:
always

Steps to Reproduce:
1. install cluster with 4.0.0-0.nightly-2019-03-15-043409 
2. $ oc get clusteroperator ingress -o yaml
3. upgrade to 4.0.0-0.nightly-2019-03-18-200009
4. $ oc get clusteroperator ingress -o yaml

Actual results:
the status.conditions is not updated and still show old timestamp.

---> step2 (before upgrade):
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: 2019-03-18T09:42:01Z
  generation: 1
  name: ingress
  resourceVersion: "15983"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/ingress
  uid: 14cb9baa-4962-11e9-a704-068f80088776
spec: {}
status:
  conditions:
  - lastTransitionTime: 2019-03-18T09:42:02Z
    status: "False"
    type: Failing
  - lastTransitionTime: 2019-03-18T09:42:02Z
    status: "False"
    type: Progressing
  - lastTransitionTime: 2019-03-18T09:47:11Z
    status: "True"
    type: Available
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-ingress-operator
    resource: namespaces
  - group: ""
    name: openshift-ingress
    resource: namespaces
  versions:
  - name: operator
    version: 4.0.0-0.nightly-2019-03-15-043409
  - name: ingress-controller
    version: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:930f94f53c1062d94d226afa25a7e85f321ba2bfd63b5c481cc2c1d8121a2a49

---> step4 (after upgrade):
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: 2019-03-18T09:42:01Z
  generation: 1
  name: ingress
  resourceVersion: "753378"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/ingress
  uid: 14cb9baa-4962-11e9-a704-068f80088776
spec: {}
status:
  conditions:
  - lastTransitionTime: 2019-03-18T09:42:02Z
    status: "False"
    type: Failing
  - lastTransitionTime: 2019-03-18T09:42:02Z
    status: "False"
    type: Progressing
  - lastTransitionTime: 2019-03-18T09:47:11Z
    status: "True"
    type: Available
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-ingress-operator
    resource: namespaces
  - group: ""
    name: openshift-ingress
    resource: namespaces
  versions:
  - name: operator
    version: 4.0.0-0.nightly-2019-03-18-200009
  - name: ingress-controller
    version: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:930f94f53c1062d94d226afa25a7e85f321ba2bfd63b5c481cc2c1d8121a2a49


Expected results:
status.conditions should be updated after upgrade

Additional info:

Comment 1 Daneyon Hansen 2019-03-25 05:00:54 UTC
I have verified this bug by upgrading from 4.0.0-0.alpha-2019-03-23-160009 > 4.0.0-0.alpha-2019-03-23-190840. The status versions are properly updated, but not the lastTransitionTime:

$ oc get clusteroperator ingress -o yaml
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: 2019-03-23T17:58:12Z
  generation: 1
  name: ingress
  resourceVersion: "11660"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/ingress
  uid: 39998a1c-4d95-11e9-aaaf-067d24cbdbb6
spec: {}
status:
  conditions:
  - lastTransitionTime: 2019-03-23T17:58:13Z
    status: "False"
    type: Failing
  - lastTransitionTime: 2019-03-23T17:58:13Z
    status: "False"
    type: Progressing
  - lastTransitionTime: 2019-03-23T17:59:01Z
    status: "True"
    type: Available
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-ingress-operator
    resource: namespaces
  - group: ""
    name: openshift-ingress
    resource: namespaces
  versions:
  - name: operator
    version: 4.0.0-0.alpha-2019-03-23-160009
  - name: ingress-controller
    version: registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-03-23-160009@sha256:43ad3327ac1aec2c5c33eaa3bc4517ca475999b2c882c366b08e5716b8d15c59
$ 
$ oc get clusteroperator ingress -o yaml
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: 2019-03-23T17:58:12Z
  generation: 1
  name: ingress
  resourceVersion: "563318"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/ingress
  uid: 39998a1c-4d95-11e9-aaaf-067d24cbdbb6
spec: {}
status:
  conditions:
  - lastTransitionTime: 2019-03-23T17:58:13Z
    status: "False"
    type: Failing
  - lastTransitionTime: 2019-03-23T17:58:13Z
    status: "False"
    type: Progressing
  - lastTransitionTime: 2019-03-23T17:59:01Z
    status: "True"
    type: Available
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-ingress-operator
    resource: namespaces
  - group: ""
    name: openshift-ingress
    resource: namespaces
  versions:
  - name: operator
    version: 4.0.0-0.alpha-2019-03-23-190840
  - name: ingress-controller
    version: registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-03-23-190840@sha256:43ad3327ac1aec2c5c33eaa3bc4517ca475999b2c882c366b08e5716b8d15c59

Comment 2 Hongan Li 2019-03-25 05:27:08 UTC
Thank you for your update, Daneyon.
Yes, the status.versions are updated, but the lastTransitionTime in status.conditions are not updated.

Comment 3 Daneyon Hansen 2019-03-25 15:19:37 UTC
I believe PR 175 [1] will fix the bug. I will test 'lastTransitionTime' is properly updated during an upgrade after the PR merges.

[1] https://github.com/openshift/cluster-ingress-operator/pull/175

Comment 6 Hongan Li 2019-04-03 09:41:52 UTC
verified with upgrading from 4.0.0-0.nightly-2019-04-02-081046 to 4.0.0-0.nightly-2019-04-02-133735 and the issue has been fixed.

The timestamp of lastTransitionTime is updated after upgrade.

---before upgrade
status:
  conditions:
  - lastTransitionTime: 2019-04-03T08:17:15Z
    status: "False"
    type: Failing
  - lastTransitionTime: 2019-04-03T08:21:09Z
    status: "False"
    type: Progressing
  - lastTransitionTime: 2019-04-03T08:21:09Z
    status: "True"
    type: Available

---after upgrade
status:
  conditions:
  - lastTransitionTime: 2019-04-03T08:17:15Z
    status: "False"
    type: Failing
  - lastTransitionTime: 2019-04-03T08:57:21Z
    status: "False"
    type: Progressing
  - lastTransitionTime: 2019-04-03T08:57:21Z
    status: "True"
    type: Available

Comment 8 errata-xmlrpc 2019-06-04 10:46:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758