Description of problem: After upgrade OCP 4.0, the clusteroperator machine-api didn't update lastTransitionTime field Version-Release number of selected component (if applicable): $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.0.0-0.nightly-2019-03-04-234414 True False 21m Cluster version is 4.0.0-0.nightly-2019-03-04-234414 How reproducible: Always Steps to Reproduce: 1. Install a cluster 4.0 with 4.0.0-0.nightly-2019-03-04-234414 version. 2. Check the clusteroperator machine-api status, as below: apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: 2019-03-14T02:59:46Z generation: 1 name: machine-api resourceVersion: "21449" selfLink: /apis/config.openshift.io/v1/clusteroperators/machine-api uid: 39cc3c3d-4605-11e9-85a3-023bb53f2bf0 spec: {} status: conditions: - lastTransitionTime: null status: "False" type: Progressing - lastTransitionTime: 2019-03-14T02:59:47Z message: 'Cluster Machine API Operator is available at operator: 4.0.0-0.nightly-2019-03-04-234414' status: "True" type: Available - lastTransitionTime: 2019-03-14T02:59:47Z status: "False" type: Failing extension: null relatedObjects: null versions: - name: operator version: 4.0.0-0.nightly-2019-03-04-234414 3. Upgrade the cluster 4. Check the status of clusteroperator machine-api status. Actual results: After upgrade the version info was updated, the status->conditions->lastTransitionTime didn't get updated. apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: 2019-03-14T02:59:46Z generation: 1 name: machine-api resourceVersion: "33053" selfLink: /apis/config.openshift.io/v1/clusteroperators/machine-api uid: 39cc3c3d-4605-11e9-85a3-023bb53f2bf0 spec: {} status: conditions: - lastTransitionTime: null status: "False" type: Progressing - lastTransitionTime: 2019-03-14T02:59:47Z message: 'Cluster Machine API Operator is available at operator: 4.0.0-0.nightly-2019-03-05-045224' status: "True" type: Available - lastTransitionTime: 2019-03-14T02:59:47Z status: "False" type: Failing extension: null relatedObjects: null versions: - name: operator version: 4.0.0-0.nightly-2019-03-05-045224 Expected results: After upgrade, the status->conditions->lastTransitionTime should be updated, too. Additional info:
> relatedObjects: null The related objects should NOT be null.
This is the result of: openshift/machine-api-operator/vendor/github.com/openshift/cluster-version-operator/lib/resourcemerge/os.go This library only compares previous condition with current condition. In the case of machine-api, we are always status=available during upgrade as we're not checking an operand yet.
Looking into this a bit more, it appears we don't update the operator version until after we wait for deployment rollout of new version, even though we don't 'check' the operand version directly. Unsure what the root cause is at this time. Do you happen to have machine-api-operator verbose logs?
Proposed fix to update transition time for progressing:true -> progressing:false https://github.com/openshift/cluster-version-operator/pull/153 Will need specific change for available transition time after update, if desired.
Verified. Upgrade from 4.0.0-0.nightly-2019-04-10-141956 to 4.0.0-0.nightly-2019-04-10-182914 Before upgrade: $ oc get clusteroperator machine-api -o yaml apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: 2019-04-11T02:31:45Z generation: 1 name: machine-api resourceVersion: "25972" selfLink: /apis/config.openshift.io/v1/clusteroperators/machine-api uid: f2d395da-5c01-11e9-a622-02a87c79c5de spec: {} status: conditions: - lastTransitionTime: 2019-04-11T02:32:09Z status: "False" type: Progressing - lastTransitionTime: 2019-04-11T02:31:45Z message: 'Cluster Machine API Operator is available at operator: 4.0.0-0.nightly-2019-04-10-141956' status: "True" type: Available - lastTransitionTime: 2019-04-11T02:31:45Z status: "False" type: Failing extension: null versions: - name: operator version: 4.0.0-0.nightly-2019-04-10-141956 After upgrade: $ oc get clusteroperator machine-api -o yaml apiVersion: config.openshift.io/v1 kind: ClusterOperator metadata: creationTimestamp: 2019-04-11T02:31:45Z generation: 1 name: machine-api resourceVersion: "122537" selfLink: /apis/config.openshift.io/v1/clusteroperators/machine-api uid: f2d395da-5c01-11e9-a622-02a87c79c5de spec: {} status: conditions: - lastTransitionTime: 2019-04-11T03:43:13Z status: "False" type: Progressing - lastTransitionTime: 2019-04-11T02:31:45Z message: 'Cluster Machine API Operator is available at operator: 4.0.0-0.nightly-2019-04-10-182914' status: "True" type: Available - lastTransitionTime: 2019-04-11T02:31:45Z status: "False" type: Failing extension: null versions: - name: operator version: 4.0.0-0.nightly-2019-04-10-182914
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758