Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1995609

Summary: [sig-node] Managed cluster should report ready nodes the entire duration of the test run [Late]
Product: OpenShift Container Platform Reporter: David Eads <deads>
Component: NodeAssignee: Ryan Phillips <rphillips>
Node sub component: Kubelet QA Contact: Sunil Choudhary <schoudha>
Status: CLOSED DUPLICATE Docs Contact:
Severity: high    
Priority: unspecified CC: aos-bugs, gmontero, harpatil, nagrawal, sippy, wking
Version: 4.7   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
[sig-node] Managed cluster should report ready nodes the entire duration of the test run [Late] [Skipped:Disconnected] [Suite:openshift/conformance/parallel]
Last Closed: 2021-08-24 14:49:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Eads 2021-08-19 13:35:57 UTC 
This test has failed on 4 of the past 5 4.7-ci payload promotion jobs.  Payload promotion status can be seen here: https://amd64.ocp.releases.ci.openshift.org/#4.7.0-0.ci

An example job is https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.7-e2e-aws-serial/1427314637354831872

It looks like

fail [github.com/openshift/origin/test/extended/machines/cluster.go:90]: Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        {
            s: "query failed: (min_over_time((max by (node) (kube_node_status_condition{condition=\"Ready\",status=\"true\"}) and (((max by (node) (kube_node_status_condition))) and (0*max by (node) (kube_node_status_condition offset 6m))))[57m14s:1s])) < 1: promQL query: (min_over_time((max by (node) (kube_node_status_condition{condition=\"Ready\",status=\"true\"}) and (((max by (node) (kube_node_status_condition))) and (0*max by (node) (kube_node_status_condition offset 6m))))[57m14s:1s])) < 1 had reported incorrect results:\n[{\"metric\":{\"node\":\"ip-10-0-194-127.us-west-1.compute.internal\"},\"value\":[1629138820.547,\"0\"]},{\"metric\":{\"node\":\"ip-10-0-169-218.us-west-1.compute.internal\"},\"value\":[1629138820.547,\"0\"]}]",
        },
    ]
    query failed: (min_over_time((max by (node) (kube_node_status_condition{condition="Ready",status="true"}) and (((max by (node) (kube_node_status_condition))) and (0*max by (node) (kube_node_status_condition offset 6m))))[57m14s:1s])) < 1: promQL query: (min_over_time((max by (node) (kube_node_status_condition{condition="Ready",status="true"}) and (((max by (node) (kube_node_status_condition))) and (0*max by (node) (kube_node_status_condition offset 6m))))[57m14s:1s])) < 1 had reported incorrect results:
    [{"metric":{"node":"ip-10-0-194-127.us-west-1.compute.internal"},"value":[1629138820.547,"0"]},{"metric":{"node":"ip-10-0-169-218.us-west-1.compute.internal"},"value":[1629138820.547,"0"]}]
occurred
Comment 1 Harshal Patil 2021-08-20 02:23:53 UTC 
*** Bug 1992242 has been marked as a duplicate of this bug. ***
Comment 3 Ryan Phillips 2021-08-24 14:49:44 UTC 
We did not backport the "stop probes on graceful shutdown." Being tracked over in 1996846.

*** This bug has been marked as a duplicate of bug 1996846 ***