Bug 1387273 - OCP nodes showing as "not ready" in topology view but as "Ready" in Container Node view
Summary: OCP nodes showing as "not ready" in topology view but as "Ready" in Container...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: UI - OPS
Version: 5.6.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: GA
: 5.8.0
Assignee: Ari Zellner
QA Contact: Einat Pacifici
URL:
Whiteboard:
: 1388971 1391619 (view as bug list)
Depends On:
Blocks: 1395857
TreeView+ depends on / blocked
 
Reported: 2016-10-20 14:03 UTC by Ian Tewksbury
Modified: 2020-04-15 14:45 UTC (History)
13 users (show)

Fixed In Version: 5.8.0.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1395857 (view as bug list)
Environment:
Last Closed: 2017-06-12 16:25:53 UTC
Category: ---
Cloudforms Team: Container Management
Target Upstream Version:
Embargoed:
epacific: automate_bug+


Attachments (Terms of Use)
screen shot of topology with errors (71.45 KB, image/png)
2016-10-20 14:10 UTC, Ian Tewksbury
no flags Details
screen shot of container node view showing no errors (141.65 KB, image/png)
2016-10-20 14:10 UTC, Ian Tewksbury
no flags Details

Description Ian Tewksbury 2016-10-20 14:03:10 UTC
Description of problem:

The CFME topology view of OpenShift provider is showing some OCP nodes as "red" / "Not Ready" state. But going to the Container Node view all nodes are shown as in "ready" state.


Version-Release number of selected component (if applicable):

CFME: v5.6.0.11-rc2.2.20160614152915_f315c68
OCP: v3.3.0.34
Kubernetes: v1.3.0+52492b4

How reproducible:

Unknown. This is the current state of an existing infrastructure.

Steps to Reproduce:
Unknown.

Actual results:

1. Topology view shows some nodes in error but Container Node view shows all nodes as "ready"
2. Unknown why toplogy view shows nodes as "not ready"


Expected results:

1. Expect the Container Node view to show the same status as the Topology View
2. understand if actually in error what the issue is

Comment 3 Ian Tewksbury 2016-10-20 14:10:15 UTC
Created attachment 1212542 [details]
screen shot of topology with errors

Comment 4 Ian Tewksbury 2016-10-20 14:10:55 UTC
Created attachment 1212543 [details]
screen shot of container node view showing no errors

Comment 5 Federico Simoncelli 2016-10-20 14:13:45 UTC
I reported this once upstream but I couldn't reproduce it anymore:

https://github.com/ManageIQ/manageiq/issues/7363

Ari can you look into this?

Comment 6 Federico Simoncelli 2016-10-27 08:27:39 UTC
*** Bug 1388971 has been marked as a duplicate of this bug. ***

Comment 7 Ari Zellner 2016-10-27 10:55:14 UTC
Upstream PR: https://github.com/ManageIQ/manageiq/pull/12236

Comment 8 Ari Zellner 2016-11-15 11:36:21 UTC
*** Bug 1391619 has been marked as a duplicate of this bug. ***

Comment 9 Oleg Barenboim 2016-11-15 16:14:57 UTC
PR https://github.com/ManageIQ/manageiq/pull/12236 has been merged.

Comment 10 Oleg Barenboim 2016-11-15 16:15:29 UTC
Should this be moved to POST?

Comment 13 Dafna Ron 2017-02-16 14:44:27 UTC
both /container_node/show/1 and container_topology/show show nodes in status Ready. 
verified on cfme-5.7.1.1-1.el7cf.x86_64

Comment 14 Dafna Ron 2017-02-16 17:02:23 UTC
just noticed this is verified for the wrong version,

Comment 15 Dafna Ron 2017-02-17 11:17:21 UTC
verified on cfme-5.8.0.1-1.el7cf.x86_64
nodes from /container_topology/show and nodes from container_node/show_list
will show the same status of nodes

Comment 16 shrader 2017-11-04 04:04:24 UTC
Not sure where to report this, but nodes no longer show as "Not Ready" or "Ready" = "False" in CFME when the nodes are not ready. They now show as "Ready" = "Unknown" in the Container Nodes information page, and in the topology the node will show as grey and "Unknown" instead of Not Ready.

Comment 17 Ari Zellner 2017-11-05 08:56:55 UTC
>Not sure where to report this, but nodes no longer show as "Not Ready" or "Ready" = "False" in CFME when the nodes are not ready. They now show as "Ready" = "Unknown" in the Container Nodes information page, and in the topology the node will show as grey and "Unknown" instead of Not Ready

All nodes? Always? What is their status on the openshift side?

Comment 18 shrader 2017-12-26 00:18:04 UTC
(In reply to Ari Zellner from comment #17)
> >Not sure where to report this, but nodes no longer show as "Not Ready" or "Ready" = "False" in CFME when the nodes are not ready. They now show as "Ready" = "Unknown" in the Container Nodes information page, and in the topology the node will show as grey and "Unknown" instead of Not Ready
> 
> All nodes? Always? What is their status on the openshift side?

Hi, sorry for the delayed response! 

All nodes that are "Ready" on the OpenShift side are displayed correctly, but if a node changes status to "NotReady," it is not recognized properly in the topology. Also, in the node information page in CFME the "Conditions -> Ready -> Status" lists as "Unknown" rather than "False" (the expected result would be to list as "False" if the node status on the OpenShift side changes to "NotReady").

In the topology, the "Condition -> Ready -> Status" is what determines how the node is displayed, and since it is listed as "Unknown" instead of "False," the topology shows the nodes as light gray (barely noticeable), rather than red. When using mouse hover over the node, it displays the status as "Unknown" rather than "NotReady."

This only happens in the condition when the node status is listed in CFME as "Unknown." OpenShift nodes listed as "NotReady" on the OpenShift side always list as "Unknown" on the CloudForms side. All nodes listed as "Ready" on the OpenShift side display correctly in CloudForms.

Thanks, and Happy Holidays!

Comment 19 shrader 2017-12-26 00:18:33 UTC
(In reply to Ari Zellner from comment #17)
> >Not sure where to report this, but nodes no longer show as "Not Ready" or "Ready" = "False" in CFME when the nodes are not ready. They now show as "Ready" = "Unknown" in the Container Nodes information page, and in the topology the node will show as grey and "Unknown" instead of Not Ready
> 
> All nodes? Always? What is their status on the openshift side?

Hi, sorry for the delayed response! 

All nodes that are "Ready" on the OpenShift side are displayed correctly, but if a node changes status to "NotReady," it is not recognized properly in the topology. Also, in the node information page in CFME the "Conditions -> Ready -> Status" lists as "Unknown" rather than "False" (the expected result would be to list as "False" if the node status on the OpenShift side changes to "NotReady").

In the topology, the "Condition -> Ready -> Status" is what determines how the node is displayed, and since it is listed as "Unknown" instead of "False," the topology shows the nodes as light gray (barely noticeable), rather than red. When using mouse hover over the node, it displays the status as "Unknown" rather than "NotReady."

This only happens in the condition when the node status is listed in CFME as "Unknown." OpenShift nodes listed as "NotReady" on the OpenShift side always list as "Unknown" on the CloudForms side. All nodes listed as "Ready" on the OpenShift side display correctly in CloudForms.

Thanks, and Happy Holidays!

Comment 20 Ari Zellner 2017-12-31 11:58:52 UTC
(In reply to shrader from comment #19)
> ....
> status as "Unknown" rather than "NotReady."
> 
> This only happens in the condition when the node status is listed in CFME as
> "Unknown." OpenShift nodes listed as "NotReady" on the OpenShift side always
> list as "Unknown" on the CloudForms side. All nodes listed as "Ready" on the
> OpenShift side display correctly in CloudForms.
> 
> Thanks, and Happy Holidays!

This BZ isnt a good fit for that. Please report this in a separate BZ.

Comment 21 Ari Zellner 2017-12-31 12:15:01 UTC
The info seems correct according to `oc describe` and not `oc get nodes`
Not sure whats correct. Either way, different BZ.

[root@vm-48-184 ~]# oc get nodes 
NAME                                STATUS                     AGE       VERSION
ocp-compute01.10.35...   NotReady                   30d       v1.7.6+a08f5eeb62
ocp-compute02.10.35....   Ready                      30d       v1.7.6+a08f5eeb62
ocp-infra01.10.35...     Ready                      30d       v1.7.6+a08f5eeb62
ocp-infra02.10.35...     Ready                      30d       v1.7.6+a08f5eeb62
ocp-master01.10.35...    Ready,SchedulingDisabled   30d       v1.7.6+a08f5eeb62
[root@vm-48-184 ~]# oc describe node ocp-compute01.10.35...
Name:			ocp-compute01.10.35...
Role:			
Labels:			beta.kubernetes.io/arch=amd64
			beta.kubernetes.io/os=linux
			kubernetes.io/hostname=ocp-compute01.10.35...
			region=primary
			zone=default
Annotations:		volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:			<none>
CreationTimestamp:	Thu, 30 Nov 2017 17:04:53 +0200
Conditions:
  Type			Status		LastHeartbeatTime			LastTransitionTime			Reason			Message
  ----			------		-----------------			------------------			------			-------
  OutOfDisk 		Unknown 	Sun, 31 Dec 2017 14:01:44 +0200 	Sun, 31 Dec 2017 14:02:24 +0200 	NodeStatusUnknown 	Kubelet stopped posting node status.
  MemoryPressure 	Unknown 	Sun, 31 Dec 2017 14:01:44 +0200 	Sun, 31 Dec 2017 14:02:24 +0200 	NodeStatusUnknown 	Kubelet stopped posting node status.
  DiskPressure 		Unknown 	Sun, 31 Dec 2017 14:01:44 +0200 	Sun, 31 Dec 2017 14:02:24 +0200 	NodeStatusUnknown 	Kubelet stopped posting node status.
  Ready 		Unknown 	Sun, 31 Dec 2017 14:01:44 +0200 	Sun, 31 Dec 2017 14:02:24 +0200 	NodeStatusUnknown 	Kubelet stopped posting node status.


Note You need to log in before you can comment on or make changes to this bug.