Bug 1927836 - Node becomes NotReady during upgrade; upgrade does not complete
Summary: Node becomes NotReady during upgrade; upgrade does not complete
Keywords:
Status: CLOSED DUPLICATE of bug 1913532
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.6
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: ---
Assignee: Ryan Phillips
QA Contact: Sunil Choudhary
URL:
Whiteboard:
Depends On:
Blocks: 1927373
TreeView+ depends on / blocked
 
Reported: 2021-02-11 16:15 UTC by Ruth Netser
Modified: 2021-02-16 14:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-16 14:49:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Ruth Netser 2021-02-11 16:15:41 UTC
Description of problem:
During OCP 4.6.16 (with CNV 2.5.3) upgrade to OCP 4.7.0-rc.1, one node becomes NotReady. The upgrade is not progressing:
version   4.6.16    True        True          143m    Unable to apply 4.7.0-rc.1: an unknown error has occurred: MultipleErrors


Version-Release number of selected component (if applicable):
OCP 4.6.16 (with CNV 2.5.3) upgrade to OCP 4.7.0-rc.1

How reproducible:
Observed 4 times, but before thought it was caused by bug 1918609

Steps to Reproduce:
1. OCP 4.6.16 with CNV 2.5.3
2. Run 3 VMs (2 can be live migrated, 1 cannot)
3. Start an upgrade to OCP 4.6.16 (with CNV 2.5.3) 

Actual results:
One node becomes NotReady. The upgrade is not progressing.
This node had running workloads on it before the upgrade. A 3 workload is not live migrated.

Expected results:
Upgrade should complete successfully.

Additional info:
===============================================
$ oc get clusterversion 
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.16    True        True          143m    Unable to apply 4.7.0-rc.1: an unknown error has occurred: MultipleErrors


===============================================
$ oc get node
NAME                         STATUS                     ROLES    AGE   VERSION
ssp04-rvkqg-master-0         Ready                      master   23h   v1.20.0+ba45583
ssp04-rvkqg-master-1         Ready                      master   23h   v1.20.0+ba45583
ssp04-rvkqg-master-2         Ready                      master   23h   v1.20.0+ba45583
ssp04-rvkqg-worker-0-9knk9   Ready                      worker   22h   v1.20.0+ba45583
ssp04-rvkqg-worker-0-ns68h   Ready,SchedulingDisabled   worker   22h   v1.19.0+e49167a
ssp04-rvkqg-worker-0-zvfkw   NotReady                   worker   22h   v1.19.0+e49167a

===============================================

$ oc get vmi
NAME           AGE    PHASE        IP            NODENAME
fedora33-hpp   90m    Scheduling                 
rhel8-nfs      178m   Running      10.131.0.47   ssp04-rvkqg-worker-0-zvfkw
win10-ocs      166m   Running      10.131.0.48   ssp04-rvkqg-worker-0-zvfkw


===============================================
$ oc get co
NAME                                       VERSION      AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.7.0-rc.1   True        False         False      26s
baremetal                                  4.7.0-rc.1   True        False         False      124m
cloud-credential                           4.7.0-rc.1   True        False         False      23h
cluster-autoscaler                         4.7.0-rc.1   True        False         False      23h
config-operator                            4.7.0-rc.1   True        False         False      23h
console                                    4.7.0-rc.1   True        False         False      79m
csi-snapshot-controller                    4.7.0-rc.1   True        False         False      86m
dns                                        4.7.0-rc.1   True        False         False      23h
etcd                                       4.7.0-rc.1   True        False         False      23h
image-registry                             4.7.0-rc.1   True        False         False      87m
ingress                                    4.7.0-rc.1   True        False         True       22h
insights                                   4.7.0-rc.1   True        False         True       23h
kube-apiserver                             4.7.0-rc.1   True        False         False      23h
kube-controller-manager                    4.7.0-rc.1   True        False         False      23h
kube-scheduler                             4.7.0-rc.1   True        False         False      23h
kube-storage-version-migrator              4.7.0-rc.1   True        False         False      85m
machine-api                                4.7.0-rc.1   True        False         False      23h
machine-approver                           4.7.0-rc.1   True        False         False      23h
machine-config                             4.6.16       False       True          True       100m
marketplace                                4.7.0-rc.1   True        False         False      78m
monitoring                                 4.7.0-rc.1   False       True          True       86m
network                                    4.7.0-rc.1   True        True          True       100m
node-tuning                                4.7.0-rc.1   True        False         False      121m
openshift-apiserver                        4.7.0-rc.1   True        False         False      72m
openshift-controller-manager               4.7.0-rc.1   True        False         False      23h
openshift-samples                          4.7.0-rc.1   True        False         False      121m
operator-lifecycle-manager                 4.7.0-rc.1   True        False         False      23h
operator-lifecycle-manager-catalog         4.7.0-rc.1   True        False         False      23h
operator-lifecycle-manager-packageserver   4.7.0-rc.1   True        False         False      79m
service-ca                                 4.7.0-rc.1   True        False         False      23h
storage                                    4.7.0-rc.1   True        True          False      79m


===============================================
$ oc describe node ssp04-rvkqg-worker-0-zvfkw
Name:               ssp04-rvkqg-worker-0-zvfkw
Roles:              worker
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/instance-type=ci.nested.virt.m1.xlarge
                    beta.kubernetes.io/os=linux
                    cluster.ocs.openshift.io/openshift-storage=
                    cpumanager=true
                    failure-domain.beta.kubernetes.io/zone=nova
                    feature.node.kubernetes.io/cpu-feature-aes=true
                    feature.node.kubernetes.io/cpu-feature-avx=true
                    feature.node.kubernetes.io/cpu-feature-avx2=true
                    feature.node.kubernetes.io/cpu-feature-bmi1=true
                    feature.node.kubernetes.io/cpu-feature-bmi2=true
                    feature.node.kubernetes.io/cpu-feature-erms=true
                    feature.node.kubernetes.io/cpu-feature-f16c=true
                    feature.node.kubernetes.io/cpu-feature-fma=true
                    feature.node.kubernetes.io/cpu-feature-fsgsbase=true
                    feature.node.kubernetes.io/cpu-feature-invpcid=true
                    feature.node.kubernetes.io/cpu-feature-movbe=true
                    feature.node.kubernetes.io/cpu-feature-pcid=true
                    feature.node.kubernetes.io/cpu-feature-pclmuldq=true
                    feature.node.kubernetes.io/cpu-feature-popcnt=true
                    feature.node.kubernetes.io/cpu-feature-rdrand=true
                    feature.node.kubernetes.io/cpu-feature-rdtscp=true
                    feature.node.kubernetes.io/cpu-feature-smep=true
                    feature.node.kubernetes.io/cpu-feature-spec-ctrl=true
                    feature.node.kubernetes.io/cpu-feature-sse4.2=true
                    feature.node.kubernetes.io/cpu-feature-svm=true
                    feature.node.kubernetes.io/cpu-feature-tsc-deadline=true
                    feature.node.kubernetes.io/cpu-feature-vme=true
                    feature.node.kubernetes.io/cpu-feature-x2apic=true
                    feature.node.kubernetes.io/cpu-feature-xsave=true
                    feature.node.kubernetes.io/cpu-model-Haswell-noTSX=true
                    feature.node.kubernetes.io/cpu-model-Haswell-noTSX-IBRS=true
                    feature.node.kubernetes.io/cpu-model-IvyBridge=true
                    feature.node.kubernetes.io/cpu-model-IvyBridge-IBRS=true
                    feature.node.kubernetes.io/cpu-model-Nehalem=true
                    feature.node.kubernetes.io/cpu-model-Nehalem-IBRS=true
                    feature.node.kubernetes.io/cpu-model-Opteron_G1=true
                    feature.node.kubernetes.io/cpu-model-Opteron_G2=true
                    feature.node.kubernetes.io/cpu-model-Penryn=true
                    feature.node.kubernetes.io/cpu-model-SandyBridge=true
                    feature.node.kubernetes.io/cpu-model-SandyBridge-IBRS=true
                    feature.node.kubernetes.io/cpu-model-Westmere=true
                    feature.node.kubernetes.io/cpu-model-Westmere-IBRS=true
                    feature.node.kubernetes.io/cpu-model-kvm32=true
                    feature.node.kubernetes.io/cpu-model-kvm64=true
                    feature.node.kubernetes.io/cpu-model-qemu32=true
                    feature.node.kubernetes.io/cpu-model-qemu64=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-base=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-frequencies=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-ipi=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-reenlightenment=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-reset=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-runtime=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synic=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synic2=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synictimer=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-time=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-tlbflush=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-vpindex=true
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=ssp04-rvkqg-worker-0-zvfkw
                    kubernetes.io/os=linux
                    kubevirt.io/schedulable=false
                    node-role.kubernetes.io/worker=
                    node.kubernetes.io/instance-type=ci.nested.virt.m1.xlarge
                    node.openshift.io/os_id=rhcos
                    topology.cinder.csi.openstack.org/zone=nova
                    topology.kubernetes.io/zone=nova
                    topology.rook.io/rack=rack2
Annotations:        csi.volume.kubernetes.io/nodeid:
                      {"cinder.csi.openstack.org":"885b9301-803e-43b9-a6c8-c07324f015e6","manila.csi.openstack.org":"ssp04-rvkqg-worker-0-zvfkw","openshift-stor...
                    kubevirt.io/heartbeat: 2021-02-11T14:46:59Z
                    machine.openshift.io/machine: openshift-machine-api/ssp04-rvkqg-worker-0-zvfkw
                    machineconfiguration.openshift.io/currentConfig: rendered-worker-e32d0c5fbb975fcd1cd2ff67ba0f0965
                    machineconfiguration.openshift.io/desiredConfig: rendered-worker-e32d0c5fbb975fcd1cd2ff67ba0f0965
                    machineconfiguration.openshift.io/reason: 
                    machineconfiguration.openshift.io/state: Done
                    node-labeller-feature.node.kubernetes.io/cpu-feature-aes: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-avx: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-avx2: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-bmi1: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-bmi2: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-erms: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-f16c: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-fma: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-fsgsbase: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-invpcid: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-movbe: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-pcid: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-pclmuldq: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-popcnt: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-rdrand: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-rdtscp: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-smep: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-spec-ctrl: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-sse4.2: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-svm: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-tsc-deadline: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-vme: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-x2apic: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-xsave: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Haswell-noTSX: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Haswell-noTSX-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-IvyBridge: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-IvyBridge-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Nehalem: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Nehalem-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Opteron_G1: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Opteron_G2: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Penryn: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-SandyBridge: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-SandyBridge-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Westmere: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Westmere-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-kvm32: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-kvm64: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-qemu32: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-qemu64: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-base: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-frequencies: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-ipi: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-reenlightenment: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-reset: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-runtime: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synic: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synic2: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synictimer: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-time: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-tlbflush: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-vpindex: true
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 10 Feb 2021 17:12:28 +0000
Taints:             node.kubernetes.io/unreachable:NoExecute
                    node.kubernetes.io/unreachable:NoSchedule
Unschedulable:      false
Lease:
  HolderIdentity:  ssp04-rvkqg-worker-0-zvfkw
  AcquireTime:     <unset>
  RenewTime:       Thu, 11 Feb 2021 14:47:18 +0000
Conditions:
  Type             Status    LastHeartbeatTime                 LastTransitionTime                Reason              Message
  ----             ------    -----------------                 ------------------                ------              -------
  MemoryPressure   Unknown   Thu, 11 Feb 2021 14:47:20 +0000   Thu, 11 Feb 2021 14:48:01 +0000   NodeStatusUnknown   Kubelet stopped posting node status.
  DiskPressure     Unknown   Thu, 11 Feb 2021 14:47:20 +0000   Thu, 11 Feb 2021 14:48:01 +0000   NodeStatusUnknown   Kubelet stopped posting node status.
  PIDPressure      Unknown   Thu, 11 Feb 2021 14:47:20 +0000   Thu, 11 Feb 2021 14:48:01 +0000   NodeStatusUnknown   Kubelet stopped posting node status.
  Ready            Unknown   Thu, 11 Feb 2021 14:47:20 +0000   Thu, 11 Feb 2021 14:48:01 +0000   NodeStatusUnknown   Kubelet stopped posting node status.
Addresses:
  InternalIP:  192.168.1.21
  Hostname:    ssp04-rvkqg-worker-0-zvfkw
Capacity:
  attachable-volumes-cinder:        256
  cpu:                              8
  devices.kubevirt.io/kvm:          110
  devices.kubevirt.io/tun:          110
  devices.kubevirt.io/vhost-net:    110
  ephemeral-storage:                41391084Ki
  hugepages-1Gi:                    0
  hugepages-2Mi:                    0
  memory:                           16418260Ki
  ovs-cni.network.kubevirt.io/br0:  1k
  pods:                             250
Allocatable:
  attachable-volumes-cinder:        256
  cpu:                              7500m
  devices.kubevirt.io/kvm:          110
  devices.kubevirt.io/tun:          110
  devices.kubevirt.io/vhost-net:    110
  ephemeral-storage:                37072281128
  hugepages-1Gi:                    0
  hugepages-2Mi:                    0
  memory:                           15267284Ki
  ovs-cni.network.kubevirt.io/br0:  1k
  pods:                             250
System Info:
  Machine ID:                             885b9301803e43b9a6c8c07324f015e6
  System UUID:                            885b9301-803e-43b9-a6c8-c07324f015e6
  Boot ID:                                8e64b1bd-f422-411c-a633-c451e40469ec
  Kernel Version:                         4.18.0-193.41.1.el8_2.x86_64
  OS Image:                               Red Hat Enterprise Linux CoreOS 46.82.202101301821-0 (Ootpa)
  Operating System:                       linux
  Architecture:                           amd64
  Container Runtime Version:              cri-o://1.19.1-7.rhaos4.6.git6377f68.el8
  Kubelet Version:                        v1.19.0+e49167a
  Kube-Proxy Version:                     v1.19.0+e49167a
ProviderID:                               openstack:///885b9301-803e-43b9-a6c8-c07324f015e6
Non-terminated Pods:                      (69 in total)
  Namespace                               Name                                                               CPU Requests  CPU Limits  Memory Requests   Memory Limits  AGE
  ---------                               ----                                                               ------------  ----------  ---------------   -------------  ---
  b4-upgarde                              virt-launcher-rhel8-nfs-rr9tx                                      100m (1%)     100m (1%)   1709Mi (11%)      40M (0%)       175m
  b4-upgarde                              virt-launcher-win10-ocs-2kx2v                                      100m (1%)     100m (1%)   4481613825 (28%)  40M (0%)       163m
  openshift-cluster-csi-drivers           openstack-cinder-csi-driver-node-h4pp8                             20m (0%)      0 (0%)      100Mi (0%)        0 (0%)         121m
  openshift-cluster-node-tuning-operator  tuned-7ll2f                                                        10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         120m
  openshift-cnv                           bridge-marker-n9pqq                                                0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           cdi-apiserver-fd7f4fb6-q2b8v                                       0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           cdi-deployment-65c8dbfdc7-hxms2                                    0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           cdi-uploadproxy-c9ff9fc78-qp99p                                    0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           cluster-network-addons-operator-69dd97cb44-l26h9                   0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-cnv                           hostpath-provisioner-operator-55d7b8b595-zhbk9                     0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-cnv                           hostpath-provisioner-vhvr7                                         0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           kube-cni-linux-bridge-plugin-zvvcf                                 60m (0%)      0 (0%)      30Mi (0%)         0 (0%)         21h
  openshift-cnv                           kubevirt-node-labeller-qmmb6                                       0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           nmstate-handler-ssqsc                                              0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           ovs-cni-amd64-t8cbz                                                0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           virt-api-6d765b6dd5-t2v5l                                          0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           virt-controller-68985f6974-mv2vb                                   0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           virt-handler-mzfgg                                                 0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           virt-operator-7d57664b89-nzjcl                                     0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-cnv                           virt-template-validator-65dbfc87d8-56dks                           0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           virt-template-validator-65dbfc87d8-89rvn                           0 (0%)        0 (0%)      0 (0%)            0 (0%)         87m
  openshift-cnv                           vm-import-controller-74d785b999-4vrkx                              0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-cnv                           vm-import-operator-7d856d4f4b-kdzh6                                0 (0%)        0 (0%)      0 (0%)            0 (0%)         87m
  openshift-dns                           dns-default-pn9k5                                                  65m (0%)      0 (0%)      110Mi (0%)        512Mi (3%)     100m
  openshift-image-registry                node-ca-lmnqz                                                      10m (0%)      0 (0%)      10Mi (0%)         0 (0%)         120m
  openshift-ingress-canary                ingress-canary-gn4fs                                               10m (0%)      0 (0%)      20Mi (0%)         0 (0%)         122m
  openshift-ingress                       router-default-6c5c8d967b-vm8qb                                    100m (1%)     0 (0%)      256Mi (1%)        0 (0%)         94m
  openshift-local-storage                 local-block-local-diskmaker-8zkvh                                  0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-local-storage                 local-block-local-provisioner-phdhq                                0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-machine-config-operator       machine-config-daemon-sqjbv                                        40m (0%)      0 (0%)      100Mi (0%)        0 (0%)         98m
  openshift-manila-csi-driver             csi-nodeplugin-nfsplugin-frgkn                                     10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         121m
  openshift-manila-csi-driver             openstack-manila-csi-nodeplugin-v9889                              15m (0%)      0 (0%)      70Mi (0%)         0 (0%)         121m
  openshift-marketplace                   certified-operators-5ghdn                                          10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         94m
  openshift-marketplace                   hco-catalogsource-s5p9d                                            10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         21h
  openshift-marketplace                   ocs-catalogsource-m5974                                            10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         87m
  openshift-marketplace                   redhat-marketplace-qfmcp                                           10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         87m
  openshift-marketplace                   redhat-operators-vhszj                                             10m (0%)      0 (0%)      50Mi (0%)         0 (0%)         94m
  openshift-monitoring                    alertmanager-main-0                                                8m (0%)       0 (0%)      270Mi (1%)        0 (0%)         94m
  openshift-monitoring                    alertmanager-main-1                                                8m (0%)       0 (0%)      270Mi (1%)        0 (0%)         121m
  openshift-monitoring                    kube-state-metrics-6c798c69f5-sdkph                                4m (0%)       0 (0%)      120Mi (0%)        0 (0%)         87m
  openshift-monitoring                    node-exporter-x6lq2                                                9m (0%)       0 (0%)      210Mi (1%)        0 (0%)         121m
  openshift-monitoring                    openshift-state-metrics-6bd7d6f65-b7c9x                            3m (0%)       0 (0%)      190Mi (1%)        0 (0%)         94m
  openshift-monitoring                    prometheus-adapter-6575b658bf-qnxj6                                1m (0%)       0 (0%)      25Mi (0%)         0 (0%)         94m
  openshift-monitoring                    prometheus-adapter-6575b658bf-rzxht                                1m (0%)       0 (0%)      25Mi (0%)         0 (0%)         87m
  openshift-monitoring                    prometheus-k8s-0                                                   76m (1%)      0 (0%)      1204Mi (8%)       0 (0%)         87m
  openshift-monitoring                    prometheus-k8s-1                                                   76m (1%)      0 (0%)      1204Mi (8%)       0 (0%)         121m
  openshift-monitoring                    thanos-querier-565fc8859b-pkzx2                                    9m (0%)       0 (0%)      92Mi (0%)         0 (0%)         94m
  openshift-multus                        multus-4w72h                                                       10m (0%)      0 (0%)      150Mi (1%)        0 (0%)         104m
  openshift-multus                        network-metrics-daemon-nm5zq                                       20m (0%)      0 (0%)      120Mi (0%)        0 (0%)         105m
  openshift-network-diagnostics           network-check-target-4bsnh                                         10m (0%)      0 (0%)      15Mi (0%)         0 (0%)         107m
  openshift-openstack-infra               coredns-ssp04-rvkqg-worker-0-zvfkw                                 100m (1%)     0 (0%)      200Mi (1%)        0 (0%)         23h
  openshift-openstack-infra               keepalived-ssp04-rvkqg-worker-0-zvfkw                              200m (2%)     0 (0%)      400Mi (2%)        0 (0%)         23h
  openshift-openstack-infra               mdns-publisher-ssp04-rvkqg-worker-0-zvfkw                          100m (1%)     0 (0%)      200Mi (1%)        0 (0%)         22h
  openshift-sdn                           ovs-k77kz                                                          15m (0%)      0 (0%)      400Mi (2%)        0 (0%)         103m
  openshift-sdn                           sdn-srksb                                                          110m (1%)     0 (0%)      220Mi (1%)        0 (0%)         107m
  openshift-storage                       csi-cephfsplugin-provisioner-6bc7cbf6f-88vxh                       0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-storage                       csi-cephfsplugin-xjcf6                                             0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-storage                       csi-rbdplugin-8rtwc                                                0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-storage                       csi-rbdplugin-provisioner-7699b8c4b8-vtf9b                         0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-storage                       noobaa-endpoint-7584c5969f-rwn7z                                   0 (0%)        0 (0%)      0 (0%)            0 (0%)         87m
  openshift-storage                       noobaa-operator-5855b5688-cfn67                                    0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-storage                       ocs-metrics-exporter-5d66d5fc59-fbqm6                              0 (0%)        0 (0%)      0 (0%)            0 (0%)         87m
  openshift-storage                       ocs-operator-cd5b866f5-29q72                                       0 (0%)        0 (0%)      0 (0%)            0 (0%)         87m
  openshift-storage                       rook-ceph-crashcollector-ssp04-rvkqg-worker-0-zvfkw-666867qr7vg    0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-storage                       rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-786bdbf7qv4qj    0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-storage                       rook-ceph-mgr-a-5576d7458b-sdsz2                                   0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-storage                       rook-ceph-operator-6cfc658d4b-prp6k                                0 (0%)        0 (0%)      0 (0%)            0 (0%)         94m
  openshift-storage                       rook-ceph-osd-1-5946d84494-rkmbx                                   0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
  openshift-storage                       rook-ceph-rgw-ocs-storagecluster-cephobjectstore-b-8b6b579tmgtf    0 (0%)        0 (0%)      0 (0%)            0 (0%)         21h
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource                         Requests           Limits
  --------                         --------           ------
  cpu                              1350m (18%)        200m (2%)
  memory                           12943622145 (82%)  616870912 (3%)
  ephemeral-storage                0 (0%)             0 (0%)
  hugepages-1Gi                    0 (0%)             0 (0%)
  hugepages-2Mi                    0 (0%)             0 (0%)
  attachable-volumes-cinder        0                  0
  devices.kubevirt.io/kvm          2                  2
  devices.kubevirt.io/tun          2                  2
  devices.kubevirt.io/vhost-net    1                  1
  ovs-cni.network.kubevirt.io/br0  0                  0
Events:                            <none>


===============================================
$ oc describe node ssp04-rvkqg-worker-0-ns68h
Name:               ssp04-rvkqg-worker-0-ns68h
Roles:              worker
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/instance-type=ci.nested.virt.m1.xlarge
                    beta.kubernetes.io/os=linux
                    cluster.ocs.openshift.io/openshift-storage=
                    cpumanager=true
                    failure-domain.beta.kubernetes.io/zone=nova
                    feature.node.kubernetes.io/cpu-feature-aes=true
                    feature.node.kubernetes.io/cpu-feature-avx=true
                    feature.node.kubernetes.io/cpu-feature-avx2=true
                    feature.node.kubernetes.io/cpu-feature-bmi1=true
                    feature.node.kubernetes.io/cpu-feature-bmi2=true
                    feature.node.kubernetes.io/cpu-feature-erms=true
                    feature.node.kubernetes.io/cpu-feature-f16c=true
                    feature.node.kubernetes.io/cpu-feature-fma=true
                    feature.node.kubernetes.io/cpu-feature-fsgsbase=true
                    feature.node.kubernetes.io/cpu-feature-invpcid=true
                    feature.node.kubernetes.io/cpu-feature-movbe=true
                    feature.node.kubernetes.io/cpu-feature-pcid=true
                    feature.node.kubernetes.io/cpu-feature-pclmuldq=true
                    feature.node.kubernetes.io/cpu-feature-popcnt=true
                    feature.node.kubernetes.io/cpu-feature-rdrand=true
                    feature.node.kubernetes.io/cpu-feature-rdtscp=true
                    feature.node.kubernetes.io/cpu-feature-smep=true
                    feature.node.kubernetes.io/cpu-feature-spec-ctrl=true
                    feature.node.kubernetes.io/cpu-feature-sse4.2=true
                    feature.node.kubernetes.io/cpu-feature-svm=true
                    feature.node.kubernetes.io/cpu-feature-tsc-deadline=true
                    feature.node.kubernetes.io/cpu-feature-vme=true
                    feature.node.kubernetes.io/cpu-feature-x2apic=true
                    feature.node.kubernetes.io/cpu-feature-xsave=true
                    feature.node.kubernetes.io/cpu-model-Haswell-noTSX=true
                    feature.node.kubernetes.io/cpu-model-Haswell-noTSX-IBRS=true
                    feature.node.kubernetes.io/cpu-model-IvyBridge=true
                    feature.node.kubernetes.io/cpu-model-IvyBridge-IBRS=true
                    feature.node.kubernetes.io/cpu-model-Nehalem=true
                    feature.node.kubernetes.io/cpu-model-Nehalem-IBRS=true
                    feature.node.kubernetes.io/cpu-model-Opteron_G1=true
                    feature.node.kubernetes.io/cpu-model-Opteron_G2=true
                    feature.node.kubernetes.io/cpu-model-Penryn=true
                    feature.node.kubernetes.io/cpu-model-SandyBridge=true
                    feature.node.kubernetes.io/cpu-model-SandyBridge-IBRS=true
                    feature.node.kubernetes.io/cpu-model-Westmere=true
                    feature.node.kubernetes.io/cpu-model-Westmere-IBRS=true
                    feature.node.kubernetes.io/cpu-model-kvm32=true
                    feature.node.kubernetes.io/cpu-model-kvm64=true
                    feature.node.kubernetes.io/cpu-model-qemu32=true
                    feature.node.kubernetes.io/cpu-model-qemu64=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-base=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-frequencies=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-ipi=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-reenlightenment=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-reset=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-runtime=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synic=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synic2=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synictimer=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-time=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-tlbflush=true
                    feature.node.kubernetes.io/kvm-info-cap-hyperv-vpindex=true
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=ssp04-rvkqg-worker-0-ns68h
                    kubernetes.io/os=linux
                    kubevirt.io/schedulable=true
                    node-role.kubernetes.io/worker=
                    node.kubernetes.io/instance-type=ci.nested.virt.m1.xlarge
                    node.openshift.io/os_id=rhcos
                    topology.cinder.csi.openstack.org/zone=nova
                    topology.kubernetes.io/zone=nova
                    topology.rook.io/rack=rack1
Annotations:        csi.volume.kubernetes.io/nodeid:
                      {"cinder.csi.openstack.org":"b171cd7d-095d-448c-b602-19d849256e73","manila.csi.openstack.org":"ssp04-rvkqg-worker-0-ns68h","openshift-stor...
                    kubevirt.io/heartbeat: 2021-02-11T16:13:43Z
                    machine.openshift.io/machine: openshift-machine-api/ssp04-rvkqg-worker-0-ns68h
                    machineconfiguration.openshift.io/currentConfig: rendered-worker-e32d0c5fbb975fcd1cd2ff67ba0f0965
                    machineconfiguration.openshift.io/desiredConfig: rendered-worker-5cfc02ef8ec6a59bf10e98d7ebc772b6
                    machineconfiguration.openshift.io/reason:
                      failed to drain node (5 tries): timed out waiting for the condition: error when evicting pods/"rook-ceph-osd-2-5786c9fb5f-vf97v" -n "opens...
                    machineconfiguration.openshift.io/state: Degraded
                    node-labeller-feature.node.kubernetes.io/cpu-feature-aes: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-avx: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-avx2: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-bmi1: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-bmi2: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-erms: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-f16c: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-fma: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-fsgsbase: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-invpcid: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-movbe: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-pcid: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-pclmuldq: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-popcnt: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-rdrand: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-rdtscp: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-smep: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-spec-ctrl: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-sse4.2: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-svm: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-tsc-deadline: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-vme: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-x2apic: true
                    node-labeller-feature.node.kubernetes.io/cpu-feature-xsave: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Haswell-noTSX: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Haswell-noTSX-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-IvyBridge: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-IvyBridge-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Nehalem: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Nehalem-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Opteron_G1: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Opteron_G2: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Penryn: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-SandyBridge: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-SandyBridge-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Westmere: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-Westmere-IBRS: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-kvm32: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-kvm64: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-qemu32: true
                    node-labeller-feature.node.kubernetes.io/cpu-model-qemu64: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-base: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-frequencies: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-ipi: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-reenlightenment: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-reset: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-runtime: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synic: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synic2: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synictimer: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-time: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-tlbflush: true
                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-vpindex: true
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 10 Feb 2021 17:12:34 +0000
Taints:             node.kubernetes.io/unschedulable:NoSchedule
Unschedulable:      true
Lease:
  HolderIdentity:  ssp04-rvkqg-worker-0-ns68h
  AcquireTime:     <unset>
  RenewTime:       Thu, 11 Feb 2021 16:13:59 +0000
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Thu, 11 Feb 2021 16:11:19 +0000   Wed, 10 Feb 2021 18:47:25 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Thu, 11 Feb 2021 16:11:19 +0000   Wed, 10 Feb 2021 18:47:25 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Thu, 11 Feb 2021 16:11:19 +0000   Wed, 10 Feb 2021 18:47:25 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Thu, 11 Feb 2021 16:11:19 +0000   Wed, 10 Feb 2021 18:47:25 +0000   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  192.168.0.70
  Hostname:    ssp04-rvkqg-worker-0-ns68h
Capacity:
  attachable-volumes-cinder:        256
  cpu:                              8
  devices.kubevirt.io/kvm:          110
  devices.kubevirt.io/tun:          110
  devices.kubevirt.io/vhost-net:    110
  ephemeral-storage:                41391084Ki
  hugepages-1Gi:                    0
  hugepages-2Mi:                    0
  memory:                           16418260Ki
  ovs-cni.network.kubevirt.io/br0:  1k
  pods:                             250
Allocatable:
  attachable-volumes-cinder:        256
  cpu:                              7500m
  devices.kubevirt.io/kvm:          110
  devices.kubevirt.io/tun:          110
  devices.kubevirt.io/vhost-net:    110
  ephemeral-storage:                37072281128
  hugepages-1Gi:                    0
  hugepages-2Mi:                    0
  memory:                           15267284Ki
  ovs-cni.network.kubevirt.io/br0:  1k
  pods:                             250
System Info:
  Machine ID:                             b171cd7d095d448cb60219d849256e73
  System UUID:                            b171cd7d-095d-448c-b602-19d849256e73
  Boot ID:                                a28163e5-e265-4b5d-99e4-f6986b939664
  Kernel Version:                         4.18.0-193.41.1.el8_2.x86_64
  OS Image:                               Red Hat Enterprise Linux CoreOS 46.82.202101301821-0 (Ootpa)
  Operating System:                       linux
  Architecture:                           amd64
  Container Runtime Version:              cri-o://1.19.1-7.rhaos4.6.git6377f68.el8
  Kubelet Version:                        v1.19.0+e49167a
  Kube-Proxy Version:                     v1.19.0+e49167a
ProviderID:                               openstack:///b171cd7d-095d-448c-b602-19d849256e73
Non-terminated Pods:                      (29 in total)
  Namespace                               Name                                         CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                               ----                                         ------------  ----------  ---------------  -------------  ---
  openshift-cluster-csi-drivers           openstack-cinder-csi-driver-node-d62s2       20m (0%)      0 (0%)      100Mi (0%)       0 (0%)         122m
  openshift-cluster-node-tuning-operator  tuned-ncbs8                                  10m (0%)      0 (0%)      50Mi (0%)        0 (0%)         121m
  openshift-cnv                           bridge-marker-jpjh7                          0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-cnv                           hostpath-provisioner-5pmkg                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-cnv                           kube-cni-linux-bridge-plugin-7vd5p           60m (0%)      0 (0%)      30Mi (0%)        0 (0%)         21h
  openshift-cnv                           kubevirt-node-labeller-wx6s2                 0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-cnv                           nmstate-handler-knshm                        0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-cnv                           ovs-cni-amd64-z4qgg                          0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-cnv                           virt-handler-v9m9k                           0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-dns                           dns-default-lnh4c                            65m (0%)      0 (0%)      110Mi (0%)       512Mi (3%)     99m
  openshift-image-registry                node-ca-vs7cs                                10m (0%)      0 (0%)      10Mi (0%)        0 (0%)         121m
  openshift-ingress-canary                ingress-canary-fxkr9                         10m (0%)      0 (0%)      20Mi (0%)        0 (0%)         122m
  openshift-local-storage                 local-block-local-diskmaker-lsfjd            0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-local-storage                 local-block-local-provisioner-6zhx6          0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-machine-config-operator       machine-config-daemon-fbtcb                  40m (0%)      0 (0%)      100Mi (0%)       0 (0%)         97m
  openshift-manila-csi-driver             csi-nodeplugin-nfsplugin-kn4rg               10m (0%)      0 (0%)      50Mi (0%)        0 (0%)         121m
  openshift-manila-csi-driver             openstack-manila-csi-nodeplugin-gf45p        15m (0%)      0 (0%)      70Mi (0%)        0 (0%)         122m
  openshift-monitoring                    node-exporter-pdqdr                          9m (0%)       0 (0%)      210Mi (1%)       0 (0%)         122m
  openshift-multus                        multus-v52tr                                 10m (0%)      0 (0%)      150Mi (1%)       0 (0%)         102m
  openshift-multus                        network-metrics-daemon-fnrvf                 20m (0%)      0 (0%)      120Mi (0%)       0 (0%)         106m
  openshift-network-diagnostics           network-check-target-lsmpc                   10m (0%)      0 (0%)      15Mi (0%)        0 (0%)         108m
  openshift-openstack-infra               coredns-ssp04-rvkqg-worker-0-ns68h           100m (1%)     0 (0%)      200Mi (1%)       0 (0%)         22h
  openshift-openstack-infra               keepalived-ssp04-rvkqg-worker-0-ns68h        200m (2%)     0 (0%)      400Mi (2%)       0 (0%)         23h
  openshift-openstack-infra               mdns-publisher-ssp04-rvkqg-worker-0-ns68h    100m (1%)     0 (0%)      200Mi (1%)       0 (0%)         22h
  openshift-sdn                           ovs-7mqt8                                    15m (0%)      0 (0%)      400Mi (2%)       0 (0%)         107m
  openshift-sdn                           sdn-kzs6k                                    110m (1%)     0 (0%)      220Mi (1%)       0 (0%)         106m
  openshift-storage                       csi-cephfsplugin-4szch                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-storage                       csi-rbdplugin-tc7rf                          0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
  openshift-storage                       rook-ceph-osd-2-5786c9fb5f-vf97v             0 (0%)        0 (0%)      0 (0%)           0 (0%)         21h
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource                         Requests      Limits
  --------                         --------      ------
  cpu                              814m (10%)    0 (0%)
  memory                           2455Mi (16%)  512Mi (3%)
  ephemeral-storage                0 (0%)        0 (0%)
  hugepages-1Gi                    0 (0%)        0 (0%)
  hugepages-2Mi                    0 (0%)        0 (0%)
  attachable-volumes-cinder        0             0
  devices.kubevirt.io/kvm          0             0
  devices.kubevirt.io/tun          0             0
  devices.kubevirt.io/vhost-net    0             0
  ovs-cni.network.kubevirt.io/br0  0             0
Events:
  Type    Reason              Age                From     Message
  ----    ------              ----               ----     -------
  Normal  NodeNotSchedulable  88m (x3 over 21h)  kubelet  Node ssp04-rvkqg-worker-0-ns68h status is now: NodeNotSchedulable


===============================================
$ oc get vm rhel8-nfs -oyaml
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  annotations:
    kubevirt.io/latest-observed-api-version: v1alpha3
    kubevirt.io/storage-observed-api-version: v1alpha3
  creationTimestamp: "2021-02-11T13:04:48Z"
  generation: 1
  labels:
    app: rhel8-nfs
    vm.kubevirt.io/template: rhel8-server-tiny-v0.11.3
    vm.kubevirt.io/template.revision: "1"
    vm.kubevirt.io/template.version: v0.12.4
  managedFields:
  - apiVersion: kubevirt.io/v1alpha3
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:app: {}
          f:vm.kubevirt.io/template: {}
          f:vm.kubevirt.io/template.revision: {}
          f:vm.kubevirt.io/template.version: {}
      f:spec:
        .: {}
        f:runStrategy: {}
        f:template:
          .: {}
          f:metadata:
            .: {}
            f:labels:
              .: {}
              f:kubevirt.io/domain: {}
              f:kubevirt.io/size: {}
          f:spec:
            .: {}
            f:domain:
              .: {}
              f:cpu:
                .: {}
                f:cores: {}
                f:sockets: {}
                f:threads: {}
              f:devices:
                .: {}
                f:disks: {}
                f:interfaces: {}
                f:networkInterfaceMultiqueue: {}
                f:rng: {}
              f:machine:
                .: {}
                f:type: {}
              f:resources:
                .: {}
                f:requests:
                  .: {}
                  f:memory: {}
            f:evictionStrategy: {}
            f:networks: {}
            f:terminationGracePeriodSeconds: {}
            f:volumes: {}
    manager: kubectl-create
    operation: Update
    time: "2021-02-11T13:04:48Z"
  - apiVersion: kubevirt.io/v1alpha3
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:conditions: {}
        f:created: {}
    manager: virt-api
    operation: Update
    time: "2021-02-11T13:17:48Z"
  - apiVersion: kubevirt.io/v1alpha3
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubevirt.io/latest-observed-api-version: {}
          f:kubevirt.io/storage-observed-api-version: {}
      f:status:
        f:conditions: {}
        f:created: {}
    manager: virt-controller
    operation: Update
    time: "2021-02-11T13:17:56Z"
  name: rhel8-nfs
  namespace: b4-upgarde
  resourceVersion: "1053441"
  selfLink: /apis/kubevirt.io/v1alpha3/namespaces/b4-upgarde/virtualmachines/rhel8-nfs
  uid: 96660cdf-d2f0-44a4-b814-fcddedb41596
spec:
  runStrategy: Manual
  template:
    metadata:
      creationTimestamp: null
      labels:
        kubevirt.io/domain: rhel8-nfs
        kubevirt.io/size: tiny
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 1
          threads: 1
        devices:
          disks:
          - disk:
              bus: virtio
            name: rootdisk
          - disk:
              bus: virtio
            name: cloudinitdisk
          interfaces:
          - masquerade: {}
            name: default
          networkInterfaceMultiqueue: true
          rng: {}
        machine:
          type: pc-q35-rhel8.2.0
        resources:
          requests:
            memory: 1536Mi
      evictionStrategy: LiveMigrate
      networks:
      - name: default
        pod: {}
      terminationGracePeriodSeconds: 180
      volumes:
      - name: rootdisk
        persistentVolumeClaim:
          claimName: rhel8
      - cloudInitNoCloud:
          userData: |-
            #cloud-config
            user: cloud-user
            password: redhat
            chpasswd: { expire: False }
        name: cloudinitdisk
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2021-02-11T13:17:53Z"
    status: "True"
    type: Ready
  created: true

Comment 1 Peter Hunt 2021-02-11 17:05:06 UTC
what's the output of `oc adm node-logs ssp04-rvkqg-worker-0-zvfkw`

Comment 2 Ruth Netser 2021-02-11 17:53:52 UTC
$ oc adm node-logs ssp04-rvkqg-worker-0-zvfkw
error: error trying to reach service: dial tcp 192.168.1.21:10250: i/o timeout

Comment 3 Peter Hunt 2021-02-11 18:08:46 UTC
are you able to access via ssh? if not, you may need to reboot the node via a cloud console to access. I believe we'll need the node logs to make any forward progress

Comment 4 Peter Hunt 2021-02-11 21:52:37 UTC
this looks like a DNS issue:
running `ssh core@ssp04-rvkqg-worker-0-zvfkw` fails but running `ssh core.1.21` (IP address gotten through `oc get nodes -o wide`) succeeds. On the node, the kubelet is complaining a bunch about
```
Feb 11 21:46:50 ssp04-rvkqg-worker-0-zvfkw hyperkube[2292]: E0211 21:46:50.878835    2292 kubelet.go:2190] node "ssp04-rvkqg-worker-0-zvfkw" not found
```
and trying with ping from another host shows:
```
[cnv-qe-jenkins@ssp04-rvkqg-executor ~]$ ping 192.168.1.21
PING 192.168.1.21 (192.168.1.21) 56(84) bytes of data.
64 bytes from 192.168.1.21: icmp_seq=1 ttl=64 time=1.24 ms
64 bytes from 192.168.1.21: icmp_seq=2 ttl=64 time=0.665 ms
64 bytes from 192.168.1.21: icmp_seq=3 ttl=64 time=0.595 ms
64 bytes from 192.168.1.21: icmp_seq=4 ttl=64 time=0.714 ms
64 bytes from 192.168.1.21: icmp_seq=5 ttl=64 time=0.654 ms
64 bytes from 192.168.1.21: icmp_seq=6 ttl=64 time=0.798 ms
64 bytes from 192.168.1.21: icmp_seq=7 ttl=64 time=0.658 ms
^C
--- 192.168.1.21 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 111ms
rtt min/avg/max/mdev = 0.595/0.761/1.244/0.206 ms
[cnv-qe-jenkins@ssp04-rvkqg-executor ~]$ ping ssp04-rvkqg-worker-0-zvfkw
ping: ssp04-rvkqg-worker-0-zvfkw: Name or service not known
```

Comment 5 Ruth Netser 2021-02-14 07:25:27 UTC
From within the cluster it works, from one of the masters:

[core@ssp04-rvkqg-master-0 ~]$ ping ssp04-rvkqg-worker-0-zvfkw
PING ssp04-rvkqg-worker-0-zvfkw.ssp04.cnv-qe.rhcloud.com (192.168.1.21) 56(84) bytes of data.
64 bytes from host-192-168-1-21.openstacklocal (192.168.1.21): icmp_seq=1 ttl=64 time=1.45 ms
64 bytes from host-192-168-1-21.openstacklocal (192.168.1.21): icmp_seq=2 ttl=64 time=0.928 ms
64 bytes from host-192-168-1-21.openstacklocal (192.168.1.21): icmp_seq=3 ttl=64 time=0.615 ms
64 bytes from host-192-168-1-21.openstacklocal (192.168.1.21): icmp_seq=4 ttl=64 time=0.609 ms
^C
--- ssp04-rvkqg-worker-0-zvfkw.ssp04.cnv-qe.rhcloud.com ping statistics ---


(the access to the cluster is done via a separate vm, which does not have DNS)

Comment 6 Ryan Phillips 2021-02-16 14:49:05 UTC

*** This bug has been marked as a duplicate of bug 1913532 ***


Note You need to log in before you can comment on or make changes to this bug.