Attachment 1683319 Details for Bug 1822875 – Added detailed info in an attachment file

Added detailed info in an attachment file

Bug_1822875_Node_info.txt (text/plain), 20.55 KB, created by Kedar Bidarkar on 2020-04-30 14:14:38 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Kedar Bidarkar

Created: 2020-04-30 14:14:38 UTC

Size: 20.55 KB

patch

obsolete

>Adding all the steps performed to configure hugePages.
>
>Configure HugePages for the Worker Node:
>------------------------
>[kbidarka@kbidarka-host hugepages]$ ls
>50-kargs-1g-hugepages.yaml
>[kbidarka@kbidarka-host hugepages]$ cat 50-kargs-1g-hugepages.yaml 
>apiVersion: machineconfiguration.openshift.io/v1
>kind: MachineConfig
>metadata:
>    labels:
>        machineconfiguration.openshift.io/role: worker
>    name: 50-kargs-1g-hugepages
>spec:
>    kernelArguments:
>        - default_hugepagesz=1Gi
>        - hugepagesz=1Gi
>        - hugepages=4
>
>oc apply -f 50-kargs-1g-hugepages.yaml
>
>
>
>Nodes rebooted and got back in Ready state.
>
>
>[kbidarka@kbidarka-host osdc]$ oc get nodes 
>NAME                              STATUS   ROLES    AGE   VERSION
>kbidarka-b33-mlkgs-master-0       Ready    master   15h   v1.17.1
>kbidarka-b33-mlkgs-master-1       Ready    master   15h   v1.17.1
>kbidarka-b33-mlkgs-master-2       Ready    master   15h   v1.17.1
>kbidarka-b33-mlkgs-worker-c6hgj   Ready    worker   15h   v1.17.1
>kbidarka-b33-mlkgs-worker-tr88n   Ready    worker   15h   v1.17.1
>kbidarka-b33-mlkgs-worker-z6z49   Ready    worker   15h   v1.17.1
>
>
>[kbidarka@kbidarka-host osdc]$ oc debug node/kbidarka-b33-mlkgs-worker-c6hgj
>Starting pod/kbidarka-b33-mlkgs-worker-c6hgj-debug ...
>To use host binaries, run `chroot /host`
>Pod IP: 192.168.0.14
>If you don't see a command prompt, try pressing enter.
>sh-4.2# chroot /host
>sh-4.4# cat /proc/cmdline 
>BOOT_IMAGE=(hd0,gpt1)/ostree/rhcos-ec6f097d71f19d2713779aeb6d2296122dec138ffa31e2fd0a15a0f39e39cada/vmlinuz-4.18.0-147.8.1.el8_1.x86_64 rhcos.root=crypt_rootfs console=tty0 console=ttyS0,115200n8 rd.luks.options=discard ostree=/ostree/boot.1/rhcos/ec6f097d71f19d2713779aeb6d2296122dec138ffa31e2fd0a15a0f39e39cada/0 ignition.platform.id=openstack default_hugepagesz=1Gi hugepagesz=1Gi hugepages=4
>sh-4.4# free -h 
>              total        used        free      shared  buff/cache   available
>Mem:           23Gi       5.7Gi        14Gi       6.0Mi       2.9Gi        17Gi
>Swap:            0B          0B          0B
>
>
>
>sh-4.4# cat /proc/meminfo | grep -i huge 
>AnonHugePages:    475136 kB
>ShmemHugePages:        0 kB
>HugePages_Total:       4
>HugePages_Free:        0
>HugePages_Rsvd:        0
>HugePages_Surp:        0
>Hugepagesize:    1048576 kB
>Hugetlb:         4194304 kB
>
>
>[kbidarka@kbidarka-host osdc]$ oc describe node kbidarka-b33-mlkgs-worker-c6hgj
>
>Capacity:
>  attachable-volumes-cinder:        256
>  cpu:                              12
>  devices.kubevirt.io/kvm:          110
>  devices.kubevirt.io/tun:          110
>  devices.kubevirt.io/vhost-net:    110
>  ephemeral-storage:                83334124Ki
>  hugepages-1Gi:                    4Gi
>  hugepages-2Mi:                    0
>  memory:                           24677332Ki
>  ovs-cni.network.kubevirt.io/br0:  1k
>  pods:                             250
>Allocatable:
>  attachable-volumes-cinder:        256
>  cpu:                              11500m
>  devices.kubevirt.io/kvm:          110
>  devices.kubevirt.io/tun:          110
>  devices.kubevirt.io/vhost-net:    110
>  ephemeral-storage:                75726986728
>  hugepages-1Gi:                    4Gi
>  hugepages-2Mi:                    0
>  memory:                           19332052Ki
>  ovs-cni.network.kubevirt.io/br0:  1k
>  pods:                             250
>
>
>
>[kbidarka@kbidarka-host osdc]$ oc get nodes 
>NAME                              STATUS   ROLES    AGE   VERSION
>kbidarka-b33-mlkgs-master-0       Ready    master   15h   v1.17.1
>kbidarka-b33-mlkgs-master-1       Ready    master   15h   v1.17.1
>kbidarka-b33-mlkgs-master-2       Ready    master   15h   v1.17.1
>kbidarka-b33-mlkgs-worker-c6hgj   Ready    worker   15h   v1.17.1
>kbidarka-b33-mlkgs-worker-tr88n   Ready    worker   15h   v1.17.1
>kbidarka-b33-mlkgs-worker-z6z49   Ready    worker   15h   v1.17.1
>[kbidarka@kbidarka-host osdc]$ oc describe node kbidarka-b33-mlkgs-worker-c6hgj
>Name:               kbidarka-b33-mlkgs-worker-c6hgj
>Roles:              worker
>Labels:             beta.kubernetes.io/arch=amd64
>                    beta.kubernetes.io/instance-type=ci.nested.m1.large.xdisk.xmem
>                    beta.kubernetes.io/os=linux
>                    cpumanager=true
>                    failure-domain.beta.kubernetes.io/zone=nova
>                    feature.node.kubernetes.io/cpu-feature-aes=true
>                    feature.node.kubernetes.io/cpu-feature-avx=true
>                    feature.node.kubernetes.io/cpu-feature-avx2=true
>                    feature.node.kubernetes.io/cpu-feature-bmi1=true
>                    feature.node.kubernetes.io/cpu-feature-bmi2=true
>                    feature.node.kubernetes.io/cpu-feature-erms=true
>                    feature.node.kubernetes.io/cpu-feature-f16c=true
>                    feature.node.kubernetes.io/cpu-feature-fma=true
>                    feature.node.kubernetes.io/cpu-feature-fsgsbase=true
>                    feature.node.kubernetes.io/cpu-feature-invpcid=true
>                    feature.node.kubernetes.io/cpu-feature-movbe=true
>                    feature.node.kubernetes.io/cpu-feature-pcid=true
>                    feature.node.kubernetes.io/cpu-feature-pclmuldq=true
>                    feature.node.kubernetes.io/cpu-feature-popcnt=true
>                    feature.node.kubernetes.io/cpu-feature-rdrand=true
>                    feature.node.kubernetes.io/cpu-feature-rdtscp=true
>                    feature.node.kubernetes.io/cpu-feature-smep=true
>                    feature.node.kubernetes.io/cpu-feature-spec-ctrl=true
>                    feature.node.kubernetes.io/cpu-feature-sse4.2=true
>                    feature.node.kubernetes.io/cpu-feature-svm=true
>                    feature.node.kubernetes.io/cpu-feature-tsc-deadline=true
>                    feature.node.kubernetes.io/cpu-feature-vme=true
>                    feature.node.kubernetes.io/cpu-feature-x2apic=true
>                    feature.node.kubernetes.io/cpu-feature-xsave=true
>                    feature.node.kubernetes.io/cpu-model-Haswell-noTSX=true
>                    feature.node.kubernetes.io/cpu-model-Haswell-noTSX-IBRS=true
>                    feature.node.kubernetes.io/cpu-model-IvyBridge=true
>                    feature.node.kubernetes.io/cpu-model-IvyBridge-IBRS=true
>                    feature.node.kubernetes.io/cpu-model-Nehalem=true
>                    feature.node.kubernetes.io/cpu-model-Nehalem-IBRS=true
>                    feature.node.kubernetes.io/cpu-model-Opteron_G1=true
>                    feature.node.kubernetes.io/cpu-model-Opteron_G2=true
>                    feature.node.kubernetes.io/cpu-model-Penryn=true
>                    feature.node.kubernetes.io/cpu-model-SandyBridge=true
>                    feature.node.kubernetes.io/cpu-model-SandyBridge-IBRS=true
>                    feature.node.kubernetes.io/cpu-model-Westmere=true
>                    feature.node.kubernetes.io/cpu-model-Westmere-IBRS=true
>                    feature.node.kubernetes.io/cpu-model-kvm32=true
>                    feature.node.kubernetes.io/cpu-model-kvm64=true
>                    feature.node.kubernetes.io/cpu-model-qemu32=true
>                    feature.node.kubernetes.io/cpu-model-qemu64=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-base=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-frequencies=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-ipi=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-reenlightenment=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-reset=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-runtime=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synic=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synic2=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-synictimer=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-time=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-tlbflush=true
>                    feature.node.kubernetes.io/kvm-info-cap-hyperv-vpindex=true
>                    kubernetes.io/arch=amd64
>                    kubernetes.io/hostname=kbidarka-b33-mlkgs-worker-c6hgj
>                    kubernetes.io/os=linux
>                    kubevirt.io/schedulable=true
>                    node-role.kubernetes.io/worker=
>                    node.kubernetes.io/instance-type=ci.nested.m1.large.xdisk.xmem
>                    node.openshift.io/os_id=rhcos
>                    topology.kubernetes.io/zone=nova
>Annotations:        kubevirt.io/heartbeat: 2020-04-30T13:52:01Z
>                    machine.openshift.io/machine: openshift-machine-api/kbidarka-b33-mlkgs-worker-c6hgj
>                    machineconfiguration.openshift.io/currentConfig: rendered-worker-c378e26244326fb9e2aebe4c9961f3c6
>                    machineconfiguration.openshift.io/desiredConfig: rendered-worker-c378e26244326fb9e2aebe4c9961f3c6
>                    machineconfiguration.openshift.io/reason: 
>                    machineconfiguration.openshift.io/state: Done
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-aes: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-avx: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-avx2: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-bmi1: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-bmi2: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-erms: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-f16c: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-fma: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-fsgsbase: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-invpcid: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-movbe: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-pcid: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-pclmuldq: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-popcnt: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-rdrand: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-rdtscp: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-smep: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-spec-ctrl: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-sse4.2: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-svm: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-tsc-deadline: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-vme: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-x2apic: true
>                    node-labeller-feature.node.kubernetes.io/cpu-feature-xsave: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Haswell-noTSX: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Haswell-noTSX-IBRS: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-IvyBridge: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-IvyBridge-IBRS: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Nehalem: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Nehalem-IBRS: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Opteron_G1: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Opteron_G2: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Penryn: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-SandyBridge: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-SandyBridge-IBRS: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Westmere: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-Westmere-IBRS: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-kvm32: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-kvm64: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-qemu32: true
>                    node-labeller-feature.node.kubernetes.io/cpu-model-qemu64: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-base: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-frequencies: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-ipi: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-reenlightenment: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-reset: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-runtime: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synic: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synic2: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-synictimer: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-time: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-tlbflush: true
>                    node-labeller-feature.node.kubernetes.io/kvm-info-cap-hyperv-vpindex: true
>                    volumes.kubernetes.io/controller-managed-attach-detach: true
>CreationTimestamp:  Thu, 30 Apr 2020 03:53:42 +0530
>Taints:             <none>
>Unschedulable:      false
>Lease:
>  HolderIdentity:  kbidarka-b33-mlkgs-worker-c6hgj
>  AcquireTime:     <unset>
>  RenewTime:       Thu, 30 Apr 2020 19:23:36 +0530
>Conditions:
>  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
>  ----             ------  -----------------                 ------------------                ------                       -------
>  MemoryPressure   False   Thu, 30 Apr 2020 19:21:12 +0530   Thu, 30 Apr 2020 16:01:04 +0530   KubeletHasSufficientMemory   kubelet has sufficient memory available
>  DiskPressure     False   Thu, 30 Apr 2020 19:21:12 +0530   Thu, 30 Apr 2020 16:01:04 +0530   KubeletHasNoDiskPressure     kubelet has no disk pressure
>  PIDPressure      False   Thu, 30 Apr 2020 19:21:12 +0530   Thu, 30 Apr 2020 16:01:04 +0530   KubeletHasSufficientPID      kubelet has sufficient PID available
>  Ready            True    Thu, 30 Apr 2020 19:21:12 +0530   Thu, 30 Apr 2020 16:01:14 +0530   KubeletReady                 kubelet is posting ready status
>Addresses:
>  Hostname:    kbidarka-b33-mlkgs-worker-c6hgj
>  InternalIP:  192.168.0.14
>Capacity:
>  attachable-volumes-cinder:        256
>  cpu:                              12
>  devices.kubevirt.io/kvm:          110
>  devices.kubevirt.io/tun:          110
>  devices.kubevirt.io/vhost-net:    110
>  ephemeral-storage:                83334124Ki
>  hugepages-1Gi:                    4Gi
>  hugepages-2Mi:                    0
>  memory:                           24677332Ki
>  ovs-cni.network.kubevirt.io/br0:  1k
>  pods:                             250
>Allocatable:
>  attachable-volumes-cinder:        256
>  cpu:                              11500m
>  devices.kubevirt.io/kvm:          110
>  devices.kubevirt.io/tun:          110
>  devices.kubevirt.io/vhost-net:    110
>  ephemeral-storage:                75726986728
>  hugepages-1Gi:                    4Gi
>  hugepages-2Mi:                    0
>  memory:                           19332052Ki
>  ovs-cni.network.kubevirt.io/br0:  1k
>  pods:                             250
>System Info:
>  Machine ID:                             acbbf4a3efc24922ad6cbcd1654f1aa3
>  System UUID:                            acbbf4a3-efc2-4922-ad6c-bcd1654f1aa3
>  Boot ID:                                2b9c5c21-fcdb-428c-8662-071f0e4466d3
>  Kernel Version:                         4.18.0-147.8.1.el8_1.x86_64
>  OS Image:                               Red Hat Enterprise Linux CoreOS 44.81.202004260825-0 (Ootpa)
>  Operating System:                       linux
>  Architecture:                           amd64
>  Container Runtime Version:              cri-o://1.17.4-8.dev.rhaos4.4.git5f5c5e4.el8
>  Kubelet Version:                        v1.17.1
>  Kube-Proxy Version:                     v1.17.1
>ProviderID:                               openstack://acbbf4a3-efc2-4922-ad6c-bcd1654f1aa3
>Non-terminated Pods:                      (20 in total)
>  Namespace                               Name                                              CPU Requests  CPU Limits  Memory Requests  Memory Limits   AGE
>  ---------                               ----                                              ------------  ----------  ---------------  -------------   ---
>  kbidarka                                virt-launcher-vm-rhel7-hugepages-8ldxp            1 (8%)        1 (8%)      169943041 (0%)   169943041 (0%)  64m
>  openshift-cluster-node-tuning-operator  tuned-m47j8                                       10m (0%)      0 (0%)      50Mi (0%)        0 (0%)          15h
>  openshift-cnv                           bridge-marker-2hbj8                               100m (0%)     0 (0%)      40Mi (0%)        0 (0%)          14h
>  openshift-cnv                           hostpath-provisioner-8hhtv                        0 (0%)        0 (0%)      0 (0%)           0 (0%)          14h
>  openshift-cnv                           kube-cni-linux-bridge-plugin-4ljnz                60m (0%)      0 (0%)      30Mi (0%)        0 (0%)          14h
>  openshift-cnv                           kubevirt-node-labeller-b5bqt                      0 (0%)        0 (0%)      0 (0%)           0 (0%)          14h
>  openshift-cnv                           nmstate-handler-worker-qnz44                      200m (1%)     0 (0%)      120Mi (0%)       0 (0%)          14h
>  openshift-cnv                           ovs-cni-amd64-kv8st                               160m (1%)     0 (0%)      70Mi (0%)        0 (0%)          14h
>  openshift-cnv                           virt-handler-l67dq                                0 (0%)        0 (0%)      0 (0%)           0 (0%)          14h
>  openshift-dns                           dns-default-cp84w                                 110m (0%)     0 (0%)      70Mi (0%)        512Mi (2%)      15h
>  openshift-image-registry                node-ca-7f6mg                                     10m (0%)      0 (0%)      10Mi (0%)        0 (0%)          15h
>  openshift-machine-config-operator       machine-config-daemon-dspvf                       40m (0%)      0 (0%)      100Mi (0%)       0 (0%)          15h
>  openshift-marketplace                   community-operators-6b9dc87959-fb7n9              10m (0%)      0 (0%)      100Mi (0%)       0 (0%)          33m
>  openshift-monitoring                    node-exporter-29fzn                               9m (0%)       0 (0%)      210Mi (1%)       0 (0%)          15h
>  openshift-multus                        multus-wpmmj                                      10m (0%)      0 (0%)      150Mi (0%)       0 (0%)          15h
>  openshift-openstack-infra               coredns-kbidarka-b33-mlkgs-worker-c6hgj           100m (0%)     0 (0%)      200Mi (1%)       0 (0%)          15h
>  openshift-openstack-infra               keepalived-kbidarka-b33-mlkgs-worker-c6hgj        100m (0%)     0 (0%)      200Mi (1%)       0 (0%)          15h
>  openshift-openstack-infra               mdns-publisher-kbidarka-b33-mlkgs-worker-c6hgj    100m (0%)     0 (0%)      200Mi (1%)       0 (0%)          15h
>  openshift-sdn                           ovs-q8pwt                                         100m (0%)     0 (0%)      400Mi (2%)       0 (0%)          15h
>  openshift-sdn                           sdn-gkxv9                                         100m (0%)     0 (0%)      200Mi (1%)       0 (0%)          15h
>Allocated resources:
>  (Total limits may be over 100 percent, i.e., overcommitted.)
>  Resource                         Requests          Limits
>  --------                         --------          ------
>  cpu                              2219m (19%)       1 (8%)
>  memory                           2424381441 (12%)  706813953 (3%)
>  ephemeral-storage                0 (0%)            0 (0%)
>  attachable-volumes-cinder        0                 0
>  devices.kubevirt.io/kvm          2                 2
>  devices.kubevirt.io/tun          1                 1
>  devices.kubevirt.io/vhost-net    1                 1
>  ovs-cni.network.kubevirt.io/br0  0                 0
>Events:                            <none>
>

Actions: View

Attachments on bug 1822875: 1683319 | 1683321