Bug 1924479
| Summary: | VMI resource guest OS info is not available after a number of minutes | ||
|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Ruth Netser <rnetser> |
| Component: | Virtualization | Assignee: | Antonio Cardace <acardace> |
| Status: | CLOSED ERRATA | QA Contact: | Kedar Bidarkar <kbidarka> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 2.6.0 | CC: | cnv-qe-bugs, kbidarka, oshoval, sgott |
| Target Milestone: | --- | ||
| Target Release: | 4.8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | hco-bundle-registry-container-v4.8.0-347 virt-operator-container-v4.8.0-58 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-07-27 14:23:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Retest this quickly with CNV-v4.8.0, once guest-agent stuff is also fixed. This bug is still seen with the very latest CNV-v4.8.0 builds. PR in review which fixes this https://github.com/kubevirt/kubevirt/pull/5539. Fixed by https://github.com/kubevirt/kubevirt/pull/5428 which was merged 15 mins before I posted my PR. https://github.com/kubevirt/kubevirt/pull/5428 was reverted, https://github.com/kubevirt/kubevirt/pull/5560 merged instead Summary: LGTM,
a) Created 2 VM's, one running since 43+hrs and the other since 24+hrs
b)As seen below even after 43hrs or 24hrs, we can cotninue to get the "Guest OS Info" from "oc describe vmi <vm-name>"
[kbidarka@localhost secureboot]$ oc describe vmi vm2-rhel84 -n default | grep -A 7 "Guest OS Info"
Guest OS Info:
Id: rhel
Kernel Release: 4.18.0-287.el8.dt4.x86_64
Kernel Version: #1 SMP Thu Feb 18 13:31:55 EST 2021
Name: Red Hat Enterprise Linux
Pretty Name: Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Version: 8.4
Version Id: 8.4
[kbidarka@localhost secureboot]$ oc get vmi vm2-rhel84 -n default
NAME AGE PHASE IP NODENAME
vm2-rhel84 43h Running 10.xxx.y.zz node-05.redhat.com
[kbidarka@localhost secureboot]$ oc describe vmi vm2-rhel84 -n oren-test | grep -A 7 "Guest OS Info"
Guest OS Info:
Id: rhel
Kernel Release: 4.18.0-287.el8.dt4.x86_64
Kernel Version: #1 SMP Thu Feb 18 13:31:55 EST 2021
Name: Red Hat Enterprise Linux
Pretty Name: Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Version: 8.4
Version Id: 8.4
[kbidarka@localhost secureboot]$ oc get vmi vm2-rhel84 -n oren-test
NAME AGE PHASE IP NODENAME
vm2-rhel84 24h Running 10.xxx.y.mm node-06.redhat.com
In the past we have issues after Live migration.
Plan is to live-migrate these 2 vm's wait for 24hrs and then move this bug to VERIFIED State.
This is just to double-check, live migration scenario as well.
Will update the Live migration updates shortly.
Summary: "we can fetch Guest OsInfo" Output even after Live Migration.
[kbidarka@localhost migration]$ oc get vmi --all-namespaces
NAMESPACE NAME AGE PHASE IP NODENAME
default vm2-rhel84 43h Running 10.xxx.y.zz node-05.redhat.com
oren-test vm2-rhel84 24h Running 10.xxx.y.mm node-06.redhat.com
[kbidarka@localhost migration]$ cat migration-job2.yaml
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachineInstanceMigration
metadata:
creationTimestamp: null
name: job22
namespace: default
spec:
vmiName: vm2-rhel84
status: {}
[kbidarka@localhost migration]$ cat migration-job2-oren.yaml
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachineInstanceMigration
metadata:
creationTimestamp: null
name: job33
namespace: oren-test
spec:
vmiName: vm2-rhel84
status: {}
[kbidarka@localhost migration]$ oc apply -f migration-job2.yaml
virtualmachineinstancemigration.kubevirt.io/job22 created
[kbidarka@localhost migration]$ oc apply -f migration-job2-oren.yaml
virtualmachineinstancemigration.kubevirt.io/job33 created
[kbidarka@localhost migration]$ oc get vmi --all-namespaces
NAMESPACE NAME AGE PHASE IP NODENAME
default vm2-rhel84 44h Running 10.xxx.y.cc node-07.redhat.com
oren-test vm2-rhel84 24h Running 10.xxx.y.dd node-05.redhat.com
[kbidarka@localhost migration]$ oc describe vmi vm2-rhel84 -n default | grep -A 7 "Guest OS Info"
Guest OS Info:
Id: rhel
Kernel Release: 4.18.0-287.el8.dt4.x86_64
Kernel Version: #1 SMP Thu Feb 18 13:31:55 EST 2021
Name: Red Hat Enterprise Linux
Pretty Name: Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Version: 8.4
Version Id: 8.4
[kbidarka@localhost migration]$ oc describe vmi vm2-rhel84 -n oren-test | grep -A 7 "Guest OS Info"
Guest OS Info:
Id: rhel
Kernel Release: 4.18.0-287.el8.dt4.x86_64
Kernel Version: #1 SMP Thu Feb 18 13:31:55 EST 2021
Name: Red Hat Enterprise Linux
Pretty Name: Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Version: 8.4
Version Id: 8.4
Will update here after 24hrs
[kbidarka@localhost secureboot]$ oc get vmi -n default
NAME AGE PHASE IP NODENAME
vm2-rhel84 2d14h Running kk.xxx.y.zzz node-07.redhat.com
vm2-rhel84-secref 17h Running kk.xxx.y.zyz node-07.redhat.com
[kbidarka@localhost secureboot]$ oc describe vmi vm2-rhel84 -n default | grep -A 10 "Guest OS Info"
Guest OS Info:
Id: rhel
Kernel Release: 4.18.0-287.el8.dt4.x86_64
Kernel Version: #1 SMP Thu Feb 18 13:31:55 EST 2021
Name: Red Hat Enterprise Linux
Pretty Name: Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Version: 8.4
Version Id: 8.4
---
[kbidarka@localhost secureboot]$ oc get vmi -n oren-test
NAME AGE PHASE IP NODENAME
vm2-rhel84 43h Running kk.xxx.y.mmm node-05.redhat.com
[kbidarka@localhost secureboot]$ oc describe vmi vm2-rhel84 -n oren-test | grep -A 10 "Guest OS Info"
Guest OS Info:
Id: rhel
Kernel Release: 4.18.0-287.el8.dt4.x86_64
Kernel Version: #1 SMP Thu Feb 18 13:31:55 EST 2021
Name: Red Hat Enterprise Linux
Pretty Name: Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Version: 8.4
Version Id: 8.4
Summary: "we can continue to fetch Guest OsInfo" Output even after Live Migration and also more than 20+hrs.
Looking at above output info.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Virtualization 4.8.0 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2920 |
Description of problem: When a VM is running, the VMI's resource 'Guest OS Info' is not available after a number of minutes. Gguest agent data provided from virtcl is available however the VMI resource one is not. Version-Release number of selected component (if applicable): 2.6.0-106 How reproducible: 100% Steps to Reproduce: 1. Start a VM 2. Monitor the VMI resource Actual results: When the VMI is running, 'Guest OS Info' is displayed. Guest OS Info: Id: rhel Kernel Release: 4.18.0-240.12.1.el8_3.x86_64 Kernel Version: #1 SMP Wed Jan 13 06:05:15 EST 2021 Name: Red Hat Enterprise Linux Pretty Name: Red Hat Enterprise Linux 8.3 (Ootpa) Version: 8.3 Version Id: 8.3 Interfaces: After a number of minutes, the information is no longer available (guest agnet info is still avialable via virtctl): Type: AgentConnected Guest OS Info: Interfaces: virtctl guestosinfo rhel8-vm { "guestAgentVersion": "4.2.0", "hostname": "rhel8-vm", "os": { "name": "Red Hat Enterprise Linux", "kernelRelease": "4.18.0-240.12.1.el8_3.x86_64", "version": "8.3 (Ootpa)", "prettyName": "Red Hat Enterprise Linux 8.3 (Ootpa)", "versionId": "8.3", "kernelVersion": "#1 SMP Wed Jan 13 06:05:15 EST 2021", "machine": "x86_64", "id": "rhel" }, "timezone": "EST, -18000", "fsInfo": { "disks": [ { "diskName": "vda2", "mountPoint": "/boot/efi", "fileSystemType": "vfat", "usedBytes": 7170048, "totalBytes": 104634368 }, { "diskName": "vda3", "mountPoint": "/", "fileSystemType": "xfs", "usedBytes": 1602134016, "totalBytes": 32094793728 } ] } } Expected results: Guset OS info should be consistent and available both in VMI resource and virtctl. Additional info: oc describe vmi rhel8-vm Name: rhel8-vm Namespace: default Labels: kubevirt.io/domain=rhel8-vm kubevirt.io/nodeName=ssp01-t6h4t-worker-0-z4qqw kubevirt.io/size=tiny Annotations: kubevirt.io/latest-observed-api-version: v1alpha3 kubevirt.io/storage-observed-api-version: v1alpha3 API Version: kubevirt.io/v1alpha3 Kind: VirtualMachineInstance Metadata: Creation Timestamp: 2021-02-01T16:21:33Z Finalizers: foregroundDeleteVirtualMachine Generation: 13 Managed Fields: API Version: kubevirt.io/v1alpha3 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:kubevirt.io/latest-observed-api-version: f:kubevirt.io/storage-observed-api-version: f:labels: .: f:kubevirt.io/domain: f:kubevirt.io/nodeName: f:kubevirt.io/size: f:ownerReferences: f:spec: .: f:domain: .: f:cpu: .: f:cores: f:sockets: f:threads: f:devices: .: f:disks: f:interfaces: f:networkInterfaceMultiqueue: f:rng: f:firmware: .: f:uuid: f:machine: .: f:type: f:resources: .: f:requests: .: f:memory: f:evictionStrategy: f:networks: f:terminationGracePeriodSeconds: f:volumes: f:status: .: f:activePods: .: f:d1d993cb-afe4-40e6-b32f-b8d5a144f001: f:guestOSInfo: f:nodeName: f:qosClass: Manager: virt-controller Operation: Update Time: 2021-02-01T16:21:38Z API Version: kubevirt.io/v1alpha3 Fields Type: FieldsV1 fieldsV1: f:status: f:conditions: f:interfaces: f:migrationMethod: f:phase: f:volumeStatus: Manager: virt-handler Operation: Update Time: 2021-02-01T16:22:13Z Owner References: API Version: kubevirt.io/v1alpha3 Block Owner Deletion: true Controller: true Kind: VirtualMachine Name: rhel8-vm UID: 265981bc-7879-4f7f-9311-a310508e52f3 Resource Version: 3720164 Self Link: /apis/kubevirt.io/v1alpha3/namespaces/default/virtualmachineinstances/rhel8-vm UID: 8e33b9c0-c70b-4616-b0ff-42bc27f80f5f Spec: Domain: Cpu: Cores: 1 Sockets: 1 Threads: 1 Devices: Disks: Disk: Bus: virtio Name: rhel8-vm Disk: Bus: virtio Name: cloudinitdisk Interfaces: Masquerade: Name: default Network Interface Multiqueue: true Rng: Features: Acpi: Enabled: true Firmware: Uuid: ee04d89a-59d6-5a2b-9935-6b9f7ec061fd Machine: Type: pc-q35-rhel8.3.0 Resources: Requests: Cpu: 100m Memory: 1536Mi Eviction Strategy: LiveMigrate Networks: Name: default Pod: Termination Grace Period Seconds: 180 Volumes: Data Volume: Name: rhel8 Name: rhel8-vm Cloud Init No Cloud: User Data: #cloud-config user: cloud-user password: jnrt-awuf-tck8 chpasswd: { expire: False } Name: cloudinitdisk Status: Active Pods: d1d993cb-afe4-40e6-b32f-b8d5a144f001: ssp01-t6h4t-worker-0-z4qqw Conditions: Last Probe Time: <nil> Last Transition Time: <nil> Status: True Type: LiveMigratable Last Probe Time: <nil> Last Transition Time: 2021-02-01T16:21:37Z Status: True Type: Ready Last Probe Time: 2021-02-01T16:22:13Z Last Transition Time: <nil> Status: True Type: AgentConnected Guest OS Info: Interfaces: Interface Name: eth0 Ip Address: 10.131.1.124 Ip Addresses: 10.131.1.124 Mac: 52:54:00:84:0a:cc Name: default Migration Method: BlockMigration Node Name: ssp01-t6h4t-worker-0-z4qqw Phase: Running Qos Class: Burstable Volume Status: Name: cloudinitdisk Target: vdb Name: rhel8-vm Target: vda Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulCreate 2m18s disruptionbudget-controller Created PodDisruptionBudget kubevirt-disruption-budget-qhxv5 Normal SuccessfulCreate 2m18s virtualmachine-controller Created virtual machine pod virt-launcher-rhel8-vm-x8rh9 Normal Started 2m13s virt-handler VirtualMachineInstance started. Normal Created 35s (x14 over 2m13s) virt-handler VirtualMachineInstance defined.