Bug 1976020
Summary: | [RFE][v2v] [upload/download disk/CBT] Failed to attach disk to the VM - disk is OK but image transfer still holds a lock on the disk | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Xiaodai Wang <xiaodwan> | |
Component: | virt-v2v | Assignee: | Virtualization Maintenance <virt-maint> | |
Status: | CLOSED NOTABUG | QA Contact: | Virtualization Bugs <virt-bugs> | |
Severity: | high | Docs Contact: | ||
Priority: | high | |||
Version: | 9.0 | CC: | aefrat, juzhou, kkiwi, lsvaty, mxie, mzhan, nsimsolo, nsoffer, pbar, ptoscano, rjones, tnisan, tyan, tzheng, Yury.Panchenko | |
Target Milestone: | beta | Keywords: | Automation, FutureFeature | |
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | Enhancement | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | 1849861 | |||
: | 1976024 (view as bug list) | Environment: | ||
Last Closed: | 2022-04-20 11:59:14 UTC | Type: | Feature Request | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | 1849861 | |||
Bug Blocks: | 1923178, 1976024 |
Description
Xiaodai Wang
2021-06-25 01:51:31 UTC
I posted patch upstream for discussion: https://listman.redhat.com/archives/libguestfs/2021-July/msg00009.html Fixed upstream in: https://github.com/libguestfs/virt-v2v/commit/79702b28329d15a7485801ed7e915d486fcc0cf4 The issue was reproduced again in virt-v2v-1.45.3-1.el9.x86_64 and 4.4.8.3-0.10.el8ev. https://libvirt-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/v2v-RHEL-9.0-runtest-x86_64-function-convert_from_file-rhel/3/testReport/rhel/convert_from_file/positive_test_linux_input_mode_ova_parse_invalid_line_in_manifest_missing_space_output_mode_rhev_rhv_upload/ The engine log: 2021-08-09 21:20:30,728+08 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-66) [5c0e1348] EVENT_ID: VM_IMPORT_FROM_CONFIGURATION_ATTACH_DISKS_FAILED(175), VM ova_vm_1Qr has been imported from the given configuration but the following disk(s) failed to attach: 1a060938-b201-4a51-95af-6b2ad1f0130b. 2021-08-09 21:20:33,253+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-66) [9ed25544-50c6-44c8-a16a-404bca5d8b4b] EVENT_ID: USER_FAILED_RUN_VM(54), Failed to run VM ova_vm_1Qr due to a failed validation: [Cannot run VM without at least one bootable disk. (In reply to Xiaodai Wang from comment #5) > The issue was reproduced again in virt-v2v-1.45.3-1.el9.x86_64 and > 4.4.8.3-0.10.el8ev. Engine reports FINISHED_SUCCESS status before it release the lock on the disk, this is bug 1923178, which is targeted to RHV 4.4.9. virt-v2v is doing the right thing, waiting on the image transfer status. We cannot do anything if engine reports incorrect status. Here are the relevant events from engine log: 1. Image transfer finish (but it is actually did not finish yet) 2021-08-09 21:20:29,411+08 INFO [org.ovirt.engine.core.bll.storage.disk.image.ImageTransferUpdater] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-64) [a3786a76-07da-4bd8-8927-2d9ebceb4f70] Updating image transfer cd785714-07ca-465 6-bbe4-8542506535b8 (image 1a060938-b201-4a51-95af-6b2ad1f0130b) phase to Finished Success 2. virt-v2v attempt to create the vm 2021-08-09 21:20:30,670+08 INFO [org.ovirt.engine.core.bll.exportimport.ImportVmFromConfigurationCommand] (default task-66) [b2c13365-4b9f-4172-962a-77bc9b106035] Running command: ImportVmFromConfigurationCommand internal: false. Entities affected : ID : f1388cb6-dc42-4e07-afe5-5a84b754c140 Type: ClusterAction group CREATE_VM with role type USER, ID: 00000000-0000-0000-0000-000000000000 Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN ... 2021-08-09 21:20:30,725+08 WARN [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-66) [5c0e1348] Validation of action 'AttachDiskToVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__ATTACH_ACTION_TO,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_LOCKED 3. Image transfer finish and release the lock 2021-08-09 21:20:38,623+08 INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-55) [a3786a76-07da-4bd8-8927-2d9ebceb4f70] Lock freed to object 'EngineLock:{exclusiveLocks='[1a060938-b201-4a51-95af-6b2ad1f0130b=DISK]', sharedLocks='[]'}' 2021-08-09 21:20:38,632+08 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-55) [a3786a76-07da-4bd8-8927-2d9ebceb4f70] EVENT_ID: TRANSFER_IMAGE_SUCCEEDED(1,032), Image Upload with disk ova_vm_1Qr-000 succeeded. I could not reproduce this in my setup (tried about 100 imports). It is possible that your engine is installed on a very old hardware and/or very low memory, so engine is extremely slow? The server didn't have heavy consumption, and only one v2v job was running when it happened. The ova file is also very small, only several megabytes. -rwxr-xr-x. 1 nobody nobody 3.9M Apr 17 2017 ova-images/case84967/invalid_line_in_manifest-missing_space.ova (In reply to Xiaodai Wang from comment #8) > The server didn't have heavy consumption, and only one v2v job was running > when it happened. > The ova file is also very small, only several megabytes. > > -rwxr-xr-x. 1 nobody nobody 3.9M Apr 17 2017 > ova-images/case84967/invalid_line_in_manifest-missing_space.ova Can you reproduce this with a real ova? I can easily reproduce this issue by uploading and downloading empty image (takes few milliseconds per uplaod/download), but I could never reproduce the issue with real images (e.g. 6g fedora image). No, I never saw this issue happened with other real OVA files. Rich - I'll move this to your capable hands to resolve. The bug is currently in POST w/o an ITR, but since you know the v2v release process better than me, I'll let you handle it. Moving this back to backlog since although the bug was thought to be fixed earlier, there is evidence that it's not really fixed and will require more investigation and a reproducer. (In reply to Richard W.M. Jones from comment #13) > Moving this back to backlog since although the bug was thought to be fixed > earlier, there is evidence that it's not really fixed and will require > more investigation and a reproducer. Thanks. Should we also do anything about the RHEL8.5 bug 1976024 which is CLOSED ERRATA? If RHEL 8 is fixed, we might need to flag this as a regression and get it fixed asap. (In reply to Richard W.M. Jones from comment #13) > Moving this back to backlog since although the bug was thought to be fixed > earlier, there is evidence that it's not really fixed and will require > more investigation and a reproducer. Where is the evidence that this is not fixed? I think the original issue in RHV was fixed, and current virt-v2v monitors the image transfer properly so this issue cannot happen now. (In reply to Klaus Heinrich Kiwi from comment #14) > Thanks. Should we also do anything about the RHEL8.5 bug 1976024 which is > CLOSED ERRATA? If RHEL 8 is fixed, we might need to flag this as a > regression and get it fixed asap. I wouldn't do anything unless a customer reports a problem. There is no customer case attached to either this bug or 1976024. (In reply to Nir Soffer from comment #15) > Where is the evidence that this is not fixed? > > I think the original issue in RHV was fixed, and current virt-v2v monitors > the image transfer properly so this issue cannot happen now. I have not tried to reproduce the bug myself, I'm just going off the preceeding comments. If it's either fixed -- or not a problem because it somehow only affects tiny OVAs which we'd never see in real life -- then let's close the bug. (In reply to Richard W.M. Jones from comment #16) > (In reply to Klaus Heinrich Kiwi from comment #14) > > Thanks. Should we also do anything about the RHEL8.5 bug 1976024 which is > > CLOSED ERRATA? If RHEL 8 is fixed, we might need to flag this as a > > regression and get it fixed asap. > > I wouldn't do anything unless a customer reports a problem. There > is no customer case attached to either this bug or 1976024. > > (In reply to Nir Soffer from comment #15) > > Where is the evidence that this is not fixed? > > > > I think the original issue in RHV was fixed, and current virt-v2v monitors > > the image transfer properly so this issue cannot happen now. > > I have not tried to reproduce the bug myself, I'm just going off the > preceeding comments. If it's either fixed -- or not a problem > because it somehow only affects tiny OVAs which we'd never see in real > life -- then let's close the bug. Let's take the latter approach unless there's evidence to do otherwise. |