Bug 1377442 - Clone from snapshot on Cinder hangs
Summary: Clone from snapshot on Cinder hangs
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backend.Core
Version: 4.0.6.3
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ovirt-4.0.5
: 4.0.5
Assignee: Tal Nisan
QA Contact: Aharon Canan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-19 17:37 UTC by Andrew Richards
Modified: 2017-02-10 22:13 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-10-05 08:53:54 UTC
oVirt Team: Storage
Embargoed:
tnisan: ovirt-4.0.z?
rule-engine: planning_ack?
tnisan: devel_ack+
rule-engine: testing_ack?

Attachments (Terms of Use)
/var/log/ovirt-engine/engine.log context of error referenced (38.26 KB, text/plain)
2016-09-19 17:37 UTC, Andrew Richards 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 64288 0 master MERGED core: Fix clone VM from snapshot with Cinder disks 2016-09-22 12:00:52 UTC
oVirt gerrit 64291 0 ovirt-engine-4.0 MERGED core: Fix clone VM from snapshot with Cinder disks 2016-09-22 12:33:20 UTC

Description Andrew Richards 2016-09-19 17:37:39 UTC 
Created attachment 1202569 [details]
/var/log/ovirt-engine/engine.log context of error referenced

Description of problem: Trying to clone a VM from s snapshot of an RBD Cinder volume results in an error and a task that will retry indefinitely. Live snapshot taken of running CentOS 7 guest is used to create a clone, but the clone task hangs. Cinder shows the new volume successfully created and available. No errors in the Cinder logs.


Version-Release number of selected component (if applicable): 
oVirt Engine v4.0.3-1.el7
Cinder (via RDO Liberty) v7.0.1-1.el7


How reproducible: Consistently reproducible 


Steps to Reproduce: all via oVirt GUI Administrative panel
1. Create VM with Cinder RBD boot volume
2. Create snapshot of this VM
3. Clone a new VM from that snapshot


Actual results: Task to create the clone never completes, but never fails either.


Expected results: Clone completes and uses the newly cloned RBD image as its boot device.


Additional info: /var/log/ovirt-engine/engine.log initial error message:
 
ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-93) [50af631a] Command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' failed: null

ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-93) [50af631a] Exception: java.lang.NullPointerException

...which is then followed repeatedly by:

ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (DefaultQuartzScheduler8) [50af631a] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure.

ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommand] (DefaultQuartzScheduler8) [50af631a] Ending command 'org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommand' with failure.

INFO  [org.ovirt.engine.core.utils.transaction.TransactionSupport] (DefaultQuartzScheduler8) [] transaction rolled back

ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler8) [] Correlation ID: 284e62be, Job ID: 1899e73a-e5f8-4cda-b166-fbd5d444dc7c, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM testRBDclone1 creation.

ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (DefaultQuartzScheduler8) [] Failed invoking callback end method 'onFailed' for command '81f6cbd0-6298-4033-99a2-6a740e10c61f' with exception 'null', the callback is marked for end method retries

Full log snippet here: http://pastebin.com/ZLqRDSNW as well as attached TXT.

Cinder is from RDO repo, Liberty version. Cinder API is running v2.
Note You need to log in before you can comment on or make changes to this bug.