Bug 1135339

Summary: active commit will be cancelled by another commit
Product: Red Hat Enterprise Linux 7 Reporter: Shanzhi Yu <shyu>
Component: libvirtAssignee: Eric Blake <eblake>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: dyuan, eblake, lmiksik, mzhan, rbalakri, xuzhang, yanyang, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.2.8-13.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 07:43:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Shanzhi Yu 2014-08-29 06:23:40 UTC
Description of problem:

active commit(from top to middle) will be cancelled by another commit(from middle to base)

Version-Release number of selected component (if applicable):

libvirt-1.2.7-2.el7.x86_64
qemu-kvm-rhev-2.1.0-2.el7.x86_64


How reproducible:

100%

Steps to Reproduce:

1.create four disk snapshot for guest

# virsh snapshot-list rhel7-raw
 Name                 Creation Time             State
------------------------------------------------------------
 s1                   2014-08-29 11:23:51 +0800 disk-snapshot
 s2                   2014-08-29 11:25:27 +0800 disk-snapshot
 s3                   2014-08-29 11:26:59 +0800 disk-snapshot
 s4                   2014-08-29 11:28:04 +0800 disk-snapshot

backing chain like:

rhel7-raw.img<-rhel7-raw.s1<-rhel7-raw.s2<-rhel7-raw.s3<-rhel7-raw.s4

2.do blockcommit from rhel7-raw.s2 to rhel7-raw.s1 when there is a blockcommit job from top to rhel7-raw.s3

2.1 active blockcommit from s4 to s3 in one terminal

# virsh blockcommit rhel7-raw vda --active --base /var/lib/libvirt/images/rhel7-raw.s3  --pivot
error: failed to pivot job for disk vda
error: Requested operation is not valid: pivot of disk 'vda' requires an active copy job

2.2 blockcommit from s2 to s1 in another terminal

# virsh blockjob rhel7-raw vda
Block Commit: [ 97 %]

# virsh blockcommit rhel7-raw vda --top /var/lib/libvirt/images/rhel7-raw.s2 --base /var/lib/libvirt/images/rhel7-raw.s1
error: internal error: unable to execute QEMU command 'block-commit': Device 'drive-virtio-disk0' is busy: block device is in use by block job: commit


3.check blockjob info
# virsh blockjob rhel7 vda --info




Actual results:


Expected results:


Additional info:

Comment 1 Yang Yang 2014-08-29 12:20:39 UTC
Updated the reproduced steps as following.

in terminal 1 do active commit,  in terminal 2 do another commit before the active commit completed,  actual results
active --> s3, s2 --> s1, active commit was canceled
active --> s1, s3 --> s2, not canceled
active --> s1, s2 --> s1, cancled
active --> s1, s2 --> base, not canceled
active --> base, s2 --> s1, not canceled
active --> base, s2 --> base, canceled               

1. created 4 disk-only snapshots
# virsh snapshot-list test1
 Name                 Creation Time             State
------------------------------------------------------------
 s1                   2014-08-29 19:27:25 +0800 disk-snapshot
 s2                   2014-08-29 19:27:25 +0800 disk-snapshot
 s3                   2014-08-29 19:27:26 +0800 disk-snapshot
 s4                   2014-08-29 19:27:27 +0800 disk-snapshot

2. Do active commit to s3 *without* option --pivot

# virsh blockcommit test1 hda --base /var/lib/libvirt/images/rhel7qcow2.s3 --active --wait --verbose
Block Commit: [16 %]

2.1. commit from s2 to s1 in another terminal before step 2 completed

# virsh blockcommit test1 hda --top /var/lib/libvirt/images/rhel7qcow2.s2 --base /var/lib/libvirt/images/rhel7qcow2.s1 --wait --verbose
error: internal error: unable to execute QEMU command 'block-commit': Device 'drive-ide0-0-0' is busy: block device is in use by block job: commit

2.2. in step 2, return with the following messages
# virsh blockcommit test1 hda --base /var/lib/libvirt/images/rhel7qcow2.s3 --active --wait --verbose
  Block Commit: [100 %]
  Now in synchronized phase

3. Do active commit to s3 with option --pivot
# virsh blockcommit test1 hda --base /var/lib/libvirt/images/rhel7qcow2.s3 --active --wait --verbose --pivot
Block Commit: [16 %]

3.1 commit from s2 to s1 in another terminal before step 2 completed

# virsh blockcommit test1 hda --top /var/lib/libvirt/images/rhel7qcow2.s2 --base /var/lib/libvirt/images/rhel7qcow2.s1 --wait --verbose
error: internal error: unable to execute QEMU command 'block-commit': Device 'drive-ide0-0-0' is busy: block device is in use by block job: commit

3.2 in step 3, return *with* the following errors

# virsh blockcommit test1 hda --base /var/lib/libvirt/images/rhel7qcow2.s3 --active --wait --verbose --pivot
Block Commit: [100 %]error: failed to pivot job for disk hda
error: Requested operation is not valid: pivot of disk 'hda' requires an active copy job

Actual results:
in step 2.2 and 3.2 active commit is canceled with different messages

Expected results:
in step 2.2 and 3.2 active commit should not be canceled.

Comment 2 Eric Blake 2015-01-12 23:25:42 UTC
Qemu does not yet support simultaneous block jobs on the same block device.  It should be impossible to ever start a second block job (whether inactive or active) while a first is running.  Prior to active commits, we had always been relying on qemu to report this on our behalf; but with active commits, relying on qemu was getting libvirt into some inconsistent states.  I've proposed an upstream patch that lets libvirt detect nested attempts by itself without relying on qemu during an active commit (it still relies on qemu during inactive commits, but those don't have as much libvirt state to mess up).

https://www.redhat.com/archives/libvir-list/2015-January/msg00351.html

Someday, when qemu supports simultaneous jobs on the same device, then libvirt should relax its restrictions and do likewise.  But that won't happen for 7.1.

Comment 5 Shanzhi Yu 2015-01-20 12:04:59 UTC
Verify this bug with libvirt-1.2.8-13.el7.x86_64.
Steps as comment 0 and comment 1

Comment 7 errata-xmlrpc 2015-03-05 07:43:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html