Bug 1140001

Summary: data-plane hotplug should be refused to start if device is already in use (drive-mirror job)
Product: Red Hat Enterprise Linux 7 Reporter: Sibiao Luo <sluo>
Component: qemu-kvm-rhevAssignee: Stefan Hajnoczi <stefanha>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: chayang, famz, hhuang, juzhang, michen, pbonzini, qzhang, rbalakri, stefanha, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.1.2-4.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1250861 (view as bug list) Environment:
Last Closed: 2015-03-05 09:54:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1250861    

Description Sibiao Luo 2014-09-10 07:11:29 UTC
Description of problem:
QEMU should print an error message and refuse to hotplug with virtio-blk data-plane while the drive-mirror job is running, this can prevent data corruption according to bug 995530.

Version-Release number of selected component (if applicable):
host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-152.el7.x86_64
qemu-kvm-rhev-2.1.0-3.el7.x86_64
guest info:
# uname -r
3.10.0-152.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.start QEMU with "-drive file=/home/RHEL-7.0-20140507.0-Server-x86_64.qcow2,if=none,id=drive-system-disk0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop"

2.the guest will be at the BIOS screen because there is no virtio-blk-pci device yet. Then start a drive-mirror operation on the QMP monitor: { "execute": "drive-mirror", "arguments": { "device": "drive-system-disk0", "target": "/root/sn1", "format": "qcow2", "mode": "absolute-paths", "sync": "full", "speed": 1000000000, "on-target-error": "stop" } }

3.do hotplug virtio-blk data-plane device.
{"execute":"device_add","arguments":{"driver":"virtio-blk-pci","drive":"drive-system-disk0","id":"system-disk0","iothread":"iothread0"}}

4.do system_rest to start KVM guest.

Actual results:
1.step 2 and step 3 QMP commands execute successfully, while the QEMU HMP monitor prompt a "cannot start dataplane thread: Device 'drive-system-disk0' is busy: block device is in use by block job: mirror" message.

{ "execute": "drive-mirror", "arguments": { "device": "drive-system-disk0", "target": "/root/sn1", "format": "qcow2", "mode": "absolute-paths", "sync": "full", "speed": 1000000000, "on-target-error": "stop" } }
{"return": {}}
{ "execute" : "query-block-jobs", "arguments" : {} }
{"return": [{"io-status": "ok", "device": "drive-system-disk0", "busy": true, "len": 10737418240, "offset": 3931963392, "paused": false, "speed": 1000000000, "type": "mirror"}]}
{"execute":"device_add","arguments":{"driver":"virtio-blk-pci","drive":"drive-system-disk0","id":"system-disk0","iothread":"iothread0"}}
{"return": {}}

(qemu) Formatting '/root/sn1', fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 lazy_refcounts=off 
cannot start dataplane thread: Device 'drive-system-disk0' is busy: block device is in use by block job: mirror

2.after step 4, guest boot up successfully.
{ "execute": "system_reset" }
{"return": {}}

Expected results:
QEMU should print an error message and refuse to hotplug with virtio-blk data-plane while the drive-mirror job is running, this can prevent data corruption.
So, step 3 should not hotplug successfully and step 4 guest should not boot up successfully.

Additional info:

Comment 1 Sibiao Luo 2014-09-10 07:12:12 UTC
my whole qemu-kvm command line:
# /usr/libexec/qemu-kvm -M pc -S -cpu SandyBridge -enable-kvm -m 2048 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm1,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:01:02:B6:40:21,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x6 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -serial unix:/tmp/ttyS0,server,nowait -spice disable-ticketing,port=5931 -vga qxl -global qxl-vga.vram_size=33554432 -monitor stdio -object iothread,id=iothread0 -drive file=/home/RHEL-7.0-20140507.0-Server-x86_64.qcow2,if=none,id=drive-system-disk0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop

Best Regards,
sluo

Comment 2 Stefan Hajnoczi 2014-09-11 12:50:49 UTC
Thanks, there was a regression upstream and I have submitted a patch:
[PATCH] dataplane: fix virtio_blk_data_plane_create() op blocker error path

Once it has been merged I will backport it.

Comment 3 Stefan Hajnoczi 2014-10-03 15:43:23 UTC
The backport has been posted.

Comment 5 Miroslav Rezanina 2014-10-20 07:29:34 UTC
Fix included in qemu-kvm-rhev-2.1.2-4.el7

Comment 7 Sibiao Luo 2014-10-31 05:23:18 UTC
verify this issue on qemu-kvm-rhev-2.1.2-5.el7.x86_64.

host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-191.el7.x86_64
qemu-kvm-rhev-2.1.2-5.el7.x86_64

e.g:...-object iothread,id=iothread0 -drive file=/home/rhel7-desktop-64.qcow2,if=none,id=drive-system-disk0,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop

{"execute":"qmp_capabilities"}
{"return": {}}
{ "execute": "drive-mirror", "arguments": { "device": "drive-system-disk0", "target": "/root/sn1", "format": "qcow2", "mode": "absolute-paths", "sync": "full", "speed": 1000000000, "on-target-error": "stop" } }
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-blk-pci","drive":"drive-system-disk0","id":"system-disk0","iothread":"iothread0"}}
{"error": {"class": "GenericError", "desc": "cannot start dataplane thread: Device 'drive-system-disk0' is busy: block device is in use by block job: mirror"}}

Base on above, this issue has been fixed correctly, move to VERIFIED status.

Best Regards,
sluo

Comment 9 errata-xmlrpc 2015-03-05 09:54:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0624.html