Bug 1250798 - Boot guest with virtio-blk data-plane, do "drive-mirror", guest cannot shutdown and qemu-kvm hang.
Boot guest with virtio-blk data-plane, do "drive-mirror", guest cannot shutdo...
Status: CLOSED DUPLICATE of bug 1265179
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev (Show other bugs)
7.2
x86_64 Linux
medium Severity high
: rc
: ---
Assigned To: Stefan Hajnoczi
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-05 23:27 EDT by huiqingding
Modified: 2016-06-21 09:53 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-06-21 09:53:04 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description huiqingding 2015-08-05 23:27:05 EDT
Description of problem:
Boot guest with virtio-blk data-plane, do "drive-mirror", guest cannot shutdown and qemu-kvm hang.

Version-Release number of selected component (if applicable):
kernel-3.10.0-302.el7.x86_64
qemu-kvm-rhev-2.3.0-15.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. boot a guest with virtio-blk data-plane
# /usr/libexec/qemu-kvm -name rhel7 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -m 4096 -cpu SandyBridge -realtime mlock=on -sandbox off -smp 4,maxcpus=4,sockets=4,cores=1,threads=1 -object iothread,id=iothread0 -drive file=/home/rhel7.2.qcow2,if=none,id=data-disk1,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=data-disk1,iothread=iothread0,bus=pci.0,addr=0x6,id=data-disk1 -drive file=/home/data-disk.qcow2,if=none,id=data-disk2,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=data-disk2,iothread=iothread0,bus=pci.0,addr=0x7,id=data-disk2 -netdev tap,id=hostnet0,vhost=on,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=b6:72:66:f5:7d:93,bus=pci.0,addr=0x9 -device sga -spice port=5910,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 -qmp tcp:0:4466,server,nowait -serial unix:/tmp/ttym,server,nowait -monitor stdio

2. do drive-mirror
{"execute":"qmp_capabilities"}
{"return": {}}
{ "execute": "drive-mirror", "arguments": { "device": "data-disk2", "target": "/root/sn1", "format": "qcow2", "sync": "full"}}
{"return": {}}

3. inside guest, do shutdown
# shutdown -h now

Actual results:
after step2, "drive-mirror" can return without error info
after step3, guest cannot shutdown and qemu-kvm hang.

Expected results:
after step3, guest can shutdown and qemu-kvm can quit without error info.

Additional info:
On host, check the serial console:

# nc -U /tmp/ttym
Red Hat Enterprise Linux Server 7.2 Beta (Maipo)
Kernel 3.10.0-302.el7.x86_64 on an x86_64

dhcp-9-29 login: Powering off.
[   90.737329] Power down.
Comment 1 huiqingding 2015-08-05 23:32:31 EDT
Remove virtio-blk data-plane and boot the guest:
/usr/libexec/qemu-kvm -name rhel7 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -m 4096 -cpu SandyBridge -realtime mlock=on -sandbox off -smp 4,maxcpus=4,sockets=4,cores=1,threads=1  -drive file=/home/rhel7.2.qcow2,if=none,id=data-disk1,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=data-disk1,bus=pci.0,addr=0x6,id=data-disk1 -drive file=/home/data-disk.qcow2,if=none,id=data-disk2,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=data-disk2,bus=pci.0,addr=0x7,id=data-disk2 -netdev tap,id=hostnet0,vhost=on,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=b6:72:66:f5:7d:93,bus=pci.0,addr=0x9 -device sga -spice port=5910,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 -qmp tcp:0:4466,server,nowait -serial unix:/tmp/ttym,server,nowait -monitor stdio

Do drive-mirror and shutdown guest same as step2 and step3 in comment #0, the guest can shutdown and qemu-kvm can quit normally.
Comment 3 Stefan Hajnoczi 2015-09-11 12:27:48 EDT
Looks like a bdrv_set_aio_context() or bdrv_drain_all() issue.  Deferring to RHEL 7.2 since the scenario to trigger it doesn't appear critical enough for an exception.
Comment 5 Stefan Hajnoczi 2016-06-21 09:53:04 EDT
This bug has the same root cause as bz#1265179.  I have posted a backport for that bug.

*** This bug has been marked as a duplicate of bug 1265179 ***

Note You need to log in before you can comment on or make changes to this bug.