Description of problem: Testing qemu 2.1.1 migration with openstack I see the source QEMU crashing during migration. This is a regression from 2.1.0 version 2014-09-19 12:46:05.760+0000: starting up LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name instance-00000006 -S -machine pc-i440fx-2.1,accel=kvm,usb=off -cpu Conroe -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ef67b53c-92ea-4bd3-a22a-b27052fc6f2a -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=2014.2.0,serial=f09d05e7-68d8-4cab-affb-69a368983b1b,uuid=ef67b53c-92ea-4bd3-a22a-b27052fc6f2a -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000006.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/berrange/src/cloud/data/nova/instances/ef67b53c-92ea-4bd3-a22a-b27052fc6f2a/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/home/berrange/src/cloud/data/nova/instances/ef67b53c-92ea-4bd3-a22a-b27052fc6f2a/disk.config,if=none,id=drive-ide0-1-1,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:4e:01:19,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/home/berrange/src/cloud/data/nova/instances/ef67b53c-92ea-4bd3-a22a-b27052fc6f2a/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on char device redirected to /dev/pts/26 (label charserial1) qemu-system-x86_64: /home/berrange/src/fedora/qemu/qemu-2.1.1/hw/net/virtio-net.c:1348: virtio_net_save: Assertion `!n->vhost_started' failed. 2014-09-19 12:51:20.423+0000: shutting down Version-Release number of selected component (if applicable): qemu-2.1.1-1.fc21.x86_64 How reproducible: Always (with openstack VM config at least) Steps to Reproduce: 1. Boot guest with a virtio NIC 2. Trigger migration 3. Actual results: Source QEMU crashes with an assert Expected results: Migration works ! Additional info:
I know there are some patches that are already accumulating for qemu 2.1.2 that I think are migration related. mst, does that assertion `!n->vhost_started' sound familiar?
yes, a known bug, will be fixed in 2.1.2
qemu-2.1.1-2.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/qemu-2.1.1-2.fc21
Package qemu-2.1.1-2.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing qemu-2.1.1-2.fc21' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-11180/qemu-2.1.1-2.fc21 then log in and leave karma (feedback).
qemu-2.1.2-2.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/qemu-2.1.2-2.fc21
qemu-2.1.2-2.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.
Because of that BZ, the latest QEMU packages for fedora21 are broken with pseries booting from a virtio disk. When looking at the source RPM, I have spotted the following patch file: 0006-virtio-pci-enable-bus-master-for-old-guests.patch From a9862ff2c205aa289b62abcb7ddd03c7630f5c7a Mon Sep 17 00:00:00 2001 From: "Michael S. Tsirkin" <mst> Date: Thu, 11 Sep 2014 18:45:33 +0200 Subject: [PATCH] virtio-pci: enable bus master for old guests commit cc943c36faa192cd4b32af8fe5edb31894017d35 pci: Use bus master address space for delivering MSI/MSI-X messages breaks virtio-net for rhel6.[56] x86 guests because they don't enable bus mastering for virtio PCI devices. For the same reason, rhel6.[56] ppc64 guests cannot boot on a virtio-blk disk anymore. Old guests forgot to enable bus mastering, enable it automatically on DRIVER (guests use some devices before DRIVER_OK). Reported-by: Greg Kurz <gkurz.ibm.com> Reviewed-by: Greg Kurz <gkurz.ibm.com> Tested-by: Greg Kurz <gkurz.ibm.com> Signed-off-by: Michael S. Tsirkin <mst> (cherry picked from commit e43c0b2ea5574efb0bedebf6a7d05916eefeba52) This commit has been reviewed as toxic and reverted from the QEMU master branch upstream: http://git.qemu-project.org/?p=qemu.git;a=commit;h=45363e46aeebfc99753389649eac7c7fc22bfe52 If fedora21 is supposed to act as an hypervisor for pseries guest, you should consider reverting this commit as well.
(In reply to Greg Kurz from comment #7) > Because of that BZ, the latest QEMU packages for fedora21 are broken with > pseries booting from a virtio disk. > > When looking at the source RPM, I have spotted the following patch file: > > 0006-virtio-pci-enable-bus-master-for-old-guests.patch > Oops my bad, wrong patch file... I meant: 0007-virtio-pci-fix-migration-for-pci-bus-master.patch From 2c8d6826ebaf1bf846edd213f8fe8f95c0c300cb Mon Sep 17 00:00:00 2001 From: "Michael S. Tsirkin" <mst> Date: Thu, 11 Sep 2014 18:34:29 +0300 Subject: [PATCH] virtio-pci: fix migration for pci bus master ... Cc: Jason Wang <jasowang> Cc: Greg Kurz <gkurz.ibm.com> Signed-off-by: Michael S. Tsirkin <mst> (cherry picked from commit 4d43d3f3c8147ade184df9a1e9e82826edd39e19)
qemu-2.1.2-4.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/qemu-2.1.2-4.fc21
Package qemu-2.1.2-4.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing qemu-2.1.2-4.fc21' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-12404/qemu-2.1.2-4.fc21 then log in and leave karma (feedback).
qemu-2.1.2-4.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.