Bug 1123698

(In reply to Xiaomei Gao from comment #0)
> Guest version:
> qemu-kvm-0.12.1.2-2.430.el6.x86_64

Here is correct guest version
kernel-2.6.32-490.el6.x86_64

We have tried to hot-plug virtio-blk disk and did not trigger the issue.

Btw, if the hot-plugged disk shares one scsi controller with system disk, the issue will not be triggered.

Rationale of the issue is noted below.

Hot plugged devices are initialized after existing devices are already running, by the guest. When the virtio-scsi is just plugged, guest initializes it with a new allocated BAR address.

When the guest reboots, the device is seen at boot time. It's initialized together with all the other devices, in a sequence as the guest prefers.

In other words, in the fist case, virtio-scsi is initialized after all the other devices, but not in the second case. This makes a different BAR in RHEL 6 (not quite in RHEL 7 though).

So when reboot, BAR is changed and the device is reset.

Currently in qemu-kvm, we need to set up ioeventfd with an address computed based on BAR at the time of device initialization, and tear it down on device reset with the same address. The issue comes when BAR is changed - we lost the original ioeventfd address and use the new BAR to unassign it. Hence the assert on success of unassign fails, with -ENOENT.

The fix is to stop ioeventfd before BAR change and start it after.

Thanks to Paolo for helping debug this issue.

Fam

Normally, ioeventfd is not started when guest changes BAR. Another bug in qemu-kvm forgets to reset virtio-scsi on vm reboot.

So two separate bugs need to be fixed here.

(In reply to Fam Zheng from comment #6)
> Normally, ioeventfd is not started when guest changes BAR. Another bug in
> qemu-kvm forgets to reset virtio-scsi on vm reboot.
> 
> So two separate bugs need to be fixed here.

Hi, Fam

Could we cover both of the two fixes with the scenario in comment 0 when we get the fixed build?

Yes, please test the scenario in comment 0.

Fix included in qemu-kvm-0.12.1.2-2.433.el6

- Reproduce the issue on qemu-kvm-0.12.1.2-2.430.el6.x86_64.

1. Prepare a data disk which will be hot-pluged
# qemu-img create -f qcow2 /home/storage1.qcow2  10G

2. Boot guest
# /usr/libexec/qemu-kvm  \
   -device virtio-scsi-pci,id=virtio_scsi_pci0,addr=0x4 \
  -drive file='/home/RHEL-Server-6.6-64bit.qcow2',if=none,id=virtio-scsi0-id0,media=disk,cache=none,snapshot=off,format=qcow2,aio=threads \
  -device scsi-hd,drive=virtio-scsi0-id0 \
  -qmp tcp:0:6666,server,nowait

3. Hot-plug the data disk
(QMP) {"execute":"__com.redhat_drive_add", "arguments": {"file":"/home/storage1.qcow2","format":"qcow2","id":"virtio-scsi0-id1"}}
(QMP){"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","id":"virtio_scsi_pci1", "addr":"0x6"}}
{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"virtio-scsi0-id1","id":"scsi1"}}

4. Reboot guest
# reboot

5. Results
Qemu-kvm core dump
(qemu) qemu-kvm: virtio_pci_set_host_notifier_internal: unable to unmap ioeventfd: -2
qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/virtio-pci.c:314: virtio_pci_stop_ioeventfd: Assertion `r >= 0' failed

- Verify the bug on qemu-kvm-0.12.1.2-2.436.el6.x86_64.

1. Host kernel : kernel-2.6.32-496.el6.x86_64
   Guest kernel version : kernel-2.6.32-496.el6.x86_64

2. Repeat the above steps 3 times.

3. Results:
Qemu-kvm works happily and smoothly

Based the above test results, the bug has been verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1490.html