Bug 1150484

libvirt.x86_64                  0.10.2-29.el6_5.8

For virtio-scsi, ioeventfd must be used on the <controller> element rather than <disk>.

This issue can be reproduced with libvirt-0.10.2-46.el6.x86_64 .

Reproduce version:
libvirt-0.10.2-46.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.448.el6.x86_64
kernel-2.6.32-506.el6.x86_64

Reproduce steps:

1.try to create a guest according to the description

#vim r6.xml
.......
 <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native' ioeventfd='on' />
      <source file='/var/lib/libvirt/images/rhel6.img'/>
      <target dev='sda'/>
    </disk>
    <controller type='scsi' model='virtio-scsi'/>
......

try to create :

# virsh create r6.xml
error: Failed to create domain from r6.xml
error: unsupported configuration: disk ioeventfd mode supported only for virtio bus

try to define :

# virsh define r6.xml
error: Failed to define domain from r6.xml
error: unsupported configuration: disk ioeventfd mode supported only for virtio bus


NOTE：
It cannot be saved successfully in domain XML if try to add ioeventfd option into <controller> element. It will be removed after saving.

The description of ioeventfd can be see in http://libvirt.org/formatdomain.html#elementsDisks
......
driver
.....
The optional ioeventfd attribute allows users to set domain I/O asynchronous handling for disk device.
......
No more description about it in http://libvirt.org/formatdomain.html#elementsControllers .

So it seems that ioeventfd option should belong to < disk driver > element .

The error is correct, the ioeventfd option needs to be added to the <controller> element (corresponding to -device virtio-scsi-pci, not -device scsi-disk).  But the bug is there nevertheless, since libvirt doesn't support ioeventfd on controllers.

Since this sounds more like a new feature request, I'd rather move it to RHEL 7 to eliminate the risk of back-porting such feature to older RHEL 6 codebase.

Yes, it's a new feature.  It's very rare that you need to disable ioeventfd.

Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2015-July/msg01027.html

Fixed upstream by v1.2.18-64-g35eecddee3fd and v1.2.18-65-gcf0404455cec:

commit 35eecddee3fd822285a4ac619ac972f3714e8edc
Author: Martin Kletzander <mkletzan>
Date:   Fri Jul 24 15:49:35 2015 +0200

    conf: Add ioeventfd option for controllers

commit cf0404455cecb47ddbfa3266eea6b66f8540facd
Author: Martin Kletzander <mkletzan>
Date:   Sun Jul 26 20:41:34 2015 +0200

    qemu: Enable ioeventfd usage for virtio-scsi controllers

Verify this bug with the following packages:
libvirt-1.2.17-10.el7.x86_64
qemu-kvm-rhev-2.3.0-22.el7.x86_64
kernel-3.10.0-319.el7.x86_64

Followings are the two concerns need to be discussed with the developer:
1. As for the step 2 of the scenario2, the error message is not accept, since the ioeventfd is supported not only virtio, it's also supported by scsi bus now. Should I file another bug to track this issue?
2. As you know, the ioeventfd is supported by virtio and scsi buses in libvirt, but the setting of "ioeventfd='on'" is in different xml section for virtio and scsi.
For the virtio bus disk: setting in the driver of disk section, such as following:
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' ioeventfd='on'/>
......
For the scsi bus disk, setting in the driver of scsi controller section, such as following:
   <controller type='scsi' index='0'>
      <driver ioeventfd='on'/>
......
Should we only setting in the driver of disk? If not, please must describe the different setting method in the document or libvirt.org. Otherwise, it will cause the costomer confuse.

Scenario1: enable the ioeventfd for the virtio disk.
1. prepare one guest like following one, then start the guest.
......
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' ioeventfd='on'/>
      <source file='/var/lib/libvirt/images/rhel7.1-virtio.img'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
......
2. the qemu command line is as expected.
-drive file=/var/lib/libvirt/images/rhel7.1-virtio.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,ioeventfd=on,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1


Scenario2: enable the ioeventfd for the scsi disk.
1. prepare one guest like following one
......
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' ioeventfd='on'/>
      <source file='/var/lib/libvirt/images/rhel7.1-virtio.img'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
    </disk>
   <controller type='scsi' index='0'>
      <driver ioeventfd='on'/>
    </controller>
......
2. but the dumpxml can not be saved, it seems the ioeventfs='on' only support for the virtio bus.
# virsh edit rhel71
error: unsupported configuration: disk ioeventfd mode supported only for virtio bus
Failed. Try again? [y,n,i,f,?]: 

3. delete the iovevntfd='on' from the disk section, only leave the ioeventfd='on' in the scsi controller. Then the guest can be started up.
# virsh start rhel71
Domain rhel71 started

4. the qemu command line is as expected.
-device virtio-scsi-pci,id=scsi0,ioeventfd=on,bus=pci.0,addr=0x3 -drive file=/var/lib/libvirt/images/rhel7.1-virtio.img,if=none,id=drive-scsi0-0-0-0,format=qcow2

5. edit the guest dumpxml, sepcify the scsi model is "model='virtio-scsi'".
......
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rhel7.1-virtio.img'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
    </disk>
   <controller type='scsi' index='0' model='virtio-scsi'>
      <driver ioeventfd='on'/>
    </controller>
......

6. destroy and start the guest, the guest is working well.
# virsh destroy rhel71
Domain rhel71 destroyed
# virsh start rhel71
Domain rhel71 started

7. the qemu command line is as expected.
-device virtio-scsi-pci,id=scsi0,ioeventfd=on,bus=pci.0,addr=0x3 -drive file=/var/lib/libvirt/images/rhel7.1-virtio.img,if=none,id=drive-scsi0-0-0-0,format=qcow2



Scenario3: disable the ioeventfd for the scsi disk.
1. prepare one guest like following one
......
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rhel7.1-virtio.img'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
    </disk>
   <controller type='scsi' index='0'>
      <driver ioeventfd='off'/>
    </controller>
......
2. the guest can be started up and working well.
# virsh start rhel71
Domain rhel71 started

3. the qemu command line is as expected.
-device virtio-scsi-pci,id=scsi0,ioeventfd=off,bus=pci.0,addr=0x3 -drive file=/var/lib/libvirt/images/rhel7.1-virtio.img,if=none,id=drive-scsi0-0-0-0,format=qcow2

4. edit the guest dumpxml, sepcify the scsi model is "model='virtio-scsi'".
......
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rhel7.1-virtio.img'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
    </disk>
   <controller type='scsi' index='0' model='virtio-scsi'>
      <driver ioeventfd='on'/>
    </controller>
......

5. destroy and start the guest, the guest is working well.
# virsh destroy rhel71
Domain rhel71 destroyed
# virsh start rhel71
Domain rhel71 started

6. the qemu command line is as expected.
-device virtio-scsi-pci,id=scsi0,ioeventfd=off,bus=pci.0,addr=0x3 -drive file=/var/lib/libvirt/images/rhel7.1-virtio.img,if=none,id=drive-scsi0-0-0-0,format=qcow2

ad. 1 Yes, the error message is incorrect, file a new bug for that, thank you.

ad. 2 The place where to specify it is correct.  The ioeventfd is always used on the virtio device.  For scsi disks on virtio-scsi, the virtio device is the virtio-scsi controller, for virtio disk it is the disk.

(In reply to Martin Kletzander from comment #12)
> ad. 1 Yes, the error message is incorrect, file a new bug for that, thank
> you.

Thanks for your confirmation, file following bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1269697

> 
> ad. 2 The place where to specify it is correct.  The ioeventfd is always
> used on the virtio device.  For scsi disks on virtio-scsi, the virtio device
> is the virtio-scsi controller, for virtio disk it is the disk.

Yeah, we know the different setting xml section for the virtio and scsi disk. but this will cause the customer confused, it's better to describe more clearly in the RHEL7.2 administrator guide or the libvirt.org.

Following is the ioeventfd info in the libvirt.org, can not see any info that "while enblae the ioeventfd, the scsi disk should be in the <controller type='scsi'...> xml section, the vrtio disk should be in the <disk...> xml section."
ioeventfd in disk xml section:
http://libvirt.org/formatdomain.html#elementsDisks
...
The optional ioeventfd attribute allows users to set domain I/O asynchronous handling for disk device. The default is left to the discretion of the hypervisor. Accepted values are "on" and "off". Enabling this allows qemu to execute VM while a separate thread handles I/O. Typically guests experiencing high system CPU utilization during I/O will benefit from this. On the other hand, on overloaded host it could increase guest I/O latency. Since 0.9.3 (QEMU and KVM only) In general you should leave this option alone, unless you are very certain you know what you are doing.
...

ioeventfd in controller xml section:
http://libvirt.org/formatdomain.html#elementsControllers
...
ioeventfd
The optional ioeventfd attribute specifies whether the controller should use I/O asynchronous handling or not. Accepted values are "on" and "off". Since 1.2.18
...

(In reply to Zhang Xuesong from comment #13)
> (In reply to Martin Kletzander from comment #12)
> > ad. 1 Yes, the error message is incorrect, file a new bug for that, thank
> > you.
> 
> Thanks for your confirmation, file following bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=1269697
> 
> > 
> > ad. 2 The place where to specify it is correct.  The ioeventfd is always
> > used on the virtio device.  For scsi disks on virtio-scsi, the virtio device
> > is the virtio-scsi controller, for virtio disk it is the disk.
> 
> Yeah, we know the different setting xml section for the virtio and scsi
> disk. but this will cause the customer confused, it's better to describe
> more clearly in the RHEL7.2 administrator guide or the libvirt.org.
> 
> Following is the ioeventfd info in the libvirt.org, can not see any info
> that "while enblae the ioeventfd, the scsi disk should be in the <controller
> type='scsi'...> xml section, the vrtio disk should be in the <disk...> xml
> section."
> ioeventfd in disk xml section:
> http://libvirt.org/formatdomain.html#elementsDisks
> ...
> The optional ioeventfd attribute allows users to set domain I/O asynchronous
> handling for disk device. The default is left to the discretion of the
> hypervisor. Accepted values are "on" and "off". Enabling this allows qemu to
> execute VM while a separate thread handles I/O. Typically guests
> experiencing high system CPU utilization during I/O will benefit from this.
> On the other hand, on overloaded host it could increase guest I/O latency.
> Since 0.9.3 (QEMU and KVM only) In general you should leave this option
> alone, unless you are very certain you know what you are doing.
> ...
> 
> ioeventfd in controller xml section:
> http://libvirt.org/formatdomain.html#elementsControllers
> ...
> ioeventfd
> The optional ioeventfd attribute specifies whether the controller should use
> I/O asynchronous handling or not. Accepted values are "on" and "off". Since
> 1.2.18
> ...

As for the 2nd concern, file one bug to doc team tracking:
https://bugzilla.redhat.com/show_bug.cgi?id=1269713

So, close this bug.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html