RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1366505 - Failed to attach vhostuser network device with mult-queues.
Summary: Failed to attach vhostuser network device with mult-queues.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.3
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: yalzhang@redhat.com
URL:
Whiteboard:
Depends On: 1366108
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-12 07:09 UTC by chhu
Modified: 2017-08-01 23:53 UTC (History)
6 users (show)

Fixed In Version: libvirt-2.5.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-01 17:11:42 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1846 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2017-08-01 18:02:50 UTC

Description chhu 2016-08-12 07:09:14 UTC 
Description of problems:
Failed to attach vhostuser network device with mult-queues.

Version:
libvirt-2.0.0-4.el7.x86_64
qemu-kvm-rhev-2.6.0-19.el7.x86_64
kernel: 3.10.0-489.el7.x86_64
openvswitch-dpdk-2.5.0-4.el7.x86_64
dpdk-2.2.0-3.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Start VM in libvirt with vhostuser multi-queues.
# virsh start r7-raw
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     r7-raw                         running

# virsh dumpxml r7-raw
<domain type='kvm' id='2'>
  <name>r7-raw</name>
  <uuid>2ccf719b-d968-48e6-a37f-8f3d3b41877a</uuid>
  <memory unit='KiB'>1024000</memory>
  <currentMemory unit='KiB'>1024000</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB' nodeset='0'/>
    </hugepages>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <numatune>
    <memory mode='strict' nodeset='0'/>
  </numatune>
......
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <numa>
      <cell id='0' cpus='0-1' memory='1024000' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
......
    <interface type='vhostuser'>
      <mac address='52:54:00:bb:25:74'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>


2. Try to attach vhostuser network device with muti-queues. Failed with error:"error: unsupported configuration: Multiqueue network is not supported for: vhostuser"

# cat net-m.xml
    <interface type='vhostuser'>
      <source type='unix' path='/var/run/openvswitch/vhost-user2' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
    </interface>

# virsh attach-device r7-raw net-m.xml
error: Failed to attach device from net-m.xml
error: unsupported configuration: Multiqueue network is not supported for: vhostuser

3. Attach vhostuser with muti-queues in qemu-kvm command line successfully.
(qemu)  chardev-add backend=socket,id=charnet2,path=/var/run/openvswitch/vhost-user2
(qemu) netdev_add vhost-user,id=hostnet2,chardev=charnet2,queues=4
(qemu) device_add virtio-net-pci,netdev=hostnet2,id=net2
(qemu) info network
net1: index=0,type=nic,model=virtio-net-pci,macaddr=14:18:77:48:01:06 hostnet1: index=0,type=vhost-user,vhost-user0 to charnet1
net1: index=1,type=nic,hostnet1: index=1,type=vhost-user,vhost-user1 to charnet1
net1: index=2,type=nic,hostnet1: index=2,type=vhost-user,vhost-user2 to charnet1
net2: index=0,type=nic,model=virtio-net-pci,macaddr=52:54:00:12:34:56 hostnet2: index=0,type=vhost-user,vhost-user0 to charnet2
net2: index=1,type=nic,hostnet2: index=1,type=vhost-user,vhost-user1 to charnet2
net2: index=2,type=nic,hostnet2: index=2,type=vhost-user,vhost-user2 to charnet2
net2: index=3,type=nic,hostnet2: index=3,type=vhost-user,vhost-user3 to charnet2

Actual results:
In step2: Try to attach the device with muti-queues. Failed with error:"error: unsupported configuration: Multiqueue network is not supported for: vhostuser"

Expected results:
In step2: Attach the device with muti-queues successfully.

Logs in libvirtd:
2016-08-09 10:00:48.796+0000: 21771: error : qemuDomainAttachNetDevice:941 : unsupported configuration: Multiqueue network is not supported for: vhostuser
2016-08-09 10:00:48.797+0000: 21771: debug : qemuDomainObjEndJob:2849 : Stopping job: modify (async=none vm=0x7fb7ac018320 name=r7-raw)
Comment 2 Michal Privoznik 2016-08-16 16:36:33 UTC 
Patches proposed upstream:

https://www.redhat.com/archives/libvir-list/2016-August/msg00806.html
Comment 4 Michal Privoznik 2016-10-14 05:37:11 UTC 
Moving to POST:

commit ff89d5cbcf266e780b4ff93960969b3a6c2e9a01
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Aug 15 18:01:55 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 11:45:01 2016 +0800

    qemu_hotplug: Support interface type of vhost-user hotplug
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1366108
    
    There are couple of things that needs to be done in order to
    allow vhost-user hotplug. Firstly, vhost-user requires a chardev
    which is connected to vhost-user bridge and through which qemu
    communicates with the bridge (no acutal guest traffic is sent
    through there, just some metadata). In order to generate proper
    chardev alias, we must assign device alias way sooner.
    
    Then, because we are plugging the chardev first, we need to do
    the proper undo if something fails - that is remove netdev too.
    We don't want anything to be left over in case attach fails at
    some point.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit e1844d85cbdcb3ed8199ae8ae5274ffcc4968aa5
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Aug 16 12:55:02 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 11:45:01 2016 +0800

    qemuBuildHostNetStr: Support VIR_DOMAIN_NET_TYPE_VHOSTUSER
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1366505
    
    So far, this function lacked support for
    VIR_DOMAIN_NET_TYPE_VHOSTUSER leaving callers to hack around the
    problem by constructing the command line on their own. This is
    not ideal as it blocks hot plug support.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit b093e8522483c689e6ffaf7bdef98991f862c88b
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Aug 16 12:21:49 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 11:45:01 2016 +0800

    qemuBuildVhostuserCommandLine: Unify -netdev creation
    
    Currently, what we do for vhost-user network is generate the
    following part of command line:
    
    -netdev type=vhost-user,id=hostnet0,chardev=charnet0
    
    There's no need for 'type=' it is the default. Drop it.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 0c61cf315859a74326da93bbb6f04a08509ef91e
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Aug 15 16:15:44 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 11:44:53 2016 +0800

    qemuBuildVhostuserCommandLine: Reuse qemuBuildChrChardevStr
    
    There's no need to reinvent the wheel here. We already have a
    function to format virDomainChrSourceDefPtr. It's called
    qemuBuildChrChardevStr(). Use that instead of some dummy
    virBufferAsprintf().
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 336d4a71feb1cd68df13e1812e86971c61553522
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Aug 16 13:06:46 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuBuildChrChardevStr: Introduce @nowait argument
    
    This alone makes not much sense. But the aim is to reuse this
    function in qemuBuildVhostuserCommandLine() where 'nowait' is not
    supported for vhost-user devices.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 1dcbef8a0f4bdfb25bf6b8cac08afd1a589c1ccd
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Aug 16 08:48:04 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuBuildHostNetStr: Explicitly enumerate net types
    
    We tend to prevent using 'default' in switches. And it is for a
    good reason - control may end up in paths we wouldn't want for
    new values. In this specific case, if qemuBuildHostNetStr is
    called over VIR_DOMAIN_NET_TYPE_VHOSTUSER it would produce
    meaningless output. Fortunately, there no such call yet.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit c266b6044003ac2f10f4cd15e58164d629508ccb
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 26 11:53:58 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuDomainAttachNetDevice: Explicitly list allowed types for hotplug
    
    Instead of blindly claim support for hot-plugging of every
    interface type out there we should copy approach we have for
    device types: white listing supported types and explicitly error
    out on unsupported ones.
    For instance, trying to hotplug vhostuser interface results in
    nothing usable from guest currently. vhostuser typed interfaces
    require additional work on our side.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 5b65d772dd909d9b1ded07a86304229674ebf4ac
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 26 11:41:36 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuDomainAttachNetDevice: Move hostdev handling a bit further
    
    The idea is to have function that does some checking at its
    beginning and then have one big switch for all the interface
    types it supports.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 0bce012d7f0bbf68c1b42a93d7d1c4e085e2695e
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 26 11:05:00 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuBuildInterfaceCommandLine: Move from if-else forest to switch
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 4a74ccdb92fd2103392b177c38846b685e141840
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 26 10:32:44 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuBuildInterfaceCommandLine: Move vhostuser handling a bit further
    
    The idea is to have function that does some checking of the
    arguments at its beginning and then have one big switch for all
    the interface types it supports. Each one of them generating the
    corresponding part of the command line.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit ec7f612a56ed8d3a2e83101b5cffcea3058fd30a
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 26 09:54:26 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    qemuBuildInterfaceCommandLine: Move hostdev handling a bit further
    
    The idea is to have function that does some checking of the
    arguments at its beginning and then have one big switch for all
    the interface types it supports. Each one of them generating the
    corresponding part of the command line.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 507032d98d390245047f69474d0671d6b0b12651
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Sep 23 17:04:53 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    virDomainNetGetActualType: Return type is virDomainNetType
    
    This function for some weird reason returns integer instead of
    virDomainNetType type. It is important to return the correct type
    so that we know what values we can expect.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 94a10de26b1fa999dbb67cfeea8e3339205d9db6
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 26 11:06:43 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Oct 14 10:15:51 2016 +0800

    virDomainNetDefParseXML: Realign
    
    There are couple of formatting issues. No functional change
    though.
    
    Signed-off-by: Michal Privoznik <mprivozn>

v2.3.0-130-gff89d5c
Comment 6 yalzhang@redhat.com 2017-02-27 12:01:02 UTC 
verified on below packages, the result is as expected, set the bug as verified.
# rpm -q qemu-kvm-rhev libvirt
qemu-kvm-rhev-2.8.0-5.el7.x86_64
libvirt-3.0.0-2.el7.x86_64

1.
# virsh dumpxml rhel7.4
<domain type='kvm'>
  <name>rhel7.4</name>
  <uuid>c1317652-1441-418b-a06f-61062d74aeb5</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB'/>
    </hugepages>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
......
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-7' memory='1048576' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
......

2. 
# cat hostuser.xml 
  <interface type='vhostuser'>
      <source type='unix' path='/var/run/openvswitch/vhost-user2' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
    </interface>

# virsh start rhel7.4
Domain rhel7.4 started

# virsh attach-device rhel7.4 hostuser.xml  
Device attached successfully

3.
# virsh dumpxml rhel7.4 | grep /interface -B7
    <interface type='vhostuser'>
      <mac address='52:54:00:c2:bb:63'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user2' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:		0
TX:		0
Other:		0
Combined:	4
Current hardware settings:
RX:		0
TX:		0
Other:		0
Combined:	1

# ethtool -L eth0 combined 4

# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:		0
TX:		0
Other:		0
Combined:	4
Current hardware settings:
RX:		0
TX:		0
Other:		0
Combined:	4
Comment 7 errata-xmlrpc 2017-08-01 17:11:42 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 8 errata-xmlrpc 2017-08-01 23:53:19 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Note You need to log in before you can comment on or make changes to this bug.