Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1392032

Summary: libvirt use wrong qmp command to hotplug a vhostuser network
Product: Red Hat Enterprise Linux 7 Reporter: Marcel Kolaja <mkolaja>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: chhu
Severity: medium Docs Contact:
Priority: high    
Version: 7.3CC: chhu, dyuan, jdenemar, jsuchane, lhuang, mprivozn, rbalakri, snagar, xuzhang, yalzhang
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-2.0.0-10.el7_3.3 Doc Type: Bug Fix
Doc Text:
Previously, the libvirt library was using wrong set of commands on the QEMU monitor, when trying to hotplug vhost-user interface to a guest. As a consequence, the hot plug of the vhost-user interface failed. This update corrects the hot-plug code for vhost-user devices. As a result, the hotplug now succeeds as expected.
Story Points: ---
Clone Of: 1366108 Environment:
Last Closed: 2017-01-17 18:27:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1366108    
Bug Blocks:    

Description Marcel Kolaja 2016-11-04 15:44:04 UTC
This bug has been copied from bug #1366108 and has been proposed
to be backported to 7.3 z-stream (EUS).

Comment 6 chhu 2016-12-23 09:38:51 UTC
Failed to verify this bug with packages:

libvirt-2.0.0-10.el7_3.3.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.2.x86_64
kernel-3.10.0-514.6.1.el7.x86_64

Hit qemu-kvm-rhev bug, which block this bug verification:
Bug 1394140 - qemu gets SIGSEGV when hot-plug a vhostuser network.

Comment 7 chhu 2017-01-05 08:34:05 UTC
Verified with packages:
libvirt-2.0.0-10.el7_3.3.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64
kernel: 3.10.0-514.6.1.el7.x86_64

Test steps:
1. Setup the openvswtich dpdk environment, start an guest without vhostuser network.
 	
# virsh start vm
Domain vm started
# virsh dumpxml vm|grep interface -A 5

2. Open another terminal to setup a systemtap of qemu monitor command:
# stap qemu-monitor.stp 
  0.000 begin

3. Attach a vhostuser network successfully.
# cat vhostuser1.xml 
    <interface type='vhostuser'>
      <mac address='52:54:00:5f:cd:11'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>

# virsh attach-device vm vhostuser1.xml 
Device attached successfully

# virsh dumpxml vm|grep interface -A 5
    <interface type='vhostuser'>
      <mac address='52:54:00:5f:cd:11'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>

4. Check the systemtap of qemu monitor command output: libvirt attach a chardev which backend is socket and then use netdev_add vhost-user net device, then use device_add to add virtio-net-pci

# stap qemu-monitor.stp 
  0.000 begin
136.125 > 0x7fa754011df0 {"execute":"chardev-add","arguments":{"id":"charnet0","backend":{"type":"socket","data":{"addr":{"type":"unix","data":{"path":"/var/run/openvswitch/vhost-user1"}},"wait":false,"server":false}}},"id":"libvirt-14"}
136.129 < 0x7fa754011df0 {"return": {}, "id": "libvirt-14"}
136.129 > 0x7fa754011df0 {"execute":"netdev_add","arguments":{"type":"vhost-user","chardev":"charnet0","id":"hostnet0"},"id":"libvirt-15"}
136.132 < 0x7fa754011df0 {"return": {}, "id": "libvirt-15"}
136.132 > 0x7fa754011df0 {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:5f:cd:11","bus":"pci.0","addr":"0x8"},"id":"libvirt-16"}
136.144 < 0x7fa754011df0 {"return": {}, "id": "libvirt-16"}

5. Attach the second vhostuser network and detach it, check the systemtap of qemu monitor command output: PASS
# virsh attach-device vm vhostuser2.xml
Device attached successfully
# virsh dumpxml vm |grep interface -A 5
    <interface type='vhostuser'>
      <mac address='52:54:00:5f:cd:11'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='vhostuser'>
      <mac address='52:54:00:5f:dd:21'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user2' mode='client'/>
      <model type='virtio'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

# stap qemu-monitor.stp
  0.000 begin
 47.410 > 0x7fa754011df0 {"execute":"chardev-add","arguments":{"id":"charnet1","backend":{"type":"socket","data":{"addr":{"type":"unix","data":{"path":"/var/run/openvswitch/vhost-user2"}},"wait":false,"server":false}}},"id":"libvirt-18"}
 47.414 < 0x7fa754011df0 {"return": {}, "id": "libvirt-18"}
 47.414 > 0x7fa754011df0 {"execute":"netdev_add","arguments":{"type":"vhost-user","chardev":"charnet1","id":"hostnet1"},"id":"libvirt-19"}
 47.417 < 0x7fa754011df0 {"return": {}, "id": "libvirt-19"}
 47.417 > 0x7fa754011df0 {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"52:54:00:5f:dd:21","bus":"pci.0","addr":"0x3"},"id":"libvirt-20"}
 47.432 < 0x7fa754011df0 {"return": {}, "id": "libvirt-20"}

Login to the guest:
# lspci | grep Ethernet
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:08.0 Ethernet controller: Red Hat, Inc Virtio network device

# virsh detach-device vm vhostuser2.xml
Device detached successfully

# virsh dumpxml vm |grep interface -A 5
    <interface type='vhostuser'>
      <mac address='52:54:00:5f:cd:11'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>

# stap qemu-monitor.stp
  0.000 begin
  3.349 > 0x7fa754011df0 {"execute":"device_del","arguments":{"id":"net1"},"id":"libvirt-22"}
  3.351 < 0x7fa754011df0 {"return": {}, "id": "libvirt-22"}
  3.411 ! 0x7fa754011df0 {"timestamp": {"seconds": 1483598085, "microseconds": 807355}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/net1/virtio-backend"}}
  3.412 ! 0x7fa754011df0 {"timestamp": {"seconds": 1483598085, "microseconds": 807439}, "event": "DEVICE_DELETED", "data": {"device": "net1", "path": "/machine/peripheral/net1"}}
  3.412 > 0x7fa754011df0 {"execute":"netdev_del","arguments":{"id":"hostnet1"},"id":"libvirt-23"}
  3.416 < 0x7fa754011df0 {"return": {}, "id": "libvirt-23"}
  3.416 > 0x7fa754011df0 {"execute":"chardev-remove","arguments":{"id":"charnet1"},"id":"libvirt-24"}
  3.417 < 0x7fa754011df0 {"return": {}, "id": "libvirt-24"}

Login to the guest:
# lspci|grep Ethernet
00:08.0 Ethernet controller: Red Hat, Inc Virtio network device

Comment 9 errata-xmlrpc 2017-01-17 18:27:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0098.html