1404186 – attaching a new multi-queue vhost-user interface to a running VM fails

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1404186 - attaching a new multi-queue vhost-user interface to a running VM fails

Summary: attaching a new multi-queue vhost-user interface to a running VM fails

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	7.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Michal Privoznik
QA Contact:	yalzhang@redhat.com
Docs Contact:
URL:
Whiteboard:
Depends On:	1386976
Blocks:
TreeView+	depends on / blocked

Reported:	2016-12-13 09:36 UTC by Marcel Kolaja
Modified:	2023-09-14 03:36 UTC (History)
CC List:	11 users (show)
Fixed In Version:	libvirt-2.0.0-10.el7_3.4
Doc Type:	Bug Fix
Doc Text:	Prior to this update, the libvirt service was not able to attach vhost-user type interface to running QEMU machine. To fix the bug, an implementation has been introduced to libvirt that enables it to attach and detach vhost-user type interfaces.
Clone Of:	1386976
Environment:
Last Closed:	2017-01-17 18:27:50 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2017:0098	0	normal	SHIPPED_LIVE	libvirt bug fix update	2017-01-17 22:58:08 UTC

Description Marcel Kolaja 2016-12-13 09:36:12 UTC

This bug has been copied from bug #1386976 and has been proposed
to be backported to 7.3 z-stream (EUS).

Comment 4 Michal Privoznik 2016-12-15 17:20:39 UTC

To POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2016-December/msg00163.html

Comment 6 Jiri Denemark 2016-12-22 07:28:27 UTC

The following commit causes a memory leak:

    libvirt-2.0.0/src/qemu/qemu_hotplug.c:1278: var_assign: Assigning: "originalError" = storage returned from "virSaveLastError()".
    libvirt-2.0.0/src/qemu/qemu_hotplug.c:1272: leaked_storage: Variable "originalError" going out of scope leaks the storage it points to.

commit 0e82fa4c345acb7ad52e0da0e54f7375eda57657
Refs: v2.4.0-50-g0e82fa4c3
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Oct 25 12:16:36 2016 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Nov 10 16:47:32 2016 +0100

    qemuDomainAttachNetDevice: Don't overwrite error on rollback

    If there is an error hotpluging a net device (for whatever
    reason) a rollback operation is performed. However, whilst doing
    so various helper functions that are called report errors on
    their own. This results in the original error to be overwritten
    and thus misleading the user.

    Signed-off-by: Michal Privoznik <mprivozn>

Comment 7 Michal Privoznik 2016-12-22 13:23:51 UTC

Moving to POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2016-December/msg00236.html

Comment 8 yalzhang@redhat.com 2017-01-05 08:22:08 UTC

Test with below packages, the result is expected.

libvirt-2.0.0-10.el7.3.x86_64 (build from libvirt-2.0.0-10.el7_3.3 with the patch in #c7)
qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64

1. # cat vhostuser2.xml 
  <interface type='vhostuser'>
      <source type='unix' path='/var/run/openvswitch/vhost-user2' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
    </interface>

2. # virsh attach-device r7 vhostuser2.xml 
Device attached successfully

3. # virsh dumpxml r7 | grep /interface -B7
    <interface type='vhostuser'>
      <mac address='52:54:00:b5:1d:c0'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user2' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>

check on the guest, the new attached interface is existed.

4. # virsh detach-interface r7 vhostuser 52:54:00:b5:1d:c0
Interface detached successfully

check the xml and on the guest, the interface is gone.

Comment 9 yalzhang@redhat.com 2017-01-06 06:27:38 UTC

Test on below packages, the result is expected, set the bug to verified.
libvirt-2.0.0-10.el7_3.4.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64

1. # virsh start r7
Domain r7 started

# cat vhostuser1.xml 
  <interface type='vhostuser'>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
    </interface>

2. # virsh attach-device r7 vhostuser1.xml
Device attached successfully

# virsh dumpxml r7 | grep /interface -B7
    <interface type='vhostuser'>
      <mac address='52:54:00:dd:7c:6e'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

3. on the guest
# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:		0
TX:		0
Other:		0
Combined:	4
Current hardware settings:
RX:		0
TX:		0
Other:		0
Combined:	1

# ethtool -L eth0 combined 4

# ethtool -l eth0 
Channel parameters for eth0:
Pre-set maximums:
RX:		0
TX:		0
Other:		0
Combined:	4
Current hardware settings:
RX:		0
TX:		0
Other:		0
Combined:	4

4. detach
# virsh detach-interface r7 vhostuser 52:54:00:dd:7c:6e
Interface detached successfully

Comment 11 errata-xmlrpc 2017-01-17 18:27:50 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0098.html

Comment 12 Michal Privoznik 2017-04-15 06:11:41 UTC

Dear bigswitch,

in bug 1386976#c10 you say that the attach fails for you. I'm moving the discussion here because this bug is for the version you have. Can you please get full libvirt debug logs and attach them here?

http://wiki.libvirt.org/page/DebugLogs

Thanks,
Michal

Comment 13 Red Hat Bugzilla 2023-09-14 03:36:07 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days

Note You need to log in before you can comment on or make changes to this bug.