Bug 1291851 - support for virtio-vsock - libvirt
support for virtio-vsock - libvirt
Status: POST
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.3
Unspecified Unspecified
high Severity unspecified
: rc
: 7.4
Assigned To: Ján Tomko
yafu
: FutureFeature
Depends On: 1378137 1584011 1591105 1291282 1291284 1315822 1382695
Blocks: 1294884 RHV4.1PPC 1363787 1518997 1558125
  Show dependency treegraph
 
Reported: 2015-12-15 12:39 EST by Ademar Reis
Modified: 2018-06-14 06:53 EDT (History)
19 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 1291284
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ademar Reis 2015-12-15 12:39:58 EST
+++ This bug was initially created as a clone of Bug #1291284 +++

Description of problem:

To enable VSOCK support in 7.3, we will need the userspace qemu changes that correspond to the kernel changes in https://bugzilla.redhat.com/show_bug.cgi?id=1291282

These are currently a work in progress[1] but should be picked from upstream qemu when merged.

1. https://github.com/stefanha/qemu/tree/vsock
Comment 3 Stefan Hajnoczi 2016-09-21 10:55:52 EDT
Please note that virtio-vsock is now available in upstream Linux and QEMU.  If you have any questions about the feature, please let me know so we can discuss it.

Basic "getting started" information is available here: http://qemu-project.org/Features/VirtioVsock
Comment 4 Stefan Hajnoczi 2016-10-20 09:26:10 EDT
There is now a copr repository so you can install the kernel, qemu-kvm, and nc-vsock utility on Fedora 24:
https://copr.fedorainfracloud.org/coprs/stefanha/vsock/

Note that the qemu-ga (guest agent) in these RPMs has AF_VSOCK support.  Use qemu-ga -m vsock-listen -p 3:1234 to listen on port 1234 (assuming the guest CID is 3 on QEMU's command-line -device vhost-vsock-pci,guest-cid=3 option).
Comment 6 Stefan Hajnoczi 2017-11-09 10:47:36 EST
Hi Ján,
I suggest the following CID allocation approach:

libvirtd instances manage a CID address range from which CIDs can be allocated automatically for guests.  This is similar to DHCP ranges and it partitions the address space, allowing some ranges to be statically assigned or even owned by other libvirtd sessions without collisions.

Users should also be able to statically assign CIDs to guests.  This is useful if a user wants a persistent CID for a specific guest.

The driver interface works as follows:

  vhostfd = open("/dev/vhost-vsock")
  uint64_t guest_cid = ...;
  ioctl(vhostfd, VHOST_VSOCK_SET_GUEST_CID, &guest_cid);

The ioctl fails with EADDRINUSE if another vhost-vsock instance already has the CID assigned.  The ioctl is idempotent and succeeds if you assign the same CID again.  It fails with EINVAL if the CID is invalid (<2 or >=UINT32_MAX).

Once the CID has been set it's time to launch QEMU:

  -device vhost-vsock-pci,vhostfd=<fd>,guest-cid=<guest-cid>

(Or using device_add hotplug.)

Note that it's necessary to pass the guest CID to QEMU, but that should be easy because libvirt already knows it.

Linux 4.13 and later have a static device number assigned to /dev/vhost-vsock so that the kernel auto-loads vhost_vsock.ko when the device node is opened for the first time.  Older Linux kernels require an explicit modprobe vhost-vsock (yes, just like /dev/vhost-net used to!).

Hope this is useful info for starting the libvirt work.  Please let me know if you have questions!
Comment 8 Ján Tomko 2018-05-21 11:39:13 EDT
Initial version of upstream patches:
https://www.redhat.com/archives/libvir-list/2018-May/msg01517.html
Comment 10 Ján Tomko 2018-05-30 02:39:28 EDT
Pushed upstream as of:
commit b8b42ca036adbfaac1741c8efe389cd1403e220b
Author:     Ján Tomko <jtomko@redhat.com>
AuthorDate: 2018-05-22 15:57:47 +0200
Commit:     Ján Tomko <jtomko@redhat.com>
CommitDate: 2018-05-29 15:42:04 +0200

    qemu: add support for vhost-vsock-pci
    
    Create a new vsock endpoint by opening /dev/vhost-vsock,
    set the requested CID via ioctl (or assign a free one if auto='yes'),
    pass the file descriptor to QEMU and build the command line.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1291851
    Signed-off-by: Ján Tomko <jtomko@redhat.com>

git describe: v4.3.0-372-gb8b42ca036
Comment 11 Ján Tomko 2018-05-30 03:25:52 EDT
Filed a bug against selinux-policy:
https://bugzilla.redhat.com/show_bug.cgi?id=1584011
Comment 12 Ján Tomko 2018-05-30 10:58:27 EDT
Follow-up series implementing hotplug:
https://www.redhat.com/archives/libvir-list/2018-May/msg02239.html
Comment 13 Ján Tomko 2018-06-01 07:37:14 EDT
And another series changing the element name from <source cid=''> to <cid address=''>:
https://www.redhat.com/archives/libvir-list/2018-June/msg00037.html
Comment 14 Ján Tomko 2018-06-01 08:34:25 EDT
The rename is now pushed:
commit 023ea2a86938a6ecb5323a561f422c4951c8bf39
Author:     Ján Tomko <jtomko@redhat.com>
AuthorDate: 2018-06-01 13:22:56 +0200
Commit:     Ján Tomko <jtomko@redhat.com>
CommitDate: 2018-06-01 14:31:19 +0200

    conf: rename <vsock><source> to <vsock><cid>
    
    To avoid the <source> vs. <target> confusion,
    change <source auto='no' cid='3'/> to:
    <cid auto='no' address='3'/>
    
    Signed-off-by: Ján Tomko <jtomko@redhat.com>
    Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
    Acked-by: Peter Krempa <pkrempa@redhat.com>
    Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

git describe: v4.4.0-rc1-12-g023ea2a869
Comment 15 Ján Tomko 2018-06-05 02:36:55 EDT
There was one more follow-up change before the release:
commit 8a7003f66944721ec391e13e65bbc5fdfdec3cea
Author:     Ján Tomko <jtomko@redhat.com>
CommitDate: 2018-06-04 21:42:40 +0200

    qemu: check for QEMU_CAPS_DEVICE_VHOST_VSOCK
    
    My commit b8b42ca added support for formatting the vsock
    command line without actually checking if it's supported.
    
    Add it to the per-device validation function.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1291851
    
    Reported-by: John Ferlan <jferlan@redhat.com>
    Signed-off-by: Ján Tomko <jtomko@redhat.com>
    Reviewed-by: Jiri Denemark <jdenemar@redhat.com>

git describe: v4.4.0-rc2-4-g8a7003f669 contains: v4.4.0~1

Backport of the hotplug series:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2018-June/msg00055.html
Comment 17 yafu 2018-06-14 06:26:55 EDT
Hi, Ján,

It reports error when i try to hotplug/coldplug vsock device to the guest with libvirt-4.4.0-2.el7.x86_64. Could you help to check whether the patch http://post-office.corp.redhat.com/archives/rhvirt-patches/2018-June/msg00055.html includes in the libvirt-4.4.0-2.el7.x86_64 or not please?
Thanks a lot.

Test steps:
1.Start a guest:
#virsh start iommu1

2.Prepare vsock device xml:
#cat vsock.xml
<vsock model='virtio'>
   <cid auto='yes'/>
</vsock>

3.Hotplug the vsock device to the guest:
#virsh attach-device iommu1 vsock.xml
error: Failed to attach device from /root/vsock.xml
error: Operation not supported: live attach of device 'vsock' is not supported

4.Coldplug the vsock device to the guest:
#virsh attach-device iommu1 vsock.xml --config
error: Failed to attach device from /root/vsock.xml
error: Operation not supported: persistent attach of device 'vsock' is not supported
Comment 18 Ján Tomko 2018-06-14 06:53:28 EDT
Hi,

libvirt-4.4.0-2.el7.x86_64 does not have the additional hotplug patches. Only what was picked up by rebase to 4.4.0.

Sorry for the confusion.

Note You need to log in before you can comment on or make changes to this bug.