Bug 2030592 - Keep QXL/SPICE support for RHEL 9 [NEEDINFO]
Summary: Keep QXL/SPICE support for RHEL 9
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: qemu-kvm
Version: 9.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Uri Lublin
QA Contact: Guo, Zhiyi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-09 08:48 UTC by Rik Theys
Modified: 2023-05-25 09:31 UTC (History)
31 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-20 07:53:40 UTC
Type: Bug
Target Upstream Version:
steven.w-ctr.mercurio: needinfo? (uril)
steven.w-ctr.mercurio: needinfo? (mtessun)
steven.w-ctr.mercurio: needinfo? (lethalwp)
steven.w-ctr.mercurio: needinfo? (mdean)
steven.w-ctr.mercurio: needinfo? (coli)
steven.w-ctr.mercurio: needinfo? (kdreyer)
steven.w-ctr.mercurio: needinfo? (jinzhao)
steven.w-ctr.mercurio: needinfo? (juzhang)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-105223 0 None None None 2021-12-09 08:53:48 UTC

Description Rik Theys 2021-12-09 08:48:36 UTC
Description of problem:

It seems RHEL 9 will no longer have support for QXL and spice console access. It seems it was deprecated with RHEL 8.3 and dropped in 9.0.

No reason why this was done is mentioned and I was unable to find the reason in other threads online.

We currently use oVirt to provide VDI access to students. Students need the ability to redirect USB devices to the virtual machines. This is supported by SPICE but not with VNC.

Spice graphics are also much more performant than VNC, so we are surprised this is being dropped.

What is the reason this support is being dropped? We would like Red Hat to reconsider this deprecation and continue to support QXL/SPICE in RHEL 9.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 John Ferlan 2021-12-09 11:40:52 UTC
Assigned to Meirav since her teams owns SPICE, but added Martin in a needinfo to give the product management perspective on the above.

Comment 2 lethalwp 2021-12-13 08:19:12 UTC
Adding my note:

I use virt-manager with spice every day for my work. I updated my laptop to centos9 stream and it broke my work setup. I was very surprised on this deprecated feature, removed with no explanation.
Maybe Centos9stream isn't the OS that i need?


Luckily i have another working pc using fedora. I don't understand why spice is getting dropped while it is shipped & usable in fedora 35. I thought Centos was a "more stable" fork from fedora.
It seems it isn't the case.

Comment 3 Mai Ling 2022-05-20 14:26:19 UTC
Hello Redhat, any update on this?

Comment 4 Martin Tessun 2022-06-20 07:53:40 UTC
So in short:

SPICE has been deprecated for qemu and has been removed in RHEL 9. We still do have SPICE client support for RHV - which is shipped in the RHV-Tools repos on RHEL 9.
Unfortunately I don't know if and how oVirt is solving this for RHEL 9. Anyways the RHV-tools repo will be supported until RHV is EOLed in 2026.

Why no SPICE:
1. Due to some license restrictions around H.264 codecs we are not able to provide a Streaming solution which is needed for modern workloads (as all drivers are using 3D today). Also vGPU support is not possible without this.
2. There are quite some 3rd party solutions that already have a proper implementation (licensed software due to H.264 restrictions unfortunately).
3. For console access and local acceleration VNC is capable of doing the job - and VNC is also used for OpenStack and KubeVirt and they are both not interested in using SPICE instead.

Mainly due to (1) and (3) we decided to deprecate our work on SPICE.

Comment 5 Andras Kovacs 2022-08-24 12:22:51 UTC
For the folks who just ended up here like me, one possible solution is:

REMMINA
https://flathub.org/apps/details/org.remmina.Remmina
It still has spice support.
In case you want to find/forward the port from a hypervisor where spice is listening on:
ps ax| grep vm-name | grep -o "spice port=[0-9]*,"

This is really sad. I'm trying to understand the reasons, but I just can't.
I'm pretty sure many people will use different hypervisor solutions because of this single decision. :(
SPICE was superior.

Comment 6 Ken Dreyer (Red Hat) 2022-09-06 18:31:02 UTC
I set up some Windows builder VMs this week and ran across this bug. Spice works fine on my Fedora 35 laptop, and I use it to copy & paste large commands. For example, installing and starting OpenSSH in Windows is more complicated than Linux. Modern MS Windows Server OSes are very slimmed down, and Windows Server admins expect to do more CLI work now. Typing long commands into virt-viewer character-by-character is painful.

What packages do we need to support Spice server-side in RHEL 9? Maybe we could maintain those in a CentOS Stream SIG.

Comment 7 static 2022-09-08 04:47:37 UTC
That would be awesome if possible...  I had to go with RHEL 8 because of SPICE removal in RHEL 9.  USB redirection, sound, etc worked great with spice even over a remote connections with Virtual Manager/SPICE.  The nice thing about SPICE also is that can be done directly to the host itself (you don't have to route to the VM network side of things which can be problematic depending on network design/security).  I really hope someone turns this around and gets SPICE back into qemu (QXL driver, etc).  Even if it is an alternative SIG/repo, that would be better than nothing.  SPICE works really well for desktop workstation use for devs and IT.

I found an article below (with a lot of time and searching trying to find the info) to at least get clipboard and auto-resize working.  I couldn't find this info on any redhat articles.  It appears the VNC driver has those functions at least on RHEL 9.  Without easy USB redirection as well as sound though, it wasn't enough for the needs where I work.

At a minimum, redhat should document the info at the bottom of that article explaining that clipboard.  I think the qemu config addition at the bottom of the linked webpage below should be set by default for windows installs using libvirt/KVM/Virtual Manager on RHEL 9.  It would at least help some that don't need the SPICE USB redirection, SPICE remote sound, or management using SPICE/VirtualManager.
https://www.reddit.com/r/qemu_kvm/comments/sytgvd/clipboard_without_spice/

Comment 8 static 2022-09-09 23:33:53 UTC
I found from my testing with a win11 VM that qemu-vdagent cannot be set in the actual XML as a controller,etc yet.  qemu-vdagent can't be parsed using virsh edit.  There is work upstream that seems to have that implemented but redhat 9 doesn't have it yet.  There is even work on VirtualManager this past month to allow qemu-vdagent to be set using the GUI.  That also is not part of RHEL 9 (yet?).  You have to use <qemu:commandline> tags manually to get it working through libvirt (make sure to add the xmls at the top of the XML file or the validator won't be able to parse the <qemu...> tag).

I also can not get dynamic screen resolution working.  I suspect that is just not a thing for qemu-vdagent.

And finally, I could not get copy paste to work until I found a post on a mailing list somewhere (after much searching) that mentioned that virt-viewer does not support copy paste so you have to use realvnc package executable (vncviewer) with qemu-vdagent.  I figured VirtualManager might be the same so I connected to the qemu instance with the realvnc vncviewer and sure enough... copy paste works.  I could never get it to work with a lot of effort with VirtualManager so all that time I thought there was a problem with how I was setting it up with the <qemu..> tags but since realvnc works, it must just be a problem with VirtualManager.

This is some things that I think need to be in place for qemu-vdagent to be a good easy solution for copy/paste(that used to use spice) on RHEL 9.  It does not solve the dynamic screen resizing, easy USB redirection (through SSH even), and sound that SPICE provided though.

1. libvirt needs to be updated (or backported feature) so know about the qemu-vdagent.
2. VirtualManager needs to be able to setup a qemu-vdagent (similar to the spicevmc).
3. Need to figure out why VirtualManager doesn't work with copy/paste but the rhel9 realvnc rpm package (vncviewer executable) does.

Comment 9 static 2022-09-23 00:25:59 UTC
Correction.  I meant to say tigervnc rpm package supports copy paste with newish VNC qemu clipboard feature instead of realvnc (realvnc third party app doesn't work with it).  Virtual Manager gui on RHEL doesn't seem to support the qemu clipboard feature... or at least doesn't work with it.

Comment 10 Joshua Kramer 2022-09-25 02:51:14 UTC
@agibson2 I wonder who would know what all packages we would need to rebuild for EL9 to provide full Spice support (i.e. both in guest drivers for VM's themselves, and the qemu infrastructure drivers, and the bits needed to re-integrate Spice with virt-manager?

Comment 11 David Hansen 2022-09-25 10:15:23 UTC
Another thing to keep is dual screens/monitors.  Spice was able to add an additional QXL display, but changing the number of heads in the XML in the adapter doesn't appear to work with VNC in this build.

Comment 12 static 2022-09-25 15:30:52 UTC
I don't have specific knowledge of what all was done to remove it but from what I read...  The XQL video driver would need to be compiled for the kernel, Not sure what USB redirection stuff would need to be put back in, qemu would need to have the patches removed that remove SPICE support(?).  Currently, SPICE is still in the GUI (Virtual Manager) to be able to add SPICE stuff, but hopefully they don't remove that stuff too in the future.
I have a feeling this would be a popular project to do something like this.

I found this bugzilla about removal of SPICE and they list things that might have been done to remove SPICE support...

https://bugzilla.redhat.com/show_bug.cgi?id=1946939

"Description of problem:
SPICE has been deprecated in RHEL-8.3 and is to be removed from RHEL-9.

Please:
- Disable support for SPICE components (graphics, smart-card, USB, etc)
- Remove SPICE components from the default configuration (including QXL)
- Remove "Requires: spice-gtk3" in virt-manager.spec"

I don't know that all of that was done though.  It would be great to get input from the developers that removed it so that any project to put it back would be much easier.  I am not sure who has the skills to do such thing though.  I have seen this appear on the Rocky Linux chat and forums too wanting a SIG for putting it back in.

Comment 13 Ladar Levison 2022-11-18 17:31:05 UTC
My virtualized development machines are almost unusable with VNC on RHEL 9. As a result, I decided to rebuild the packages myself, so I could enable SPICE support. It felt like the like the superior alternative, with said alternative being a commercial/propitiatory solution.

I'm not an expert, but I was wondering if Red Hat couldn't simply use the VP 8 / VP 9 codec(s) to for streaming, and avoid the H264 licensing issue? 

Either way, anybody else facing this same problem can use my RPMs (or use the included script to build their own). The files are available here:

https://github.com/ladar/qemu-spice-el9

If there is enough interest, I can setup a DNF repo in the future.

Comment 14 static 2022-11-18 20:25:35 UTC
I was going with RHEL8 because of this but I installed a RHEL9 to keep experimenting with it to see how close I could get to a good experience on RHEL 9 and libvirt/qemu/VirtualManager but it has a lot of issue.

Spice USB redirection was very easy to use on previous RHEL versions.  There is native host USB passthrough in RHEL9 but it is a bit troublesome to use.  Adding a USB host device to the VM is relatively easy but then the host requires it to be running and unplugging and plugging back in the device, the VM doesn't always see it.  I never had that issue with SPICE usb redirection.

Audio is another huge issue.  With spice on previous RHEL versions, audio worked great.  You now have to manually figure out how to connect the VM audio to jack or pulseaudio which after lots of struggling, I still haven't been successful and seems to require manual XML edits to even attempt it.  I tried a workaround of using a USB headset and USB Host passthrough to listen to audio but that has the problems mentioned above where sometimes plugging and unplugging it, it stops working.  Using the native audio hardware though would be much preferred anyway.

Then to work around copy-paste issue, I had to do lots of searching to find manual XML edits to get that working and that only works with tigervnc VNC client so the native Virtual Manager doesn't support qemu's native copy-paste feature.

All of this just adds up to a very bad experience for my use case compared to previous RHEL versions that supported SPICE.

Comment 15 Michael Lipp 2023-02-28 16:19:22 UTC
(In reply to Ladar Levison from comment #13)
> Either way, anybody else facing this same problem can use my RPMs (or use
> the included script to build their own). The files are available here:
> 
> https://github.com/ladar/qemu-spice-el9
> 
> If there is enough interest, I can setup a DNF repo in the future.

I've tested it (AlmaLinux 9.1). Works, but you get the feeling that half of the system has been replaced (several hundred packages), so this rises some security issues. And I assume that maintaining the packages will be an effort of its own (I'm impressed by your work).

I've just tested to run Fedora libvirtd in a container as outlined here (https://projectatomic.io/blog/2014/10/libvirtd_in_containers/). Although this is from 2014 it basically still works (VM runs, sound and USB redirect work, some minor issues remain to be clarified). I think this will be my way to go (and if RedHat drops spice support from Fedora as well, I can switch to some other base system for libvirtd).

Remote access to my Workstation (running as a VM) over the internet is a major use case for running my server in the first place and I'm just setting up a cluster to make this available for others as well. So dropping the support for this just now is really bad timing. And I still don't get the reasons. I have found no indication that the qemu project is going to drop spice support. So why does RedHat deliberately cripple a great solution?

Comment 16 Ladar Levison 2023-03-01 17:52:29 UTC
(In reply to Michael Lipp from comment #15)
> 
> I've tested it (AlmaLinux 9.1). Works, but you get the feeling that half of
> the system has been replaced (several hundred packages), so this rises some
> security issues. And I assume that maintaining the packages will be an
> effort of its own (I'm impressed by your work).

In terms of maintaining it, I've automated the entired process. I run one (albeit very long) command, and 2-3 hours later, I have a set of updated RPMs. The trick is knowing when a rebuild is necessary, and then tolerating all the huffing and puffing as my middle aged laptop does the work.

The _total_ number of packages in the git repo is rather inflated, since it includes all of the source/debug/debugsource packages, plus some of the dependencies which aren't available from the default repos (or EPEL). I include those so users can won't need to install/setup a new DNF repo just to install these packages.

As for your other concerns. they have also crossed my mind, especially given that I have started rebuilding the libvirt packages as well. Which is why, I only plan to use the rebuilt packages on my workstations, I plan on sticking with the official RPMs on my servers, where stability/security take precendence. At least for now, as the servers are rarely run graphical applications of any kind. Plus, my servers don't have discrete GPUs, so the impetus for needing SPICE is missing.

Long term I'm hoping RH will realize how many customers (and developers) they're hurting, and find a long term solution which allows for accelerated virtualized environments, and included with a point release.

If not, then perhaps a SiG will need to form which can maintain an alternative package set, like the CentOS advanced virt SiG did once upon a time.
 
> I've just tested to run Fedora libvirtd in a container as outlined here
> (https://projectatomic.io/blog/2014/10/libvirtd_in_containers/). Although
> this is from 2014 it basically still works (VM runs, sound and USB redirect
> work, some minor issues remain to be clarified). I think this will be my way
> to go (and if RedHat drops spice support from Fedora as well, I can switch
> to some other base system for libvirtd).

Part of the reason I have to rebuild so many packages is because I'm actually rebuilding the Fedora packages, which provide newer versions. Since Fedora provides more functionality, that means more dependencies which aren't available on RHEL (or the RHEL version isn't new enough). 

I believe someone else forked my work, and used it to created a less ambitious variant which only recompiles a handful of RHEL packagers with SPICE support. 

> Remote access to my Workstation (running as a VM) over the internet is a
> major use case for running my server in the first place and I'm just setting
> up a cluster to make this available for others as well. So dropping the
> support for this just now is really bad timing. And I still don't get the
> reasons. I have found no indication that the qemu project is going to drop
> spice support. So why does RedHat deliberately cripple a great solution?

It's because RH is focused on containers, and Kubernetes, and OpenStack (in my opinion) and not on graphical desktops (aka workstations), or virtualization. The cloud is where their business is focused right now, and unfortunately, it led them to drop a lot of critical tools/packages in v9 that developers/power users with GUI Workstations running RHEL depend on. Supporting those packages isn't critical for cloud/corporate/container users/customers.

Comment 17 Michael Lipp 2023-03-02 20:20:27 UTC
(In reply to Michael Lipp from comment #15)
> I think this will be my way to go 

See https://github.com/mnlipp/libvirt-container

Comment 18 Paul Bransford 2023-03-20 15:47:29 UTC
(In reply to static from comment #14)
> Audio is another huge issue.  With spice on previous RHEL versions, audio worked great.  You now have to manually figure out how to connect the VM audio to jack or pulseaudio which after lots of struggling, I still haven't been successful and seems to require manual XML edits to even attempt it. I tried a workaround of using a USB headset and USB Host passthrough to listen to audio but that has the problems mentioned above where sometimes plugging and unplugging it, it stops working.  Using the native audio hardware though would be much preferred anyway.

This is the blocker, for me. For my use case audio with sane latency is a requirement. With SPICE getting yanked out, I'll have to <shudder> go back to VirtualBox or VMWare Workstation. I'm not happy about that.

Unfortunately, I'm a self-support user - but I thought I'd throw my $0.02 in here.

----

I don't understand why H.264 licensing has anything to do with this - that specific codec is a hard requirement for the protocol? That seems like a bad architectural decision, if true. There's plenty of open-source friendly A/V codecs and containers out there.

Comment 19 Fanky W 2023-04-08 12:24:02 UTC
Seems like the commercial incentive to develop better codec doesn't exist anymore. I wonder if ZFS/btrfs is stuck a similar situation. RHEL can be a more versatile solution with open transcoding, vGPU, and ZFS.

Comment 20 Aram Minasyan 2023-04-11 22:08:37 UTC
I have built rpm packages in my Copr project -  https://copr.fedorainfracloud.org/coprs/aminasyan/el9-collection
Based on the github fork of qemu-spice-el9  https://github.com/corrmaan/qemu-spice-el9
Feel free to use or rebuild from the spec/srpm.

Comment 21 Jean-Marc Liger 2023-04-27 20:19:41 UTC
I rebuilt the QEMU KVM stack only with SPICE enabled for our own purposes:
https://copr.fedorainfracloud.org/coprs/ligenix/enterprise-qemu-spice/

Comment 22 Steven Mercurio 2023-05-23 15:47:46 UTC
This basically KILLS RHEL/RHV as a VDI solution 100%.

How can you do PIV card logins if you can not dynamically redirect the USB card reader???  On a dedicated desktop you can try to manually add the USB device but if the user unplugs it and plugs it into a different USB port????


Don't care about the video.  If the video is all that was hitting the license issue then just take it out.  Even sound is not a must have BUT *DYNAMIC* USB redirection is an ABSOLUTE BLOCKER.


Is there any way to just start with bringing back USP redirection???   Maybe by creating a virtual mapper device node on the KVM server and using redir to redirect one TCP port for each USB device then on the client create a virtual dev device for each USB device that channels the USB into redir and over to the server virtual device that gets auto-mapped to the VM???

USB is just unfortunately too essential these dats.  Smart card readers, printers, USB drives, etc.  The clipboard support should just be in VNC but even that is not as essential as USB.


Note You need to log in before you can comment on or make changes to this bug.