Description of problem: vgamem_mb option was moved from QXL driver to qemu but neither vdsm/libvirt/engine takes care of this. As the result VDI is broken and can be used only with vdsm hooks which are not supported in RHEVH. See BZ#1140765 I think it's rather regression than RFE tbh Version-Release number of selected component (if applicable): qemu-kvm-rhev-0.12.1.2-2.479.el6_7.1.x86_64 How reproducible: 100%
There is some confusion and documentation deficit about video memory options, so let's start with their analysis to make things clearer (everything concerns just QXL and current versions of software, meanings of the same options for other drivers may be different): - Domain XML as produced by VDSM contains `ram' and `vram' attributes. - `ram' defines the primary bar memory size, in KB. `vram' defines secondary bar memory size, in KB. - libvirt transforms `ram' and `vram' attributes to ram_size and vram_size QEMU command line options (in bytes) respectively. - I couldn't find documentation of the QEMU command line options, but according to the source code they set primary bar memory size and secondary bar 32-bit surfaces memory size (no big surprise). Furthermore: - There is also `vgamem' attribute, in KB. libvirt describes it as "optional attribute to set the size of VGA framebuffer for fallback mode of QXL device". - VDSM doesn't set it so the corresponding value falls back to the libvirt's default value, which is 16 MB. - The corresponding QEMU command line option is vgamem_mb, in MB. Note that QEMU ensures that ram_size is set to at least 2 * vgamem_mb * MB. Looking into QEMU source code and poking around it seems to me that we need to set `vgamem', 32 MB is probably needed for single-headed very high resolutions. One option is to set it to "ram / 2" in VDSM (note that Engine adjusts `ram' according to the number of heads). Or we can compute `vgamem' in Engine and just pass it to domain XML in VDSM. Please note this is just my wild guess so if anybody has a real idea what we should set to what under which circumstances, please tell us.
After the "enable-resizable-surface0.patch" was removed from the xorg-x11-drv-qxl rpm as described in referenced BZ #1140765, higher resolutions stopped working on our RHEL 6 VDI's. Even clients with 2x 1920x1200 displays failed. Downgrading the xorg-x11-drv-qxl rpm re-enabled large displays, but as described in other BZ entries which led to it's removal, this patch brings other issues with it. So we kept the later rpm with the patch removed. At first I tried adding the vgamem attribute to the libvirt xml with another vdsm hook, however even though the attribute was added to the libvirt xml (as displayed in /var/vdsm/vdsm.log when the vm is started), the vgamem_mb arg was never added to the qemu-kvm command line. I worked around the issue by writing a vdsm hook using qemu cmd to add vgamem_mb=64 for all Linux VDI vm's. After the vgamem_mb was added, high resolutions started working again. I have not done much research/calculating of required size of vgamem_mb as this was a break-fix until it's properly fixed in the upstream code. I found other posts where a vgsmem_mb size of 32 was required for 2x 1920x1200. All of our Linux VDI vms are created with 4 SPICE displays, so the ram size is set to 256MB for each vm. We have to support clients having dual 30" monitors, where the resolution is 2 x 2560x1600. So I f tested with a value of 64 for vgamem_mb. 2x 1920x1200 started working fine in my own tests, and I had no further reports of any issues from users having 2x 2560x1600 with a value of 64.
Thank you for your information. As for vgamem_mb presence: I checked the QEMU command line option is present with the correct value in my setup when I set `vgamem' attribute in domain XML. This is with libvirt 1.2.8, we should be careful to check the option presence with other libvirt versions. As for the value: I was told that it should be something like WIDTH * HEIGHT * 4 * NUMBER-OF-DISPLAYS rounded to the nearest higher power of 2, i.e. indeed 32 MB for two 1920x1200 or 2560x1600 displays. I've got still no better info about ram_size and vram_size. I'm looking for an authoritative answer from QEMU/QXL developers, that should clarify it all.
Here is the information from a QEMU developer: - vgamem_mb should indeed be WIDTH * HEIGHT * 4 * NUMBER-OF-DISPLAYS, power of 2. - ram_size: minimum = 2 * vgamem_mb * MB; recommended = 4 * vgamem_mb * MB - vram_size: safe bet = ram_size; but it depends on the particular driver whether it is actually used and gives better performance or not - Using safe values larger than necessary may mean (depending on the particular driver) utilizing the corresponding amount of physical RAM; the driver may use it not only for current pixels but also for many other purposes when it is available. Now when we know what to set, we should decide how to handle it in Engine and VDSM. I'd say the values should be decided by Engine and just transferred to domain XML by VDSM. And let's not forget about checking whether libvirt downstream version transforms the values as expected.
Thank you for the information. Did you check the RHEV-H 6.7 image when you tested the vgamem attribute with libvirt? I used the latest 6.7 image available when I was writing and testing the hooks some weeks ago. I fully agree the values should be decided by the Engine and transferred to domain XML by VDSM. If I did the calculations correctly, this should expand to a vgamem_mb of 32 for up to 2 x 2560x1600 displays, and a vgamem_mb of 64 for up to 4 x 2560x1600 displays? The ram and vram size currently set by Engine allocates 64Mb of memory per configured display. This results in 128Mb for 2 displays and 256Mb for 4 displays. I believe these sizes should still be OK for the 2 and 4 display configurations as described by the QEMU developer above?
I didn't check it with RHEL 6.7, I'm trying to get confirmation about proper domain XML use from libvirt developers. I think your calculations are correct. Just vram may be different, at least current version of Engine seems to set it to 32 MB regardless of the number of heads (may or may not be a problem, no idea).
According to libvirt developers about libvirt domain XML: RHEL-7: What we claim above is correct and supported since RHEL 7.1. RHEL-6: While vgamem_mb QEMU option is available since RHEL 6.7, vgamem libvirt attribute is not and there are no plans to backport it. The qemu:commandline workaround may work, but it's completely unsupported and without any guarantees.
(In reply to Milan Zamazal from comment #8) > According to libvirt developers about libvirt domain XML: > > RHEL-7: What we claim above is correct and supported since RHEL 7.1. > > RHEL-6: While vgamem_mb QEMU option is available since RHEL 6.7, vgamem > libvirt attribute is not and there are no plans to backport it. The > qemu:commandline workaround may work, but it's completely unsupported and > without any guarantees. on the host side it doesn't matter, 3.6 supports RHEL/CentOS 7.2+ only
For the record, here is additional information about vnc/vga and vnc/cirrus: - The better choice of the two is clearly vnc/vga. - Multihead is not supported on any of the drivers. - Default vram_size, 16 MB, is fine for most common uses, except for full HD on Wayland (needs twice as much due to two framebuffers + page-flipping). - Again, RHEL-7 only.
Patches to Engine and VDSM are available for review. Based on all the available information and consideration, we are going to use the following settings: - QXL: vgamem = 16 MB * number_of_heads, ram = 4 * vgamem, vram = 2 * vgamem for RHEL-7 guest OS, vram = 8 MB for other guest systems - vnc/vga and vnc/cirrus: vram = 16 MB Things are likely to change in future, but this is what should work now and for 3.6.
Thank you. Will the vgamem attribute be set for rhel6 vms too?
(In reply to Sigbjorn Lie from comment #13) > Will the vgamem attribute be set for rhel6 vms too? Yes, it should be with (future) up-to-date RHEV-M + RHEV-H.
Thank you for your reply. I was referring to vms having rhel6 installed inside the vm. I had a quick glance at the patches and it seem like the vrammultiplier is set for rhel7 guests only, in the osinfo-defaults.properties file. Does this imply that rhel6 vms will not have the vgamem var calculated and set?
(In reply to Sigbjorn Lie from comment #15) > I had a quick glance at the patches and it seem like the vrammultiplier is > set for rhel7 guests only, in the osinfo-defaults.properties file. > > Does this imply that rhel6 vms will not have the vgamem var calculated and > set? No. The guest OS dependent multiplier applies to vram calculation as vram is of fixed size in RHEL-6 and dependent on vgamem in RHEL-7. ram and vgamem values are guest OS independent and are also set in the patch, according to the formulas in comment #12.
(In reply to Milan Zamazal from comment #16) > (In reply to Sigbjorn Lie from comment #15) > > I had a quick glance at the patches and it seem like the vrammultiplier is > > set for rhel7 guests only, in the osinfo-defaults.properties file. > > > > Does this imply that rhel6 vms will not have the vgamem var calculated and > > set? > > No. The guest OS dependent multiplier applies to vram calculation as vram is > of fixed size in RHEL-6 and dependent on vgamem in RHEL-7. ram and vgamem > values are guest OS independent and are also set in the patch, according to > the formulas in comment #12. Excellent. I guess that's what reviewing code on my phone get's me. :)
Everything has been merged to 3.6 branch.
Verified with rhevm-3.6.3-0.1.el6.noarch. Tested changes in video entity values of vm's xml according to setting in engine. tested the following matrix: guest OS: RHEL7/RHEL6. display type: QXL/VNC-Cirrus single PCI: true/false # of monitors: 1/2/4 All values in the guest's xml match the calculation defined in this patch.
Works with gotchas: * VMs have to be reconfigured for change to take effect (at least for el7 VMs): 4 monitors -> 1 monitor -> 4 monitors again * 4 2560x1600 monitors can't fit into maximum screen size of 8192x8192 side-by-side, they have to be arranged into a 2x2 matrix otherwise there will be overlaps (same screen area will be displayed by more monitors...) So let's keep this bug verified but more work will still be needed to ensure really smooth experience.
(In reply to David Jaša from comment #25) Thanks for testing. > Works with gotchas: > * VMs have to be reconfigured for change to take effect (at least for el7 > VMs): > 4 monitors -> 1 monitor -> 4 monitors again I think this could be solved within https://bugzilla.redhat.com/1302664. > * 4 2560x1600 monitors can't fit into maximum screen size of 8192x8192 > side-by-side, they have to be arranged into a 2x2 matrix otherwise there > will > be overlaps (same screen area will be displayed by more monitors...) Good point. Not sure what to do about that (documentation?).
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0376.html
*** Bug 1275297 has been marked as a duplicate of this bug. ***
*** Bug 1324896 has been marked as a duplicate of this bug. ***
Thanks for wrapping this BZ up! With this being resolved in RHEV-M 3.6, is it required to have RHEL 7 hosts for this functionality or is this something that can be obtained on RHEV 3.6 with RHEL 6 hosts and RHEV-M in 3.4 compatibility mode? The errata isn't clear on this and I'd like to have a better understanding of where the fix actually lies.
Hi Robert, the functionality added in this bug is set to work on 3.6 and higher compatibility modes, therefore with RHEL/CentOS 7.2 and higher only.
Sefi, thanks for the quick turnaround and a succinct answer. That resolves it for my customer.