Bug 1275539 - Allocate memory for high resolutions/multihead
Allocate memory for high resolutions/multihead
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.5.4
Unspecified Unspecified
high Severity high
: ovirt-3.6.3
: 3.6.3
Assigned To: Milan Zamazal
sefi litmanovich
:
: 1275297 1324896 (view as bug list)
Depends On: 1076098
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-27 04:11 EDT by Pavel Zhukov
Modified: 2017-03-16 09:11 EDT (History)
20 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Video memory sizes in previous Red Hat Enterprise Virtualization versions were not up to date with the common contemporary screen setups and current state of graphics drivers. As a result, the allocated video memory could be insufficient for high resolutions and/or multihead setups. Now, video memory sizes have been updated to handle typical screen arrangements for all supported screen resolutions in 3.6 clusters. If those settings still do not work well for special requirements, they can be adjusted by advanced users using VDSM hooks, newly including the ‘vgamem’ attribute of the <video> domain XML element.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-09 16:14:04 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2971281 None None None 2017-03-16 09:11 EDT
oVirt gerrit 50091 master MERGED virt: Add support for vgamem attribute Never
oVirt gerrit 50092 master MERGED core: Video RAM size settings reworked 2016-01-11 05:54 EST
oVirt gerrit 50507 ovirt-3.6 MERGED virt: Add support for vgamem attribute 2015-12-17 03:26 EST
oVirt gerrit 51660 ovirt-engine-3.6 MERGED core: Video RAM size settings reworked 2016-01-13 05:40 EST

  None (edit)
Description Pavel Zhukov 2015-10-27 04:11:33 EDT
Description of problem:
vgamem_mb option was moved from QXL driver to qemu but neither vdsm/libvirt/engine takes care of this. As the result VDI is broken and can be used only with vdsm hooks which are not supported in RHEVH. See BZ#1140765
I think it's rather regression than RFE tbh

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-0.12.1.2-2.479.el6_7.1.x86_64


How reproducible:
100%
Comment 2 Milan Zamazal 2015-11-23 09:08:40 EST
There is some confusion and documentation deficit about video memory options,
so let's start with their analysis to make things clearer (everything concerns
just QXL and current versions of software, meanings of the same options for
other drivers may be different):

- Domain XML as produced by VDSM contains `ram' and `vram' attributes.
- `ram' defines the primary bar memory size, in KB.  `vram' defines secondary
  bar memory size, in KB.
- libvirt transforms `ram' and `vram' attributes to ram_size and vram_size QEMU
  command line options (in bytes) respectively.
- I couldn't find documentation of the QEMU command line options, but
  according to the source code they set primary bar memory size and secondary
  bar 32-bit surfaces memory size (no big surprise).

Furthermore:

- There is also `vgamem' attribute, in KB.  libvirt describes it as "optional
  attribute to set the size of VGA framebuffer for fallback mode of QXL
  device".
- VDSM doesn't set it so the corresponding value falls back to the libvirt's
  default value, which is 16 MB.
- The corresponding QEMU command line option is vgamem_mb, in MB.  Note that
  QEMU ensures that ram_size is set to at least 2 * vgamem_mb * MB.

Looking into QEMU source code and poking around it seems to me that we need to
set `vgamem', 32 MB is probably needed for single-headed very high resolutions.
One option is to set it to "ram / 2" in VDSM (note that Engine adjusts `ram'
according to the number of heads).  Or we can compute `vgamem' in Engine and
just pass it to domain XML in VDSM.

Please note this is just my wild guess so if anybody has a real idea what we
should set to what under which circumstances, please tell us.
Comment 3 Sigbjorn Lie 2015-11-23 10:25:26 EST
After the "enable-resizable-surface0.patch" was removed from the xorg-x11-drv-qxl rpm as described in referenced BZ #1140765, higher resolutions stopped working on our RHEL 6 VDI's. Even clients with  2x 1920x1200 displays failed.

Downgrading the xorg-x11-drv-qxl rpm re-enabled large displays, but as described in other BZ entries which led to it's removal, this patch brings other issues with it. So we kept the later rpm with the patch removed.

At first I tried adding the vgamem attribute to the libvirt xml with another vdsm hook, however even though the attribute was added to the libvirt xml (as displayed in /var/vdsm/vdsm.log when the vm is started), the vgamem_mb arg was never added to the qemu-kvm command line. I worked around the issue by writing a vdsm hook using qemu cmd to add vgamem_mb=64 for all Linux VDI vm's.

After the vgamem_mb was added, high resolutions started working again.

I have not done much research/calculating of required size of vgamem_mb as this was a break-fix until it's properly fixed in the upstream code.  I found other posts where a vgsmem_mb size of 32 was required for 2x 1920x1200. All of our Linux VDI vms are created with 4 SPICE displays, so the ram size is set to 256MB for each vm. We have to support clients having dual 30" monitors, where the resolution is 2 x 2560x1600. So I f tested with a value of 64 for vgamem_mb. 2x 1920x1200 started working fine in my own tests, and I had no further reports of any issues from users having 2x 2560x1600 with a value of 64.
Comment 4 Milan Zamazal 2015-11-24 04:37:01 EST
Thank you for your information.

As for vgamem_mb presence: I checked the QEMU command line option is present
with the correct value in my setup when I set `vgamem' attribute in domain XML.
This is with libvirt 1.2.8, we should be careful to check the option presence
with other libvirt versions.

As for the value: I was told that it should be something like 
WIDTH * HEIGHT * 4 * NUMBER-OF-DISPLAYS rounded to the nearest higher 
power of 2, i.e. indeed 32 MB for two 1920x1200 or 2560x1600 displays.
I've got still no better info about ram_size and vram_size.  I'm looking for an
authoritative answer from QEMU/QXL developers, that should clarify it all.
Comment 5 Milan Zamazal 2015-11-24 05:08:44 EST
Here is the information from a QEMU developer:

- vgamem_mb should indeed be WIDTH * HEIGHT * 4 * NUMBER-OF-DISPLAYS, power
  of 2.
- ram_size: minimum = 2 * vgamem_mb * MB; recommended = 4 * vgamem_mb * MB
- vram_size: safe bet = ram_size; but it depends on the particular driver
  whether it is actually used and gives better performance or not
- Using safe values larger than necessary may mean (depending on the particular
  driver) utilizing the corresponding amount of physical RAM; the driver may
  use it not only for current pixels but also for many other purposes when it
  is available.

Now when we know what to set, we should decide how to handle it in Engine and
VDSM.  I'd say the values should be decided by Engine and just transferred to
domain XML by VDSM.  And let's not forget about checking whether libvirt
downstream version transforms the values as expected.
Comment 6 Sigbjorn Lie 2015-11-24 06:40:42 EST
Thank you for the information.

Did you check the RHEV-H 6.7 image when you tested the vgamem attribute with libvirt? I used the latest 6.7 image available when I was writing and testing the hooks some weeks ago.

I fully agree the values should be decided by the Engine and transferred to domain XML by VDSM. 

If I did the calculations correctly, this should expand to a vgamem_mb of 32 for up to 2 x 2560x1600 displays, and a vgamem_mb of 64 for up to 4 x 2560x1600 displays?

The ram and vram size currently set by Engine allocates 64Mb of memory per configured display. This results in 128Mb for 2 displays and 256Mb for 4 displays. I believe these sizes should still be OK for the 2 and 4 display configurations as described by the QEMU developer above?
Comment 7 Milan Zamazal 2015-11-25 04:27:01 EST
I didn't check it with RHEL 6.7, I'm trying to get confirmation about proper domain XML use from libvirt developers.

I think your calculations are correct.  Just vram may be different, at least current version of Engine seems to set it to 32 MB regardless of the number of heads (may or may not be a problem, no idea).
Comment 8 Milan Zamazal 2015-11-25 07:05:07 EST
According to libvirt developers about libvirt domain XML:

RHEL-7: What we claim above is correct and supported since RHEL 7.1.

RHEL-6: While vgamem_mb QEMU option is available since RHEL 6.7, vgamem libvirt attribute is not and there are no plans to backport it.  The qemu:commandline workaround may work, but it's completely unsupported and without any guarantees.
Comment 9 Michal Skrivanek 2015-11-25 09:19:45 EST
(In reply to Milan Zamazal from comment #8)
> According to libvirt developers about libvirt domain XML:
> 
> RHEL-7: What we claim above is correct and supported since RHEL 7.1.
> 
> RHEL-6: While vgamem_mb QEMU option is available since RHEL 6.7, vgamem
> libvirt attribute is not and there are no plans to backport it.  The
> qemu:commandline workaround may work, but it's completely unsupported and
> without any guarantees.

on the host side it doesn't matter, 3.6 supports RHEL/CentOS 7.2+ only
Comment 10 Milan Zamazal 2015-11-26 04:02:24 EST
For the record, here is additional information about vnc/vga and vnc/cirrus:

- The better choice of the two is clearly vnc/vga.
- Multihead is not supported on any of the drivers.
- Default vram_size, 16 MB, is fine for most common uses, except for full HD on Wayland (needs twice as much due to two framebuffers + page-flipping).
- Again, RHEL-7 only.
Comment 12 Milan Zamazal 2015-12-08 12:02:18 EST
Patches to Engine and VDSM are available for review.

Based on all the available information and consideration, we are going to use the following settings:

- QXL:

  vgamem = 16 MB * number_of_heads,
  ram = 4 * vgamem,
  vram = 2 * vgamem for RHEL-7 guest OS,
  vram = 8 MB for other guest systems

- vnc/vga and vnc/cirrus:

  vram = 16 MB

Things are likely to change in future, but this is what should work now and for 3.6.
Comment 13 Sigbjorn Lie 2015-12-10 13:27:18 EST
Thank you. Will the vgamem attribute be set for rhel6 vms too?
Comment 14 Milan Zamazal 2015-12-11 05:20:02 EST
(In reply to Sigbjorn Lie from comment #13)
> Will the vgamem attribute be set for rhel6 vms too?

Yes, it should be with (future) up-to-date RHEV-M + RHEV-H.
Comment 15 Sigbjorn Lie 2015-12-11 14:19:34 EST
Thank you for your reply. I was referring to vms having rhel6 installed inside the vm.

I had a quick glance at the patches and it seem like the vrammultiplier is set for rhel7 guests only, in the osinfo-defaults.properties file.

Does this imply that rhel6 vms will not have the vgamem var calculated and set?
Comment 16 Milan Zamazal 2015-12-11 14:42:23 EST
(In reply to Sigbjorn Lie from comment #15)
> I had a quick glance at the patches and it seem like the vrammultiplier is
> set for rhel7 guests only, in the osinfo-defaults.properties file.
> 
> Does this imply that rhel6 vms will not have the vgamem var calculated and
> set?

No. The guest OS dependent multiplier applies to vram calculation as vram is of fixed size in RHEL-6 and dependent on vgamem in RHEL-7. ram and vgamem values are guest OS independent and are also set in the patch, according to the formulas in comment #12.
Comment 17 Sigbjorn Lie 2015-12-11 16:03:14 EST
(In reply to Milan Zamazal from comment #16)
> (In reply to Sigbjorn Lie from comment #15)
> > I had a quick glance at the patches and it seem like the vrammultiplier is
> > set for rhel7 guests only, in the osinfo-defaults.properties file.
> > 
> > Does this imply that rhel6 vms will not have the vgamem var calculated and
> > set?
> 
> No. The guest OS dependent multiplier applies to vram calculation as vram is
> of fixed size in RHEL-6 and dependent on vgamem in RHEL-7. ram and vgamem
> values are guest OS independent and are also set in the patch, according to
> the formulas in comment #12.

Excellent. I guess that's what reviewing code on my phone get's me. :)
Comment 20 Milan Zamazal 2016-01-14 06:23:19 EST
Everything has been merged to 3.6 branch.
Comment 21 sefi litmanovich 2016-01-28 11:25:19 EST
Verified with rhevm-3.6.3-0.1.el6.noarch.

Tested changes in video entity values of vm's xml according to setting in engine.
tested the following matrix:

guest OS: RHEL7/RHEL6.
display type: QXL/VNC-Cirrus
single PCI: true/false
# of monitors: 1/2/4

All values in the guest's xml match the calculation defined in this patch.
Comment 25 David Jaša 2016-02-19 12:27:31 EST
Works with gotchas:
  * VMs have to be reconfigured for change to take effect (at least for el7 VMs):
    4 monitors -> 1 monitor -> 4 monitors again
  * 4 2560x1600 monitors can't fit into maximum screen size of 8192x8192
    side-by-side, they have to be arranged into a 2x2 matrix otherwise there will
    be overlaps (same screen area will be displayed by more monitors...)

So let's keep this bug verified but more work will still be needed to ensure really smooth experience.
Comment 26 Milan Zamazal 2016-02-23 04:08:17 EST
(In reply to David Jaša from comment #25)

Thanks for testing.

> Works with gotchas:
>   * VMs have to be reconfigured for change to take effect (at least for el7
> VMs):
>     4 monitors -> 1 monitor -> 4 monitors again

I think this could be solved within https://bugzilla.redhat.com/1302664.

>   * 4 2560x1600 monitors can't fit into maximum screen size of 8192x8192
>     side-by-side, they have to be arranged into a 2x2 matrix otherwise there
> will
>     be overlaps (same screen area will be displayed by more monitors...)

Good point. Not sure what to do about that (documentation?).
Comment 28 errata-xmlrpc 2016-03-09 16:14:04 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0376.html
Comment 29 Fabiano Fidêncio 2016-03-15 12:28:58 EDT
*** Bug 1275297 has been marked as a duplicate of this bug. ***
Comment 30 Michal Skrivanek 2016-04-14 03:42:27 EDT
*** Bug 1324896 has been marked as a duplicate of this bug. ***
Comment 31 Robert McSwain 2016-05-18 12:51:32 EDT
Thanks for wrapping this BZ up! With this being resolved in RHEV-M 3.6, is it required to have RHEL 7 hosts for this functionality or is this something that can be obtained on RHEV 3.6 with RHEL 6 hosts and RHEV-M in 3.4 compatibility mode? The errata isn't clear on this and I'd like to have a better understanding of where the fix actually lies.
Comment 32 sefi litmanovich 2016-05-19 04:10:26 EDT
Hi Robert, the functionality added in this bug is set to work on 3.6 and higher compatibility modes, therefore with RHEL/CentOS 7.2 and higher only.
Comment 33 Robert McSwain 2016-05-19 11:48:12 EDT
Sefi, thanks for the quick turnaround and a succinct answer. That resolves it for my customer.

Note You need to log in before you can comment on or make changes to this bug.