Bug 1076098

Some patches have been posted to the ML:
http://www.redhat.com/archives/libvir-list/2014-July/msg00644.html

Gerd, This will also allow to configure >16mb framebuffer/surface0 with QXL device, right? Then I suggest to make the bug title a bit more generic (add vgamem_mb parameter?)

It would be nice to have it backported in rhel6 too

(In reply to Marc-Andre Lureau from comment #2)
> Some patches have been posted to the ML:
> http://www.redhat.com/archives/libvir-list/2014-July/msg00644.html
> 
> Gerd, This will also allow to configure >16mb framebuffer/surface0 with QXL
> device, right?

Yes, the series posted does it for stdvga,vmware-svga,qxl,secondary-vga.
vmware-svga is not supported on rhel though.

This bug prevents use of supported configuration: RHEL 7 guest with 4 Full HD monitors ---> proposing again for 7.1.

*** Bug 1145939 has been marked as a duplicate of this bug. ***

Posted upstream patches:

https://www.redhat.com/archives/libvir-list/2014-November/msg00024.html

Upstream commits:

commit 742d49fa170bf72ec1fee516fda9f6797b1124f9
Author: Pavel Hrdina <phrdina>
Date:   Thu Nov 20 19:52:00 2014 +0100

    qemu-command: introduce new vgamem attribute for QXL video device
    
    Add attribute to set vgamem_mb parameter of QXL device for QEMU. This
    value sets the size of VGA framebuffer for QXL device. Default value in
    QEMU is 8MB so reuse it also in libvirt to not break things.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1076098
    
    Signed-off-by: Pavel Hrdina <phrdina>

commit 24c6ca860e2cf25aa28b33e19c318296b291e42c
Author: Pavel Hrdina <phrdina>
Date:   Thu Nov 20 19:51:12 2014 +0100

    qemu-command: use vram attribute for all video devices
    
    So far we didn't have any option to set video memory size for qemu video
    devices. There was only the vram (ram for QXL) attribute but it was valid
    only for the QXL video device.
    
    To provide this feature to users QEMU has a dedicated device attribute
    called 'vgamem_mb' to set the video memory size. We will use the 'vram'
    attribute for setting video memory size for other QEMU video devices.
    
    For the cirrus device we will ignore the vram value because it has
    hardcoded video size in QEMU.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1076098
    
    Signed-off-by: Pavel Hrdina <phrdina>

commit f480a87aa6ebaef0f478f2c67bfbeeee872e63e0
Author: Pavel Hrdina <phrdina>
Date:   Wed Nov 12 13:58:33 2014 +0100

    caps: introduce new QEMU capability for vgamem_mb device property
    
    Allow setting vgamem size for video devices.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1076098
    
    Signed-off-by: Pavel Hrdina <phrdina>

commit c32cfc6d3ff53e6493928ac4bc14a19cd162bc4b
Author: Pavel Hrdina <phrdina>
Date:   Tue Nov 18 17:21:48 2014 +0100

    QXL: fix setting ram and vram values for QEMU QXL device
    
    QEMU has two different type of QXL display device. The first "qxl-vga"
    is for primary video device and second "qxl" is for secondary video
    device.
    
    There are also two different ways how to specify those devices on qemu
    command line, the first one and obsolete is using "-vga" option and the
    current new one is using "-device" option. The "-vga" could be used only
    to setup primary video device, so the "-vga qxl" equal to
    "-device qxl-vga". Unfortunately the "-vga qxl" doesn't support setting
    additional parameters for the device and "-global" option must be used
    for this purpose. It's mandatory to use "-global qxl-vga...." to set the
    parameters of primary video device previously defined with "-vga qxl".
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1076098
    
    Signed-off-by: Pavel Hrdina <phrdina>

commit 81ba2298b27aa3257e7b5136f5dd5055076d7d9c
Author: Pavel Hrdina <phrdina>
Date:   Mon Nov 24 11:58:53 2014 +0100

    video: cleanup usage of vram attribute and update documentation
    
    The vram attribute was introduced to set the video memory but it is
    usable only for few hypervisors excluding QEMU/KVM and the old XEN
    driver. Only in case of QEMU the vram was used for QXL.
    
    This patch updates the documentation to reflect current code in libvirt
    and also changes the cases when we will set the default vram attribute.
    It also fixes existing strange default value for VGA devices 9MB to 16MB
    because the video ram should be rounded to power of two.
    
    The change of default value could affect migrations but I found out that
    QEMU always round the video ram to power of two internally so it's safe
    to change the default value to the next closest power of two and also
    silently correct every domain XML definition. And it's also safe because
    we don't pass the value to QEMU.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1076098
    
    Signed-off-by: Pavel Hrdina <phrdina>

Verify this bug with libvirt-1.2.8-9.el7:

Step:

1. Check the doc of libvirt 

in domainformate.html

4230           attribute <code>vram</code> specifies the secondary bar size.
4231           If "ram" or "vram" are not supplied a default value is used. The ram
4232           should also be rounded to power of two as vram. There is also optional
4233           attribute <code>vgamem</code> (<span class="since">since 1.2.11 (QEMU
4234           only)</span>) to set the size of VGA framebuffer for fallback mode of


4190   ...</pre>
4191         <dl><dt><code>video</code></dt><dd>
4192         <p>
4193           The <code>video</code> element is the container for describing
4194           video devices. For backwards compatibility, if no <code>video</code>
4195           is set but there is a <code>graphics</code> in domain xml, then
4196           libvirt will add a default <code>video</code> according to the guest
4197           type.
4198         </p>
4199         <p>


4209         <p>
4210           The <code>model</code> element has a mandatory <code>type</code>
4211           attribute which takes the value "vga", "cirrus", "vmvga", "xen",
4212           "vbox", or "qxl" (<span class="since">since 0.8.6</span>) depending
4213           on the hypervisor features available.
4214         </p>
4215         <p>
4216           You can provide the amount of video memory in kibibytes (blocks of
4217           1024 bytes) using <code>vram</code>. This is supported only for guest
4218           type of "libxl", "parallels", "qemu", "vbox", "vmx" and "xen". If no
4219           value is provided the default is used. If the size is not a power of
4220           two it will be rounded to closest one.
4221         </p>
4222         <p>
4223           The number of screen can be set using <code>heads</code>. This is
4224           supported only for guests type of "parallels", "kvm", "vbox" and "vmx".
4225         </p>
4226         <p>
4227           For guest type of kvm the optional attribute <code>ram</code>
4228           (<span class="since">since 1.0.2</span>) is allowed for "qxl" type
4229           only and specifies the size of the primary bar, while the optional
4230           attribute <code>vram</code> specifies the secondary bar size.
4231           If "ram" or "vram" are not supplied a default value is used. The ram
4232           should also be rounded to power of two as vram. There is also optional
4233           attribute <code>vgamem</code> (<span class="since">since 1.2.11 (QEMU
4234           only)</span>) to set the size of VGA framebuffer for fallback mode of
4235           QXL device.
4236         </p>



domaincommon.rng

2871               <optional>
2872                 <attribute name="vgamem">
2873                   <ref name="unsignedInt"/>
2874                 </attribute>
2875               </optional>
2876             </group>

2.set qxl video to guest and start it:
# virsh dumpxml test3 --inactive
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='8192' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# ps aux|grep qemu|grep vgamem
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=8,bus=pci.0,addr=0x2

3.set vga video to guest and start it
# virsh dumpxml test3 --inactive

    <video>
      <model type='vga' vram='65536' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# ps aux|grep qemu|grep vgamem
-device VGA,id=video0,vgamem_mb=64,bus=pci.0,addr=0x2


4.check the error output when set vgamem < 1M and not a power of 2.

# virsh edit test3
error: unsupported configuration: value for 'vgamem' must be at least 1 MiB (1024 KiB)
Failed. Try again? [y,n,f,?]:


# virsh edit test3
error: unsupported configuration: value for 'vgamem' must be power of two
Failed. Try again? [y,n,f,?]:


5.Test for comment 4:
RHEL 7 guest with 4 Full HD monitors:


5.1 prepare a running guest with 64M vgamem
# virsh dumpxml r7
   <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# ps aux|grep vgamem

qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=64,bus=pci.0,addr=0x2

5.2 start spice-vdagentd in both guest and host and use virt-viewer start 4 full monitors

# virt-viewer r7 -f

5.3 check the 4 full screen work well on host four monitors

Also i have done a basic test for cross migrate:

steps:

1.a running guest in 7.0 host
# virsh dumpxml r6.5
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

2.# ps aux|grep qemu

-vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 

3..migrate to a target 7.1 host
# virsh migrate r6.5 qemu+ssh://lhuang/system --verbose
Migration: [100 %]

4.check in target host:
# virsh dumpxml r6.5
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='8192' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

5.check qemu command line:
#ps aux|grep qemu
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=8,bus=pci.0,addr=0x2

6.migrate back to 7.0 host:

# virsh migrate r6.5 --live qemu+ssh://test1/system --verbose
Migration: [100 %]

7.check the xml in 7.0 host:
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

8.check qemu command line in 7.0 host:
-vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864




Also test a *unsupported* cross migrate from 7.1 host to 7.0 host:

if guest  vgamem='8192', migrate will success

but if guest vgamem='65536', migrate will fail.

error: operation failed: migration job: unexpectedly failed

And cross migrate from 7.0->7.1->7.0 work well when open a spice client (use virt-viewer here), and spice client still connect to server after these migrate.

Hi, thanks for testing the migration. It would be probably good to test migration from RHEL-6 hosts to RHEL-7.1 to see if everything works as there is a big difference between QEMU versions.

(In reply to Pavel Hrdina from comment #13)
> Hi, thanks for testing the migration. It would be probably good to test
> migration from RHEL-6 hosts to RHEL-7.1 to see if everything works as there
> is a big difference between QEMU versions.

Hi,Pavel.
Okay, i have done some basic test the rhel6.6 migrate to rhel7.1, then migrate back to rhel6.6:

Versions:
RHEL6.6:
qemu-kvm-rhev-0.12.1.2-2.448.el6.x86_64
libvirt-0.10.2-46.el6_6.2.x86_64

RHEL7:
qemu-kvm-rhev-2.1.2-14.el7.x86_64
libvirt-1.2.8-10.el7.x86_64

Steps:

1.a running guest in 6.6 host and have this settings in xml.
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

2.check qemu cmd line:
 -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864

3.migrate to rhel7.1 host:
# virsh migrate r6.5 --live qemu+ssh://lhuang/system --verbose
Migration: [100 %]

4.Check guest xml in RHEL7.1 host:
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='8192' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

5.check cmd line:
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=8,bus=pci.0,addr=0x2

6.migrate back to rhel6.6 host:
# virsh migrate r6.5 --live qemu+ssh://rhel6/system --verbose
Migration: [100 %]

7.check guest xml in rhel6.6 host:
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>



8.check qemu cmd line in RHEL7.1 host:
-vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 

And i cannot test spice now, i will test with connect spice tommorrow, and give a result.

But seems there is some issue when use vga video...
1.RHEL6.6 -> rhel7.1
# virsh dumpxml r6.5
    <video>
      <model type='vga' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# virsh migrate r6.5 --live qemu+ssh://lhuang/system --verbose 
error: operation failed: migration job: unexpectedly failed

2.RHEL7.0 -> rhel7.1
# virsh dumpxml r6.5
    <video>
      <model type='vga' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# virsh migrate r6.5 qemu+ssh://lhuang/system
error: operation failed: migration job: unexpectedly failed

Hi Pavel,

Please have a look about comment 15, i think this should be a issue when we do cross migrate.But i still need confirm with you, would you please help me to check out if this issue will fix in this bug? or need i open another bug for it?

Thanks in advance for your answer.

Well yes, there is an issue with migrating guest from older libvirt the the new one with this feature included if you set different vram than default for VGA device. It's because the old libvirt don't pass the vram value to QEMU and QEMU will use it's default size 16MB, but the new QEMU process will start with different then default ram because now we will pass the vram value to QEMU process.

However we still don't know what to do with this issue.

*** Bug 1156339 has been marked as a duplicate of this bug. ***

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html