Upstream patch posted:

v1: https://www.redhat.com/archives/libvir-list/2014-December/msg00628.html

v2: https://www.redhat.com/archives/libvir-list/2015-January/msg00223.html

v3: https://www.redhat.com/archives/libvir-list/2015-January/msg00237.html

Upstream commits:

commit ce745914b33e3f9a136d91655600b931e7a4178f
Author: Pavel Hrdina <phrdina>
Date:   Wed Dec 10 15:31:23 2014 +0100

    qemu_process: detect updated video ram size values from QEMU
    
    QEMU internally updates the size of video memory if the domain XML had
    provided too low memory size or there are some dependencies for a QXL
    devices 'vgamem' and 'ram' size. We need to know about the changes and
    store them into the status XML to not break migration or managedsave
    through different libvirt versions.
    
    The values would be loaded only if the "vgamem_mb" property exists for
    the device.  The presence of the "vgamem_mb" also tells that the
    "ram_size" and "vram_size" exists for QXL devices.
    
    Signed-off-by: Pavel Hrdina <phrdina>

commit cc41c64878590d7016e0dfa488d345e8634c3bf2
Author: Pavel Hrdina <phrdina>
Date:   Tue Dec 9 16:21:45 2014 +0100

    qemu_monitor: introduce new function to get QOM path
    
    The search is done recursively only through QOM object that has a type
    prefixed with "child<" as this indicate that the QOM is a parent for
    other QOM objects.
    
    The usage is that you give known device name with starting path where to
    search.
    
    Signed-off-by: Pavel Hrdina <phrdina>

I can produce this bug on build with following steps:
libvirt-1.2.8-12.el7.x86_64
qemu-kvm-rhev-2.1.2-18.el7.x86_64
3.10.0-222.el7.x86_64

1. prepare a cross migration between RHEL7.1 to RHEL7.0 with nfs

2. prepare a health guest with -M pc-i440fx-rhel7.0.0 and edit XML as below, set ram and vgamem for qxl device to same value
# virsh edit vm1
...
 <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type>
    <boot dev='hd'/>
  </os>
...
 <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

3. start guest success, and dumpxml to check ram and vgamem value does not changed
# virsh start vm1
Domain vm1 started

# virsh dumpxml vm1
...
 <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

4. do migration from RHEL7.1 RHEL7.0 with above configration, migration failed
# virsh migrate vm1 --live qemu+ssh://10.66.4.201/system --verbose --unsafe
root.4.201's password: 
error: operation failed: migration job: unexpectedly failed

5. check qemu log from target RHEL7.0 host
qemu: warning: error while loading state for instance 0x0 of device 'ram'
load of migration failed
2015-01-15 07:50:27.442+0000: shutting down
~

I can verify this bug on build
libvirt-1.2.8-13.el7.x86_64
qemu-kvm-rhev-2.1.2-19.el7.x86_64
3.10.0-222.el7.x86_64

verify steps:
1.prepare a cross migration between RHEL7.1 to RHEL7.0 with nfs

2. prepare a health guest with -M pc-i440fx-rhel7.0.0 and edit XML as below, set ram and vgamem for qxl device to same value
# virsh edit rhel7
...
 <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type>
    <boot dev='hd'/>
  </os>
...
 <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

3. start guest success, and dumpxml to check ram and vgamem value has been changed

# virsh start rhel7
Domain rhel7 started

# virsh dumpxml rhel7
...
<video>
      <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>   ====> here, ram value following the rule "ram" >= 2*"vgamem_mb"
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

4. check qemu cml
ps aux |grep rhel7
...
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=64,bus=pci.0,addr=0x2
...

5. do migration from RHEL7.1 to RHEL7.0, migration success without error
# virsh migrate rhel7 --live qemu+ssh://xx.xx.xx.xx/system --verbose --unsafe
root.xx.xx's password: 
Migration: [100 %]

6. check target RHEL7.0 host, dumpxml and qemu cml
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running

# virsh dumpxml rhel7
...
 <video>
      <model type='qxl' ram='131072' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

# ps aux |grep rhel7
 -vga qxl -global qxl-vga.ram_size=134217728 -global qxl-vga.vram_size=67108864
===> here notice the ram_size value


7. do migration back from RHEL7.0 to RHEL7.1, also can success

# virsh migrate rhel7 --live qemu+ssh://xx.xx.xx.xx/system --verbose --unsafe
root.xx.xx's password: 
Migration: [100 %]

8. check guest again from RHEL7.1 host, guest works well
# virsh dumpxml rhel7
...
<video>
      <model type='qxl' ram='131072' vram='65536' vgamem='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...
# ps aux |grep rhel7
-device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2
====> notice vgamem_mb has been set to default value 16MB


9. reset above steps， do migration from RHEL7.1 to RHEL7.0 with --xml 
# virsh dumpxml rhel7 > /tmp/rhel7.xml

edit /tmp/rhel7.xml 
from
 <video>
      <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
to
 <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose --unsafe --xml /tmp/rhel7.xml 
root.106.11's password: 
error: unsupported configuration: Target video card ram 65536 does not match source 131072

the report error is expected

Pavel Hrdina,
are the above steps to verify this bug is enough?

Hello, Pavel Hrdina

I found another issue during my verification test

1, edit guest xml as below, and save it:
# virsh edit rhel7
...
<video>
      <model type='qxl' ram='1' vram='1' vgamem='1024' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

2. start guest success, and dumpxml find the vgamem value change from 1024 to 8192, it is not normal

# virsh start rhel7
Domain rhel7 started

# virsh dumpxml rhel7
...
 <video>
      <model type='qxl' ram='16384' vram='4' vgamem='8192' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...

3. check qemu cml

# ps aux |grep rhel7
...
-device qxl-vga,id=video0,ram_size=1024,vram_size=1024,vgamem_mb=1,bus=pci.0,addr=0x2
...

it is obviously that the dumpxml vgamem_mb value are not consistent with edit vgamem value, so the qemu cmdline use the edit value vgamem_mb=1. And at the same time, guest with this configuration will reboot once again and again.

4.then do migration from RHEL7.1 to RHEL7.0 will report error because of xml and cmdline parameter with vgamem value are not the same
# virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose
root.106.11's password: 
error: operation failed: migration job: unexpectedly failed



i am wondering maybe this is a new bug for qxl vgamem setting, do you think so?

hi, Pavel Hrdina
i found that the issue described in comment 8 can be produced with setting vgamem to power of 2 but lower than 8192, such as vgamem='1024','2048','4096'.

(In reply to vivian zhang from comment #8)
> Hello, Pavel Hrdina
> 
> I found another issue during my verification test
> 
> 1, edit guest xml as below, and save it:
> # virsh edit rhel7
> ...
> <video>
>       <model type='qxl' ram='1' vram='1' vgamem='1024' heads='1'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> ...
> 
> 2. start guest success, and dumpxml find the vgamem value change from 1024
> to 8192, it is not normal
> 
> # virsh start rhel7
> Domain rhel7 started
> 
> # virsh dumpxml rhel7
> ...
>  <video>
>       <model type='qxl' ram='16384' vram='4' vgamem='8192' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> ...
> 
> 3. check qemu cml
> 
> # ps aux |grep rhel7
> ...
> -device
> qxl-vga,id=video0,ram_size=1024,vram_size=1024,vgamem_mb=1,bus=pci.0,addr=0x2
> ...
> 
> it is obviously that the dumpxml vgamem_mb value are not consistent with
> edit vgamem value, so the qemu cmdline use the edit value vgamem_mb=1. And
> at the same time, guest with this configuration will reboot once again and
> again.

The reason that the guest is rebooting in loop is QEMU allow to set vram to 4KB and after that QXL driver in the guests system will crash attempting to access the vram memory but it's probably to low. This may be QEMU or QXL driver bug because it is possible to start such guest also without libvirt.

> 
> 4.then do migration from RHEL7.1 to RHEL7.0 will report error because of xml
> and cmdline parameter with vgamem value are not the same
> # virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose
> root.106.11's password: 
> error: operation failed: migration job: unexpectedly failed
> 
> 
> 
> i am wondering maybe this is a new bug for qxl vgamem setting, do you think
> so?

The migration fails because QEMU will allocate only 8MB for vgamem framebuffer on RHEL-7.1. Then you try to migrate to RHEL-7.0 host, but he doesn't know the "vgamem" attribute and therefore he cannot pass it to QEMU. QEMU will use it's own default value which is 16MB and it will except from incoming migration data that the memory section used by vgamem framebuffer is 16MB but it isn't.

This behavior is correct and there is nothing what we can do about that because there is no way how to tell QEMU on RHEL-7.0 to set the vgamem_mb to different value than the default 16MB.

Migration from RHEL-7.1 to RHEL-7.0 will work correctly only in case the "vgamem" is set to 16MB.

Pavel

(In reply to vivian zhang from comment #7)
> I can verify this bug on build
> libvirt-1.2.8-13.el7.x86_64
> qemu-kvm-rhev-2.1.2-19.el7.x86_64
> 3.10.0-222.el7.x86_64
> 
> verify steps:
> 1.prepare a cross migration between RHEL7.1 to RHEL7.0 with nfs
> 
> 2. prepare a health guest with -M pc-i440fx-rhel7.0.0 and edit XML as below,
> set ram and vgamem for qxl device to same value
> # virsh edit rhel7
> ...
>  <os>
>     <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type>
>     <boot dev='hd'/>
>   </os>
> ...
>  <video>
>       <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> ...
> 
> 3. start guest success, and dumpxml to check ram and vgamem value has been
> changed
> 
> # virsh start rhel7
> Domain rhel7 started
> 
> # virsh dumpxml rhel7
> ...
> <video>
>       <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
> ====> here, ram value following the rule "ram" >= 2*"vgamem_mb"
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> ...
> 
> 4. check qemu cml
> ps aux |grep rhel7
> ...
> -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=64,bus=pci.
> 0,addr=0x2
> ...
> 
> 5. do migration from RHEL7.1 to RHEL7.0, migration success without error
> # virsh migrate rhel7 --live qemu+ssh://xx.xx.xx.xx/system --verbose --unsafe
> root.xx.xx's password: 
> Migration: [100 %]

As I've wrote in the Comment 10, the migration from RHEL-7.1 to RHEL-7.0 will work only if the "vgamem" is 16MB. This should not work and QEMU should fail on destination.

> 
> 6. check target RHEL7.0 host, dumpxml and qemu cml
> # virsh list
>  Id    Name                           State
> ----------------------------------------------------
>  2     rhel7                          running
> 
> # virsh dumpxml rhel7
> ...
>  <video>
>       <model type='qxl' ram='131072' vram='65536' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> ...
> 
> # ps aux |grep rhel7
>  -vga qxl -global qxl-vga.ram_size=134217728 -global
> qxl-vga.vram_size=67108864
> ===> here notice the ram_size value
> 
> 
> 7. do migration back from RHEL7.0 to RHEL7.1, also can success
> 
> # virsh migrate rhel7 --live qemu+ssh://xx.xx.xx.xx/system --verbose --unsafe
> root.xx.xx's password: 
> Migration: [100 %]
> 
> 8. check guest again from RHEL7.1 host, guest works well
> # virsh dumpxml rhel7
> ...
> <video>
>       <model type='qxl' ram='131072' vram='65536' vgamem='16384' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> ...
> # ps aux |grep rhel7
> -device
> qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.
> 0,addr=0x2
> ====> notice vgamem_mb has been set to default value 16MB

This is expected because the QEMU on RHEL-7.0 will always use 16MB for "vgamem_mb" and there is no way to tell it otherwise from libvirt point of view.

> 
> 
> 9. reset above steps， do migration from RHEL7.1 to RHEL7.0 with --xml 
> # virsh dumpxml rhel7 > /tmp/rhel7.xml
> 
> edit /tmp/rhel7.xml 
> from
>  <video>
>       <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> to
>  <video>
>       <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
> 
> # virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose
> --unsafe --xml /tmp/rhel7.xml 
> root.106.11's password: 
> error: unsupported configuration: Target video card ram 65536 does not match
> source 131072
> 
> the report error is expected
> 
> Pavel Hrdina,
> are the above steps to verify this bug is enough?

Yes, those steps are enough to verify this bug, but use only 16384 for "vgamem" for cross migration. Setting it to lower value will cause the migration to fail and setting it to higher value should also cause the migration to fail (QEMU on destination should fail because of memory mismatch).

(In reply to Pavel Hrdina from comment #11)
> (In reply to vivian zhang from comment #7)
> > I can verify this bug on build
> > libvirt-1.2.8-13.el7.x86_64
> > qemu-kvm-rhev-2.1.2-19.el7.x86_64
> > 3.10.0-222.el7.x86_64
> > 
> > verify steps:
> > 1.prepare a cross migration between RHEL7.1 to RHEL7.0 with nfs
> > 
> > 2. prepare a health guest with -M pc-i440fx-rhel7.0.0 and edit XML as below,
> > set ram and vgamem for qxl device to same value
> > # virsh edit rhel7
> > ...
> >  <os>
> >     <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type>
> >     <boot dev='hd'/>
> >   </os>
> > ...
> >  <video>
> >       <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
> >       <alias name='video0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >     </video>
> > ...
> > 
> > 3. start guest success, and dumpxml to check ram and vgamem value has been
> > changed
> > 
> > # virsh start rhel7
> > Domain rhel7 started
> > 
> > # virsh dumpxml rhel7
> > ...
> > <video>
> >       <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
> > ====> here, ram value following the rule "ram" >= 2*"vgamem_mb"
> >       <alias name='video0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >     </video>
> > ...
> > 
> > 4. check qemu cml
> > ps aux |grep rhel7
> > ...
> > -device
> > qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=64,bus=pci.
> > 0,addr=0x2
> > ...
> > 
> > 5. do migration from RHEL7.1 to RHEL7.0, migration success without error
> > # virsh migrate rhel7 --live qemu+ssh://xx.xx.xx.xx/system --verbose --unsafe
> > root.xx.xx's password: 
> > Migration: [100 %]
> 
> As I've wrote in the Comment 10, the migration from RHEL-7.1 to RHEL-7.0
> will work only if the "vgamem" is 16MB. This should not work and QEMU should
> fail on destination.
> 
> > 
> > 6. check target RHEL7.0 host, dumpxml and qemu cml
> > # virsh list
> >  Id    Name                           State
> > ----------------------------------------------------
> >  2     rhel7                          running
> > 
> > # virsh dumpxml rhel7
> > ...
> >  <video>
> >       <model type='qxl' ram='131072' vram='65536' heads='1'/>
> >       <alias name='video0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >     </video>
> > ...
> > 
> > # ps aux |grep rhel7
> >  -vga qxl -global qxl-vga.ram_size=134217728 -global
> > qxl-vga.vram_size=67108864
> > ===> here notice the ram_size value
> > 
> > 
> > 7. do migration back from RHEL7.0 to RHEL7.1, also can success
> > 
> > # virsh migrate rhel7 --live qemu+ssh://xx.xx.xx.xx/system --verbose --unsafe
> > root.xx.xx's password: 
> > Migration: [100 %]
> > 
> > 8. check guest again from RHEL7.1 host, guest works well
> > # virsh dumpxml rhel7
> > ...
> > <video>
> >       <model type='qxl' ram='131072' vram='65536' vgamem='16384' heads='1'/>
> >       <alias name='video0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >     </video>
> > ...
> > # ps aux |grep rhel7
> > -device
> > qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.
> > 0,addr=0x2
> > ====> notice vgamem_mb has been set to default value 16MB
> 
> This is expected because the QEMU on RHEL-7.0 will always use 16MB for
> "vgamem_mb" and there is no way to tell it otherwise from libvirt point of
> view.
> 
> > 
> > 
> > 9. reset above steps， do migration from RHEL7.1 to RHEL7.0 with --xml 
> > # virsh dumpxml rhel7 > /tmp/rhel7.xml
> > 
> > edit /tmp/rhel7.xml 
> > from
> >  <video>
> >       <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
> >       <alias name='video0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >     </video>
> > to
> >  <video>
> >       <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
> >       <alias name='video0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >     </video>
> > 
> > # virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose
> > --unsafe --xml /tmp/rhel7.xml 
> > root.106.11's password: 
> > error: unsupported configuration: Target video card ram 65536 does not match
> > source 131072
> > 
> > the report error is expected
> > 
> > Pavel Hrdina,
> > are the above steps to verify this bug is enough?
> 
> Yes, those steps are enough to verify this bug, but use only 16384 for
> "vgamem" for cross migration. Setting it to lower value will cause the
> migration to fail and setting it to higher value should also cause the
> migration to fail (QEMU on destination should fail because of memory
> mismatch).

thanks, Pavel Hrdina for your reply

according to your answer, I still have a confusion when setting vgamem > 16MB, cross migration should success like below:

1. setting vgamem>=16384 & ram =random  in XML
2. guest boot up success with 
Edit XML:
<model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>

Dumpxml:
<model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
Qemu CML:
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=64,bus=pci.0,addr=0x2

3. do cross migration from RHEL7.1->RHEL7.0, it should success
# virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose --unsafe
root.106.11's password: 
Migration: [100 %]

on RHEL7.0:
Dumpxml:
<model type='qxl' ram='131072' vram='65536' heads='1'/>
Qemu CML:
-vga qxl -global qxl-vga.ram_size=134217728 -global qxl-vga.vram_size=671088

are the above steps works as design?

thanks
wzhang

(In reply to vivian zhang from comment #12)

[...]

> 
> thanks, Pavel Hrdina for your reply
> 
> according to your answer, I still have a confusion when setting vgamem >
> 16MB, cross migration should success like below:
> 
> 1. setting vgamem>=16384 & ram =random  in XML
> 2. guest boot up success with 
> Edit XML:
> <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1'/>
> 
> Dumpxml:
> <model type='qxl' ram='131072' vram='65536' vgamem='65536' heads='1'/>
> Qemu CML:
> -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=64,bus=pci.
> 0,addr=0x2
> 
> 3. do cross migration from RHEL7.1->RHEL7.0, it should success
> # virsh migrate rhel7 --live qemu+ssh://10.66.106.11/system --verbose
> --unsafe
> root.106.11's password: 
> Migration: [100 %]
> 
> on RHEL7.0:
> Dumpxml:
> <model type='qxl' ram='131072' vram='65536' heads='1'/>
> Qemu CML:
> -vga qxl -global qxl-vga.ram_size=134217728 -global qxl-vga.vram_size=671088
> 
> are the above steps works as design?
> 
> thanks
> wzhang

Hi, as I said, only vgamem='16384' is supported for cross migration (RHEL-7.0 -> RHEL-7.1 -> RHEL-7.0) and QEMU should probably fail if the value is different. That steps accidentally work but it's not designed to work.

hi, Gerd

For this bug comment12 & 13, could you please help libvirt confirm it from qemu view?

thanks in advance

  Hi,

> Hi, as I said, only vgamem='16384' is supported for cross migration
> (RHEL-7.0 -> RHEL-7.1 -> RHEL-7.0) and QEMU should probably fail if the
> value is different. That steps accidentally work but it's not designed to
> work.

Correct.  When vgamem can't be set (because either qemu or libvirt or both don't support it), then it must be 16MB because that is what qemu uses by default.

Live migration will not fail if this isn't the case.  vgamem just specifies how ram is partitioned, so the total size of the memory doesn't change -> no problem for qemu here.  The device configuration will change underneath the guest though, which may confuse the driver and/or cause rendering problems further down the road.

So, it's not valid even if live migration doesn't throw an error.

(In reply to Gerd Hoffmann from comment #15)
>   Hi,
> 
> > Hi, as I said, only vgamem='16384' is supported for cross migration
> > (RHEL-7.0 -> RHEL-7.1 -> RHEL-7.0) and QEMU should probably fail if the
> > value is different. That steps accidentally work but it's not designed to
> > work.
> 
> Correct.  When vgamem can't be set (because either qemu or libvirt or both
> don't support it), then it must be 16MB because that is what qemu uses by
> default.
> 
> Live migration will not fail if this isn't the case.  vgamem just specifies
> how ram is partitioned, so the total size of the memory doesn't change -> no
> problem for qemu here.  The device configuration will change underneath the
> guest though, which may confuse the driver and/or cause rendering problems
> further down the road.
> 
> So, it's not valid even if live migration doesn't throw an error.

Thanks Gerd

According to your opinion, do you agree with that we should file a qemu bug for this invalid migration?

  Hi,

> According to your opinion, do you agree with that we should file a qemu bug
> for this invalid migration?

Doesn't make much sense.  We can catch it on the receiving side only, and it can happen when migration from new (vgamem configurable) to old (vgamem not configurable) qemu.  Merging a fix upstream now doesn't buy us anything, we would have to z-stream fix the old qemu versions.  Not worth the hassle IMO.

verify steps on build
libvirt-1.2.8-15.el7.x86_64
qemu-kvm-rhev-2.1.2-20.el7.x86_64
3.10.0-223.el7.x86_64

1. prepare migration env from rhel7.1 -> rhel7.0

2. start a guest with configuration

edit XML:
...
<video>
      <model type='qxl' ram='16384' vram='16384' vgamem='16384' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </video>
...

start guest

# virsh start r
Domain r started


dumpxml :
...
 <video>
      <model type='qxl' ram='32768' vram='16384' vgamem='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </video>
...
notice ram value has been automatically changed to 2*vgamem

# ps aux |grep r |grep qxl
-device qxl-vga,id=video0,ram_size=16777216,vram_size=16777216,vgamem_mb=16,bus=pci.0,addr=0xa

3. do cross migration from rhel7.1 -> rhel7.0

# virsh migrate r --live qemu+ssh://10.66.106.11/system --verbose --unsafe
root.106.11's password: 
Migration: [100 %]


4. check guest on rhel7.0
# virsh dumpxml r
 <video>
      <model type='qxl' ram='32768' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

Qemu CML
-vga qxl -global qxl-vga.ram_size=33554432 -global qxl-vga.vram_size=16777216

5. back migrate from rhel7.0 -> rhel7.1 should also success, and choose vgamem with default 16MB

6. when setting vgamem<='8192', cross migration from rhel7.1->rhel7.0 should fail, it is expected.

7. when setting vgmem>'16384', cross migration from rhel7.1 -> rhel7.0 success, it is not a valid operation from Qemu side as comment 17.

so set this bug to verified

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html