Bug 823535 - Libvirt is sensitive to the order in which the video devices are passed
Libvirt is sensitive to the order in which the video devices are passed
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.0
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Libvirt Maintainers
Virtualization Bugs
:
Depends On:
Blocks: 1008365
  Show dependency treegraph
 
Reported: 2012-05-21 09:28 EDT by Oved Ourfali
Modified: 2016-04-26 11:11 EDT (History)
11 users (show)

See Also:
Fixed In Version: libvirt-1.2.7-1.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1008365 (view as bug list)
Environment:
Last Closed: 2015-03-05 02:19:14 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Oved Ourfali 2012-05-21 09:28:50 EDT
Description of problem:

Looks like libvirt is sensitive to the order in which the video devices are passed.
Passing them in a wrong order causes the error below.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
Tried with two VMs, one in which the order was:
<video>
 <address  domain="0x0000"  function="0x0"  slot="0x07"  type="pci" bus="0x00"/>
 <model heads="1" type="qxl" vram="65536"/>
</video>
<video>
 <address  domain="0x0000"  function="0x0"  slot="0x02"  type="pci" bus="0x00"/>
 <model heads="1" type="qxl" vram="65536"/>
</video>

Failed on (from libvirt.log):
error : virNetClientProgramDispatchError:174 : internal error Primary video card must have PCI address 0:0:2.0


In the other VM the order was:

<video>
 <address  domain="0x0000"  function="0x0"  slot="0x02"  type="pci" bus="0x00"/>
 <model heads="1" type="qxl" vram="65536"/>
</video>
<video>
 <address  domain="0x0000"  function="0x0"  slot="0x07"  type="pci" bus="0x00"/>
 <model heads="1" type="qxl" vram="65536"/>
</video>

And it succeeded.
  
Actual results:
The VM in which the order is wrong fails with error:
error : virNetClientProgramDispatchError:174 : internal error Primary video card must have PCI address 0:0:2.0

Expected results:
Both VMs should start correctly.
Comment 4 Gunannan Ren 2012-11-06 08:31:57 EST
New QEMU option -device permits specification of PCI device * address. It doesn't work for VGA devices yet, but QEMU always assigns PCI device address 2 to the VGA. 
https://fedoraproject.org/wiki/Features/KVM_Stable_PCI_Addresses

For libvirt, there are hardcoded slot values for the first PCI devices like
 *  - Host bridge (slot 0)
 *  - PIIX3 ISA bridge, IDE controller, something else unknown, USB controller (slot 1)
 *  - Video (slot 2)

So the error message "Primary video card must have PCI address 0:0:2.0" makes sense. Either you assign slot="0x02" to the primary video device or remove <address> line and let libvirt assign it automatically.
Comment 5 Dave Allan 2012-11-26 10:36:56 EST
Guannan, does qemu now permit the configuration of the video card address, or is there a qemu BZ to make the video card address configurable?
Comment 6 Gunannan Ren 2012-12-03 03:58:19 EST
The i440FX(motherboard), PIIX3, cirrus-vga, rtl8139(or e1000) are qemu built-in pci device, they are allocated pci address from 0x0 to 0x3 in turn by qemu.
libvirt makes sure the assignement of their pci address is valid by users based on this order.
Currently, qemu-kvm doesn't support the address configuration of these built-in PCI devices.

For example, when I try to populate a vga device in 0x0, it will report error as follows:

# /usr/bin/qemu-kvm -m 1024 -smp 1 -name "rhel" -drive file=/var/lib/libvirt/images/rhel6u3qcow2.img   \
-device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x0

qemu-kvm: -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x0: PCI: slot 0 function 0 not available for qxl, in use by i440FX

The same errors for the rest of other three address(0x1, 0x2, 0x3)
PCI: slot 1 function 0 not available for qxl, in use by PIIX3
PCI: slot 2 function 0 not available for qxl, in use by cirrus-vga
PCI: slot 3 function 0 not available for qxl, in use by rtl8139

About if the qemu will support such configuration of pci address, I have pinged qemu-kvm guys waiting for repsonce.
Comment 7 Gerd Hoffmann 2012-12-04 02:24:06 EST
i440FX is fixed at slot 0.
PIIX3 is fixed at slot 1.

That isn't going to change ever, these are parts of the emulated chipset.

vga is fixed at slot 2 in rhel-6.  You have the option to run without vga though (-vga none), then assign slot 2 to something else.

Newer qemu versions (qemu 1.2+ IIRC, will work for sure in rhel-7) can create vga devices using -device, which allows to put them into another slot using the addr property, i.e. '-vga none -device qxl-vga,addr=7'.

There is nothing fixed in slot 3.  The NIC just ends up there because slot 3 is the first free slot and if not specified otherwise qemu will assign the first free slot to devices you add.  You can change the nic slot number in the libvirt xml to something else and it will work just fine.
Comment 8 Gunannan Ren 2012-12-04 04:09:27 EST
Thanks for above infomation, in my box,  '-vga none' and '-net none' could explicitly remove built-in VGA and NIC devices. then, we could use '-device qxl-vga' instead for putting VGA device into another available slot.
Comment 9 Gerd Hoffmann 2012-12-04 06:10:49 EST
Same goes for all vga devices, not only qxl.

'-vga std' maps to '-device VGA'
'-vga cirrus' maps to '-device cirrus-vga'
'-vga qxl' maps to '-device qxl-vga'
             (there is also '-device qxl' for secondary devices)
'-vga vmware' maps to '-device vmware-svga'

Remember that you have to keep using the -vga switch for older qemu versions.

Also note that '-M isapc' uses different vga devices too (isa-vga + isa-cirrus-vga), it is probably easiest to keep using -vga for isapc.
Comment 10 Gunannan Ren 2012-12-10 03:56:36 EST
patch sent out to upstreamhttps://www.redhat.com/archives/libvir-list/2012-December/msg00406.html
Comment 11 Gunannan Ren 2012-12-17 01:35:49 EST
For this request of non-fixed PCI address for video device, you have to use qemu which version is equal or greater than 1.2, libvirt >= 1.0.2

commit aa51202b723ac6a2b7b99827abf60a7593afcba6
Author: Guannan Ren <gren@redhat.com>
Date:   Fri Dec 14 15:08:01 2012 +0800

    qemu: use newer -device video device in qemu commandline
    
    '-device VGA' maps to '-vga std'
    '-device cirrus-vga' maps to '-vga cirrus'
    '-device qxl-vga' maps to '-vga qxl'
                 (there is also '-device qxl' for secondary devices)
    '-device vmware-svga' maps to '-vga vmware'
    
    For qemu(>=1.2), we can use -device to replace -vga for video
    device. For the primary video device, the patch tries to use 0x2
    slot for matching old qemu. If the 0x2 slot is allocated already,
    the addr property could help for using any available slot.
    For qemu(< 1.2), we keep using -vga for primary device.
Comment 12 zhe peng 2012-12-20 22:03:45 EST
I can reproduce this with libvirt-1.0.1-1.el7.x86_64
Comment 15 zhe peng 2013-07-03 03:08:37 EDT
verify with build:
libvirt-1.1.0-1.el7.x86_64
qemu-kvm-1.5.0-2.el7.x86_64
kernel-3.10.0-0.rc7.64.el7.x86_64

#virsh dumpxml $guest
....
 <video>
      <model type='qxl' ram='65536' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='9216' heads='1'/>
      <alias name='video1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='9216' heads='1'/>
      <alias name='video2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

....

# virsh start spice
Domain spice started

#ps -ef | grep qemu-kvm
....
-spice port=5900,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=67108864,vram_size=9437184,bus=pci.0,addr=0x7 -device qxl,id=video1,ram_size=67108864,vram_size=9437184,bus=pci.0,addr=0x8 -device qxl,id=video2,ram_size=67108864,vram_size=9437184,bus=pci.0,addr=0x2
....

verification passed.
Comment 16 Xuesong Zhang 2013-08-02 01:17:46 EDT
After update the libvirt build to libvirt-1.1.1-1.el7.x86_64, it seems this bug is appeared again in the latest libvirt build. So change the bug status to assigned again.


Version-Release number of selected component (if applicable):
libvirt-1.1.1-1.el7.x86_64
qemu-kvm-1.5.2-1.el7.x86_64
3.10.0-0.rc7.64.el7.x86_64

How reproducible:
100%


Steps:
1. prepare one health guest with the following xml, on one host which the libvirt version is old.

old version build:
libvirt-1.1.0-1.el7.x86_64
qemu-kvm-1.5.0-2.el7.x86_64
kernel-3.10.0-0.rc7.64.el7.x86_64

#virsh dumpxml $guest
....
 <video>
      <model type='qxl' ram='65536' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='9216' heads='1'/>
      <alias name='video1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='9216' heads='1'/>
      <alias name='video2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

....

2. make sure the guest can be started normally.
# virsh start a
Domain a started

3. update the libvirt to the latest version, and restart the libvirtd service. Then start the same guest again.
# virsh start a
error: Failed to start domain a
error: internal error: Primary video card must have PCI address 0:0:2.0


Actual results:
The guest failed to be started up in step3.

Expected results:
The guest should be started up in step3.
Comment 17 Gerd Hoffmann 2013-08-02 02:12:11 EDT
That is intentional.  There is a qemu bug (fixed in upcoming 1.6) and disabling this feature (for qxl) is the workaround for it.  It should continue working for other vga cards.  /me also submitted the qemu fix to stable, so it should show up in 1.5.3 and thereby find it's way into rhel7.
Comment 19 CongDong 2013-12-23 22:37:22 EST
Still can reproduce this on rhel7 with:
qemu-kvm-1.5.3-30.el7.x86_64
libvirt-1.1.1-16.el7.x86_64

step:
1. virsh edit $vm
...
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
...
2. Save xml
:wq
error: internal error: Primary video card must have PCI address 0:0:2.0
Failed. Try again? [y,n,f,?]:
Comment 28 Luyao Huang 2014-11-18 22:13:44 EST
Verify this bug with libvirt-1.2.8-7 and qemu-kvm-rhev-2.1.2-6.el7.x86_64:

# virsh edit test5

    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# virsh start test5
Domain test5 started

# virsh dumpxml test5

    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </video>
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

# ps aux|grep qemu|grep test5|grep qxl
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x7 -device qxl,id=video1,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2

And also test with qemu-kvm still report error, but this is expected.

# virsh edit r6
error: internal error: Primary video card must have PCI address 0:0:2.0
Failed. Try again? [y,n,f,?]:
Comment 30 errata-xmlrpc 2015-03-05 02:19:14 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html

Note You need to log in before you can comment on or make changes to this bug.