Bug 1283924

Summary: ppc64le: VFIO doesn't work for small guests (1 GiB) [7.2.z]
Product: Red Hat Enterprise Linux 7 Reporter: Jan Kurik <jkurik>
Component: libvirtAssignee: Andrea Bolognani <abologna>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: urgent Docs Contact:
Priority: high    
Version: 7.2CC: abologna, dgibson, dyuan, dzheng, gklein, gsun, hannsj_uhl, jdenemar, jherrman, jsuchane, juzhang, lvivier, mavital, michen, mpoledni, mzhan, pkrempa, qzhang, rbalakri, tlavigne, virt-maint
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.17-13.el7_2.1 Doc Type: Bug Fix
Doc Text:
Prior to this update, guest virtual machines with 1 GiB or less allocated memory in some cases failed to start on IBM Power systems. With this update, the memory lock limit is calculated using architecture-specific logic, which allows the affected guests to start as expected.
Story Points: ---
Clone Of: 1273480 Environment:
Last Closed: 2015-12-08 10:41:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1273480    
Bug Blocks: 1201513    

Description Jan Kurik 2015-11-20 10:01:50 UTC
This bug has been copied from bug #1273480 and has been proposed
to be backported to 7.2 z-stream (EUS).

Comment 6 Dan Zheng 2015-11-23 05:54:38 UTC
Test with below packages:
libvirt-1.2.17-13.el7_2.1.ppc64le
qemu-kvm-rhev-2.3.0-31.el7_2.3.ppc64le
kernel-3.10.0-327.el7.ppc64le
*************************************************************
Test 1: MaxMemory + small guest memory 1G + Hostdev can start up now (failed before the fix)
# virsh nodedev-dumpxml pci_0003_09_00_0
...
    <iommuGroup number='1'>
      <address domain='0x0003' bus='0x09' slot='0x00' function='0x0'/>
      <address domain='0x0003' bus='0x09' slot='0x00' function='0x1'/>
      <address domain='0x0003' bus='0x09' slot='0x00' function='0x2'/>
      <address domain='0x0003' bus='0x09' slot='0x00' function='0x3'/>
    </iommuGroup>

Guest XML:
  <maxMemory slots='2' unit='KiB'>10485760</maxMemory>
  <memory unit='KiB'>1048576</memory>
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>POWER8</model>
    <numa>
      <cell id='0' cpus='0' memory='1048576' unit='KiB'/>
    </numa>
  </cpu>
<devices>
      <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
</devices>

# virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started
*************************************************************

Test 2: Small guest memory 1G + without hostdev can start and 
        small guest memory 1G + hostdev can start up now (failed before the fix)

Guest XML:
  <memory unit='KiB'>1048576</memory>
  <devices>
...
      <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0003' bus='0x09' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
</devices>

# virsh start avocado-vt-vm1
Device attached successfully

Within the guest # lspci
00:07.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
00:08.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
00:09.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
00:0a.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
*************************************************************

Test 3: Small guest memory 1G + hotplug PCI device can work now (failed before the fix)
Guest XML:
<memory unit='KiB'>1048576</memory>

# virsh start avocado-vt-vm1
Device attached successfully

attach.xml:
<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
    <address domain='0x0003' bus='0x09' slot='0x00' function='0x0'/>
  </source>
</hostdev>
# virsh attach-device avocado-vt-vm1 attach.xml
Device attached successfully

within the guest: # lspci 
00:07.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)


**************************
Pass.
**************************

Comment 8 errata-xmlrpc 2015-12-08 10:41:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2567.html