Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1359135

Summary: Libvirt fails to assign persistent DIMM slot addresses
Product: Red Hat Enterprise Linux 7 Reporter: Daniel Berrangé <berrange>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: Luyao Huang <lhuang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: dyuan, jdenemar, lhuang, pkrempa, rbalakri, xuzhang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-2.5.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 17:11:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Daniel Berrangé 2016-07-22 11:40:21 UTC 
Description of problem:
When defining XML for a guest, libvirt will immediately assign addresses for all devices. This ensures that devices get stable addresses every time the guest is started.

This is not happening for memory DIMM devices. Their addresses are only getting assigned at the time the guest is started.

So consider the following sequence

 - Define guest with 2 DIMMs present in node 0
 - Start the guest
 - DIMMS get assigned slots 0 + 1
 - Hotplug a 3rd dimm which gets assigned slot 2

 - Hotplug DIMMin slot 1
 - Shutdown the guest
 - Start the guest

At this point we should have DIMMS in slots 0 and 2, but we in fact get DIMMs in slot 0 + 1.

We must assign slot numbers for the guests at define time, ie in qemuDomainAssignAddresses()

As a further issue, the QEMU saved state file encodes the 'ID' used with the DIMM/memorybackend as the name of the data section.  So this means we must also ensure we use stable ID names, based off the slot number, instead of assigning IDs simply based on how many dimms are present.


Version-Release number of selected component (if applicable):
2.0.0
Comment 2 Peter Krempa 2016-11-10 16:57:18 UTC 
Fixed upstream:

commit b7798a07f93e015cade95297b2bc4eadbb7cce27
Author: Peter Krempa <pkrempa>
Date:   Tue Nov 1 06:21:36 2016 +0100

    qemu: Generate memory device aliases according to slot number
    
    The memory device alias needs to be treated as machine ABI as qemu is
    using it in the migration stream for section labels. To simplify this
    generate the alias from the slot number unless an existing broken
    configuration is detected.
    
    With this patch the aliases are predictable and even certain
    configurations which would not be migratable previously are fixed.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1359135

commit ce1ee02a25c046f176c265e7fd920e7c4afb2d06
Author: Peter Krempa <pkrempa>
Date:   Tue Nov 1 06:07:09 2016 +0100

    qemu: Assign slots to memory devices prior to usage
    
    As with other devices assign the slot number right away when adding the
    device. This will make the slot numbers static as we do with other
    addressing elements and it will ultimately simplify allocation of the
    alias in a static way which does not break with qemu.

commit 93d9ff3da0eab9676c52ea1bc0302162cdf440c0
Author: Peter Krempa <pkrempa>
Date:   Mon Oct 31 16:49:49 2016 +0100

    qemu: process: detect if dimm aliases are broken on reconnect
    
    Detect on reconnect to a running qemu VM whether the alias of a
    hotpluggable memory device (dimm) does not match the dimm slot number
    where it's connected to. This is necessary as qemu is actually
    considering the alias as machine ABI used to connect the backend object
    to the dimm device.
    
    This will require us to keep them consistent so that we can reliably
    restore them on migration. In some situations it was currently possible
    to create a mismatched configuration and qemu would refuse to restore
    the migration stream.
    
    To avoid breaking existing VMs we'll need to keep the old algorithm
    though.

commit 810e9a80618f73a22b13af3d8be8d1136b977ca7
Author: Peter Krempa <pkrempa>
Date:   Thu Oct 13 14:55:47 2016 +0200

    conf: Allow specifying only the slot number for hotpluggable memory
    
    Simplify handling of the 'dimm' address element by allowing to specify
    the slot number only. This will allow libvirt to allocate slot numbers
    before starting qemu.
Comment 7 Luyao Huang 2017-05-26 07:22:47 UTC 
Verify this bug with libvirt-3.2.0-6.el7.x86_64:

S1: test boot guest with dimm device + hot-plug dimm device

1. start a guest with 3 dimm device which on slot 0, 2, 4:

# virsh dumpxml r7

    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm0'/>
      <address type='dimm' slot='0' base='0x100000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm2'/>
      <address type='dimm' slot='2' base='0x120000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm4'/>
      <address type='dimm' slot='4' base='0x140000000'/>
    </memory>

2. check qemu cmd line:

# ps aux|grep qemu
qemu     10993 11.7  1.7 4055200 572864 ?      Sl   02:07   0:30 ... -object memory-backend-ram,id=memdimm0,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0 -object memory-backend-ram,id=memdimm2,size=536870912 -device pc-dimm,node=0,memdev=memdimm2,id=dimm2,slot=2 -object memory-backend-ram,id=memdimm4,size=536870912 -device pc-dimm,node=0,memdev=memdimm4,id=dimm4,slot=4

3. attach a mem device on slot 12:

# cat mem.xml 
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <address type='dimm' slot='12'/>
    </memory>

4. save guest and restore:

# virsh managedsave r7

Domain r7 state saved by libvirt

# virsh start r7
Domain r7 started

# ps aux|grep qemu
qemu     11306 24.6  1.5 3995472 522324 ?      Sl   02:13   0:00 ... -object memory-backend-ram,id=memdimm0,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0,addr=4294967296 -object memory-backend-ram,id=memdimm2,size=536870912 -device pc-dimm,node=0,memdev=memdimm2,id=dimm2,slot=2,addr=4831838208 -object memory-backend-ram,id=memdimm4,size=536870912 -device pc-dimm,node=0,memdev=memdimm4,id=dimm4,slot=4,addr=5368709120 -object memory-backend-ram,id=memdimm12,size=536870912 -device pc-dimm,node=0,memdev=memdimm12,id=dimm12,slot=12,addr=5905580032


S2: prepare a guest on a old libvirt and then update to libvirt-3.2.0-6.el7.x86_64:

1. prepare guest like this:
# virsh dumpxml r7

    <memory model='dimm'>
      <target>
        <size unit='KiB'>131072</size>
        <node>0</node>
      </target>
      <alias name='dimm0'/>
      <address type='dimm' slot='0' base='0x100000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm1'/>
      <address type='dimm' slot='2' base='0x128000000'/>
    </memory>

2. update libvirt:
# yum update -y libvirt

3. recheck xml:

# virsh dumpxml r7

    <memory model='dimm'>
      <target>
        <size unit='KiB'>131072</size>
        <node>0</node>
      </target>
      <alias name='dimm0'/>
      <address type='dimm' slot='0' base='0x100000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm1'/>
      <address type='dimm' slot='2' base='0x128000000'/>
    </memory>

4. attach a new memory device:

# cat mem.xml 
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
    </memory>

# virsh attach-device r7 mem.xml 
Device attached successfully

5. recheck xml:

# virsh dumpxml r7
    <memory model='dimm'>
      <target>
        <size unit='KiB'>131072</size>
        <node>0</node>
      </target>
      <alias name='dimm0'/>
      <address type='dimm' slot='0' base='0x100000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm1'/>
      <address type='dimm' slot='2' base='0x128000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm2'/>
      <address type='dimm' slot='1' base='0x108000000'/>
    </memory>

6. destroy and restart guest and check xml:

# virsh destroy r7
Domain r7 destroyed

# virsh start r7
Domain r7 started

7. recheck xml:

    <memory model='dimm'>
      <target>
        <size unit='KiB'>131072</size>
        <node>0</node>
      </target>
      <alias name='dimm0'/>
      <address type='dimm' slot='0' base='0x100000000'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <alias name='dimm2'/>
      <address type='dimm' slot='2' base='0x128000000'/>
    </memory>
Comment 8 errata-xmlrpc 2017-08-01 17:11:42 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 9 errata-xmlrpc 2017-08-01 23:53:19 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846