Bug 1214369

Summary: RFE: ability to configure file backed guest RAM with regular page sizes
Product: Red Hat Enterprise Linux 7 Reporter: Daniel Berrangé <berrange>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Luyao Huang <lhuang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: dyuan, kchamart, lhuang, lmiksik, mprivozn, mzhan, rbalakri, xuzhang, yalzhang
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-3.2.0-10.el7 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 17:06:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1154205    

Description Daniel Berrangé 2015-04-22 14:46:54 UTC 
Description of problem:
Some customer applications need to have to file backed guest RAM so that external application processes can interact with QEMU guest RAM. Currently the only way to do this is to use huge pages, but the applications in question do not particularly care about the page size. In fact using huge pages is a complication because it looses memory overcommit ability and has extra administrative burden.  It is thus desirable to enable file backed guest RAM with regular (small) page sizes. This shouldn't need help from QEMU, it should be a simple matter of using /dev/shm filesystem instead of /dev/hugetlbfs filesystem. The main question is how to represent this in the libvirt XML schema.

Version-Release number of selected component (if applicable):
1.2.14
Comment 2 Michal Privoznik 2015-04-23 07:36:21 UTC 
(In reply to Daniel Berrange from comment #0)
> The main question is how to represent this in the
> libvirt XML schema.
> 

One possibility I see is to use /dev/shm whenever <hugepages/> (without any specific page size) requested and //cpu/num/cell/memAccess='shared' is set. Downside is, guest is required to have numa nodes configured, even though it may not be necessary for their scenario.
Comment 5 Michal Privoznik 2017-03-03 15:04:21 UTC 
I think this is fixed upstream now by the following commits:

commit 0857a3bf5ceff01a62593a5e3389246948dd11a4
Author:     Jaroslav Safka <jaroslavx.safka>
AuthorDate: Thu Feb 9 14:24:51 2017 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Feb 9 14:27:19 2017 +0100

    docs: Document new <memoryBacking/> elements
    
    Document the extensions made to domain <memoryBacking/> element
    made in previous commits.
    
    Signed-off-by: Michal Privoznik <mprivozn>

commit 1c4f3b56f8f6e325c1073ca85fa1ed532cc40e36
Author:     Jaroslav Safka <jaroslavx.safka>
AuthorDate: Thu Feb 2 13:27:33 2017 +0000
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Feb 9 14:27:19 2017 +0100

    qemu: Add args generation for file memory backing
    
    This patch add support for file memory backing on numa topology.
    
    The specified access mode in memoryBacking can be overriden
    by specifying token memAccess in numa cell.

commit bc6d3121a4c18bbf211fb3f426d1ff34482b6459
Author:     Jaroslav Safka <jaroslavx.safka>
AuthorDate: Thu Feb 2 13:27:32 2017 +0000
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Feb 9 14:27:19 2017 +0100

    conf: Add new xml elements for file memorybacking support
    
    This part introduces new xml elements for file based
    memorybacking support and their parsing.
    (It allows vhost-user to be used without hugepages.)
    
    New xml elements:
    <memoryBacking>
      <source type="file|anonymous"/>
      <access mode="shared|private"/>
      <allocation mode="immediate|ondemand"/>
    </memoryBacking>

commit 48d9e6cdccda5355941ca2eb4c00021d2524955f
Author:     Jaroslav Safka <jaroslavx.safka>
AuthorDate: Thu Feb 2 13:27:31 2017 +0000
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Feb 9 14:27:19 2017 +0100

    qemu_conf: Add param memory_backing_dir
    
    Add new parameter memory_backing_dir where files will be stored when memoryBacking
    source is selected as file.
    
    Value is stored inside char* memoryBackingDir

commit 7c0c5f6d4be228f2c2fb7ab181a0b413bdae0229
Author:     Jaroslav Safka <jaroslavx.safka>
AuthorDate: Thu Feb 2 13:27:30 2017 +0000
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Feb 9 14:27:19 2017 +0100

    qemu, conf: Rename virNumaMemAccess to virDomainMemoryAccess
    
    Rename to avoid duplicate code. Because virDomainMemoryAccess will be
    used in memorybacking for setting default behaviour.
    
    NOTE: The enum cannot be moved to qemu/domain_conf because of headers
    dependency

v3.0.0-151-g0857a3bf5
Comment 7 Luyao Huang 2017-05-27 08:21:29 UTC 
Found a problem when using file as the memory backends:

1. prepare 2 guest which use file backends, and there memory is different:

# virsh dumpxml r7
<domain type='qemu'>
  <name>r7</name>
  <uuid>67c7a123-5415-4136-af62-a2ee098ba6cd</uuid>
  <maxMemory slots='16' unit='KiB'>15242882</maxMemory>
  <memory unit='KiB'>1548287</memory>
  <currentMemory unit='KiB'>1280000</currentMemory>
  <memoryBacking>
    <source type='file'/>
  </memoryBacking>

# virsh dumpxml r7-2
<domain type='kvm'>
  <name>r7-2</name>
  <uuid>c7af9385-47ce-4ffd-8289-f2debb0380f7</uuid>
  <maxMemory slots='16' unit='KiB'>15242882</maxMemory>
  <memory unit='KiB'>3084287</memory>
  <currentMemory unit='KiB'>3084287</currentMemory>
  <memoryBacking>
    <source type='file'/>
  </memoryBacking>

2. start the first guest and then the second:

# virsh start r7
Domain r7 started

# virsh start r7-2
error: Failed to start domain r7-2
error: internal error: process exited while connecting to monitor: 2017-05-27T07:59:23.280312Z qemu-kvm: -chardev pty,id=charserial0: char device redirected to /dev/pts/3 (label charserial0)
2017-05-27T07:59:23.280748Z qemu-kvm: -chardev pty,id=charredir0: char device redirected to /dev/pts/4 (label charredir0)
2017-05-27T07:59:23.289984Z qemu-kvm: -object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,size=1310720000: backing store /var/lib/libvirt/qemu/ram size 0x1f400000 does not match 'size' option 0x4e200000

The problem is: libvirt set the default memory file backing path is /var/lib/libvirt/qemu/ram, and this will effect to all guest, but libvirt won't create this dir, when qemu find there is no /var/lib/libvirt/qemu/ram and it will create it as a file. Then the next guest will use this file as it memory backend, and will fail to start after checking the file size.


Hi Michal,

Could you please help to check if this is expected ? IMHO, Libvirt should give a basic check for this path to avoid some unexpected problem or need mention this in the document.

Thanks in advance for your answer
Comment 8 Michal Privoznik 2017-05-27 10:45:58 UTC 
(In reply to Luyao Huang from comment #7)
> Found a problem when using file as the memory backends:
> 

Thank you, patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2017-May/msg01112.html
Comment 9 Luyao Huang 2017-05-31 01:41:34 UTC 
(In reply to Michal Privoznik from comment #8)
> (In reply to Luyao Huang from comment #7)
> > Found a problem when using file as the memory backends:
> > 
> 
> Thank you, patch proposed upstream:
> 
> https://www.redhat.com/archives/libvir-list/2017-May/msg01112.html

You are welcome :)

Since more patch set need add in this bug, move this bug to assigned.
Comment 10 Michal Privoznik 2017-05-31 13:20:56 UTC 
To POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2017-May/msg00908.html
Comment 11 Luyao Huang 2017-06-08 03:27:27 UTC 
Hi Michal,

Just like we talked, there are some problems when use access mode is shared with hugepage:

1. hugepage + access shared + numa (+ memory device)
guest xml

  <memoryBacking>
    <hugepages/>
    <access mode='shared'/>
  </memoryBacking>

    <numa>
      <cell id='0' cpus='0-3' memory='524288' unit='KiB'/>
      <cell id='1' cpus='4-7' memory='524288' unit='KiB'/>
    </numa>

    <memory model='dimm'>
      <target>
        <size unit='KiB'>524288</size>
        <node>0</node>
      </target>
      <address type='dimm' slot='0' base='0x100000000'/>
    </memory>

qemu cmd line:

-mem-path /dev/hugepages/libvirt/qemu/8-r7 -numa node,nodeid=0,cpus=0-3,mem=512 -numa node,nodeid=1,cpus=4-7,mem=512 -object memory-backend-ram,id=memdimm0,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0

Libvirt didn't add shared=on in the cmd line.

2. when there is no numa element in guest xml, and the memaccess shared won't work:

xml:
  <memoryBacking>
    <hugepages/>
    <access mode='shared'/>
  </memoryBacking>


cmd line:
-mem-path /dev/hugepages/libvirt/qemu/11-r7 

There is no way to add the shared=on
(actually there is a way: libvirt could add one numa node 0 and add the shared=on on that node, but i am not sure if this is ok)


3. the conflict of hugepage and file backend:

xml:

  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB' nodeset='1'/>
    </hugepages>
    <source type='file'/>
    <access mode='shared'/>
  </memoryBacking>

    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB'/>
      <cell id='1' cpus='4-7' memory='512000' unit='KiB'/>
    </numa>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=524288000 -numa node,nodeid=0,cpus=0-3,memdev=ram-node0 -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=524288000 -numa node,nodeid=1,cpus=4-7,memdev=ram-node1

Libvirt use the /var/lib/libvirt/qemu/ram for all the place, but the hugepage cannot be that place, and this make the hugepage element will be useless when configure with the source type='file'.


Could your please help check these problems ?

Thanks in advance for your answer !
Comment 12 Michal Privoznik 2017-06-08 13:35:13 UTC 
(In reply to Luyao Huang from comment #11)
> Hi Michal,
> 
> Just like we talked, there are some problems when use access mode is shared
> with hugepage:
> 
> 1. hugepage + access shared + numa (+ memory device)
> guest xml
> 
>   <memoryBacking>
>     <hugepages/>
>     <access mode='shared'/>
>   </memoryBacking>
> 
>     <numa>
>       <cell id='0' cpus='0-3' memory='524288' unit='KiB'/>
>       <cell id='1' cpus='4-7' memory='524288' unit='KiB'/>
>     </numa>
> 
>     <memory model='dimm'>
>       <target>
>         <size unit='KiB'>524288</size>
>         <node>0</node>
>       </target>
>       <address type='dimm' slot='0' base='0x100000000'/>
>     </memory>
> 
> qemu cmd line:
> 
> -mem-path /dev/hugepages/libvirt/qemu/8-r7 -numa
> node,nodeid=0,cpus=0-3,mem=512 -numa node,nodeid=1,cpus=4-7,mem=512 -object
> memory-backend-ram,id=memdimm0,size=536870912 -device
> pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0
> 
> Libvirt didn't add shared=on in the cmd line.

That's because libvirt had chosen memory-backend-ram. I think we need to use memory-backend-file which has 'shared' attribute.

> 
> 2. when there is no numa element in guest xml, and the memaccess shared
> won't work:
> 
> xml:
>   <memoryBacking>
>     <hugepages/>
>     <access mode='shared'/>
>   </memoryBacking>
> 
> 
> cmd line:
> -mem-path /dev/hugepages/libvirt/qemu/11-r7 
> 
> There is no way to add the shared=on
> (actually there is a way: libvirt could add one numa node 0 and add the
> shared=on on that node, but i am not sure if this is ok)

I don't think that solution would work. I think it would change guest ABI. Ideally, we could have '-mem-access' cmd line argument for qemu which would complete -mem-path. Alternatively, we can use memory-backend-file again. 

> 
> 
> 3. the conflict of hugepage and file backend:
> 
> xml:
> 
>   <memoryBacking>
>     <hugepages>
>       <page size='2048' unit='KiB' nodeset='1'/>
>     </hugepages>
>     <source type='file'/>
>     <access mode='shared'/>
>   </memoryBacking>
> 
>     <numa>
>       <cell id='0' cpus='0-3' memory='512000' unit='KiB'/>
>       <cell id='1' cpus='4-7' memory='512000' unit='KiB'/>
>     </numa>
> 
> cmdline:
> 
> -object
> memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,
> share=yes,size=524288000 -numa node,nodeid=0,cpus=0-3,memdev=ram-node0
> -object
> memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,
> share=yes,size=524288000 -numa node,nodeid=1,cpus=4-7,memdev=ram-node1

Oh, we don't have a preference documented. I think the correct cmd line would be the following:

-object memory-backend-file,id=ram-node0,mem-path=/path/to/2M/hugepages
-object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram.

At any rate, this bug is getting closely related to bug 1458638.
Comment 13 Luyao Huang 2017-06-09 10:00:09 UTC 
(In reply to Michal Privoznik from comment #12)
> (In reply to Luyao Huang from comment #11)
> > Hi Michal,
> > 
> > Just like we talked, there are some problems when use access mode is shared
> > with hugepage:

[...]

> > 
> > Libvirt didn't add shared=on in the cmd line.
> 
> That's because libvirt had chosen memory-backend-ram. I think we need to use
> memory-backend-file which has 'shared' attribute.
> 

Yes, that is what i want to say, and also the we need use memory-backend-file with shared attribute when build numa command line.

> > 
> > 2. when there is no numa element in guest xml, and the memaccess shared
> > won't work:
> > 

[...]

> > 
> > There is no way to add the shared=on
> > (actually there is a way: libvirt could add one numa node 0 and add the
> > shared=on on that node, but i am not sure if this is ok)
> 
> I don't think that solution would work. I think it would change guest ABI.
> Ideally, we could have '-mem-access' cmd line argument for qemu which would
> complete -mem-path. Alternatively, we can use memory-backend-file again. 
> 

Okay, so this will need a new qemu bug to against qemu and ask them support this.

> > 
> > 
> > 3. the conflict of hugepage and file backend:
> > 

[...]

> 
> -object memory-backend-file,id=ram-node0,mem-path=/path/to/2M/hugepages
> -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram.
> 

Thanks a lot for your reply

> At any rate, this bug is getting closely related to bug 1458638.
Comment 14 Michal Privoznik 2017-06-13 14:28:06 UTC 
Patches posted upstream:


https://www.redhat.com/archives/libvir-list/2017-June/msg00569.html
https://www.redhat.com/archives/libvir-list/2017-June/msg00568.html
Comment 15 Michal Privoznik 2017-06-13 16:14:16 UTC 
To POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2017-June/msg00332.html
http://post-office.corp.redhat.com/archives/rhvirt-patches/2017-June/msg00333.html
Comment 16 Luyao Huang 2017-06-14 07:27:27 UTC 
Hi Michal,

I have test your patch set and i have found 2 problems:

1. hugepages with default page size + source file:

xml:

  <memoryBacking>
    <hugepages/>
    <source type='file'/>
    <access mode='shared'/>
  </memoryBacking>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1

In this case libvirt still use the /var/lib/libvirt/qem/ram


2. no hugepage + memAccess shared:

    <numa>
      <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
      <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
    </numa>

guest will failed to start:

# virsh start r7
error: Failed to start domain r7
error: internal error: process exited while connecting to monitor: 2017-06-14T07:19:29.864949Z qemu-kvm: -chardev pty,id=charserial0: char device redirected to /dev/pts/2 (label charserial0)
2017-06-14T07:19:29.865534Z qemu-kvm: -chardev pty,id=charredir0: char device redirected to /dev/pts/3 (label charredir0)
2017-06-14T07:19:29.875530Z qemu-kvm: -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/4-r7,share=yes,size=536870912: can't open backing store /dev/hugepages/libvirt/qemu/4-r7 for guest RAM: Permission denied

And check the full qemu command line:

-object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/4-r7,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-ram,id=ram-node1,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1

And after check the libvirtd log, seems that libvirt didn't create the hugepage path for guest but still create this unexisted path in cmdline. But i think use shared without hugepage is a invalid config, maybe libvirt should report error
Comment 17 Michal Privoznik 2017-06-14 08:49:45 UTC 
(In reply to Luyao Huang from comment #16)
> Hi Michal,
> 
> I have test your patch set and i have found 2 problems:
> 
> 1. hugepages with default page size + source file:
> 
> xml:
> 
>   <memoryBacking>
>     <hugepages/>
>     <source type='file'/>
>     <access mode='shared'/>
>   </memoryBacking>
> 
> cmdline:
> 
> -object
> memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,
> size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object
> memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,
> share=yes,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1
> 
> In this case libvirt still use the /var/lib/libvirt/qem/ram

I think this is correct. Your config says <source type='file'/>. And the preference function now is as follows: use hugepages for explicitly configured nodes, and use /var/lib/libvirt/qemu/ram for everything else (see comment 12). Since you haven't explicitly configured hugepages for any numa node, <source type='file'/> was applied.

> 
> 
> 2. no hugepage + memAccess shared:
> 
>     <numa>
>       <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
>       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
>     </numa>
> 
> guest will failed to start:
> 
> # virsh start r7
> error: Failed to start domain r7
> error: internal error: process exited while connecting to monitor:
> 2017-06-14T07:19:29.864949Z qemu-kvm: -chardev pty,id=charserial0: char
> device redirected to /dev/pts/2 (label charserial0)
> 2017-06-14T07:19:29.865534Z qemu-kvm: -chardev pty,id=charredir0: char
> device redirected to /dev/pts/3 (label charredir0)
> 2017-06-14T07:19:29.875530Z qemu-kvm: -object
> memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/
> libvirt/qemu/4-r7,share=yes,size=536870912: can't open backing store
> /dev/hugepages/libvirt/qemu/4-r7 for guest RAM: Permission denied
> 
> And check the full qemu command line:
> 
> -object
> memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/
> libvirt/qemu/4-r7,share=yes,size=536870912 -numa
> node,nodeid=0,cpus=0-2,memdev=ram-node0 -object
> memory-backend-ram,id=ram-node1,size=536870912 -numa
> node,nodeid=1,cpus=3-5,memdev=ram-node1
> 

Yup, this is a bug. Patch posted upstream:

https://www.redhat.com/archives/libvir-list/2017-June/msg00595.html
Comment 18 Michal Privoznik 2017-06-14 14:59:26 UTC 
And here's v2 of the backport:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2017-June/msg00347.html
Comment 19 Luyao Huang 2017-06-15 01:41:45 UTC 
(In reply to Michal Privoznik from comment #17)
> (In reply to Luyao Huang from comment #16)
> > Hi Michal,
> > 
> > I have test your patch set and i have found 2 problems:
> > 
[...]
> 
> I think this is correct. Your config says <source type='file'/>. And the
> preference function now is as follows: use hugepages for explicitly
> configured nodes, and use /var/lib/libvirt/qemu/ram for everything else (see
> comment 12). Since you haven't explicitly configured hugepages for any numa
> node, <source type='file'/> was applied.
> 

Okay, make sense.

> > 
> > 
> > 2. no hugepage + memAccess shared:
> > 
> >     <numa>
> >       <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
> >       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
> >     </numa>
> > 
> > guest will failed to start:
> > 
> > # virsh start r7
> > error: Failed to start domain r7
> > error: internal error: process exited while connecting to monitor:
> > 2017-06-14T07:19:29.864949Z qemu-kvm: -chardev pty,id=charserial0: char
> > device redirected to /dev/pts/2 (label charserial0)
> > 2017-06-14T07:19:29.865534Z qemu-kvm: -chardev pty,id=charredir0: char
> > device redirected to /dev/pts/3 (label charredir0)
> > 2017-06-14T07:19:29.875530Z qemu-kvm: -object
> > memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/
> > libvirt/qemu/4-r7,share=yes,size=536870912: can't open backing store
> > /dev/hugepages/libvirt/qemu/4-r7 for guest RAM: Permission denied
> > 
> > And check the full qemu command line:
> > 
> > -object
> > memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/
> > libvirt/qemu/4-r7,share=yes,size=536870912 -numa
> > node,nodeid=0,cpus=0-2,memdev=ram-node0 -object
> > memory-backend-ram,id=ram-node1,size=536870912 -numa
> > node,nodeid=1,cpus=3-5,memdev=ram-node1
> > 
> 
> Yup, this is a bug. Patch posted upstream:
> 
> https://www.redhat.com/archives/libvir-list/2017-June/msg00595.html

Hi Michal,

Test with libvirt-3.2.0-10.el7.x86_64, there are 3 problems:

1. no memory backing + memAccess shared:

xml:
    <numa>
      <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
      <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
    </numa>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-ram,id=ram-node1,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1

I didn't set the memory backend is file, but libvirt use the /var/lib/libvirt/qem/ram. I think this config is not right, libvirt should report error.

2. hugepage + dimm device:

xml:

  <memoryBacking>
    <hugepages/>
  </memoryBacking>

    <numa>
      <cell id='0' cpus='0-2' memory='524288' unit='KiB'/>
      <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
    </numa>

    <memory model='dimm' access='private'>
      <target>
        <size unit='KiB'>524287</size>
        <node>0</node>
      </target>
      <address type='dimm' slot='0'/>
    </memory>


cmdline:

-mem-path /dev/hugepages/libvirt/qemu/2-r7 -numa node,nodeid=0,cpus=0-2,mem=512 -numa node,nodeid=1,cpus=3-5,mem=512 -object memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0

You can see that libvirt use the /var/lib/libvirt/qemu/ram for the dimm device, but this is not right, should use the hugepage path.

3. hugepages + memAccess shared:

xml:

  <memoryBacking>
    <hugepages/>
  </memoryBacking>

  <cpu mode='host-model' check='partial'>
    <model fallback='allow'>qemu64</model>
    <numa>
      <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
      <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
    </numa>
  </cpu>

cmdline:

 -object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-ram,id=ram-node1,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1

This just like the problem 2, libvirt use /var/lib/libvirt/qem/ram instead of hugepage path and node 1 didn't use the hugepage.
Comment 20 Michal Privoznik 2017-06-15 09:03:38 UTC 
(In reply to Luyao Huang from comment #19)
> (In reply to Michal Privoznik from comment #17)
> > (In reply to Luyao Huang from comment #16)
> > > Hi Michal,
> > > 
> > > I have test your patch set and i have found 2 problems:
> > > 
> [...]
> > 
> > I think this is correct. Your config says <source type='file'/>. And the
> > preference function now is as follows: use hugepages for explicitly
> > configured nodes, and use /var/lib/libvirt/qemu/ram for everything else (see
> > comment 12). Since you haven't explicitly configured hugepages for any numa
> > node, <source type='file'/> was applied.
> > 
> 
> Okay, make sense.
> 
> > > 
> > > 
> > > 2. no hugepage + memAccess shared:
> > > 
> > >     <numa>
> > >       <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
> > >       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
> > >     </numa>
> > > 
> > > guest will failed to start:
> > > 
> > > # virsh start r7
> > > error: Failed to start domain r7
> > > error: internal error: process exited while connecting to monitor:
> > > 2017-06-14T07:19:29.864949Z qemu-kvm: -chardev pty,id=charserial0: char
> > > device redirected to /dev/pts/2 (label charserial0)
> > > 2017-06-14T07:19:29.865534Z qemu-kvm: -chardev pty,id=charredir0: char
> > > device redirected to /dev/pts/3 (label charredir0)
> > > 2017-06-14T07:19:29.875530Z qemu-kvm: -object
> > > memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/
> > > libvirt/qemu/4-r7,share=yes,size=536870912: can't open backing store
> > > /dev/hugepages/libvirt/qemu/4-r7 for guest RAM: Permission denied
> > > 
> > > And check the full qemu command line:
> > > 
> > > -object
> > > memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/
> > > libvirt/qemu/4-r7,share=yes,size=536870912 -numa
> > > node,nodeid=0,cpus=0-2,memdev=ram-node0 -object
> > > memory-backend-ram,id=ram-node1,size=536870912 -numa
> > > node,nodeid=1,cpus=3-5,memdev=ram-node1
> > > 
> > 
> > Yup, this is a bug. Patch posted upstream:
> > 
> > https://www.redhat.com/archives/libvir-list/2017-June/msg00595.html
> 
> Hi Michal,
> 
> Test with libvirt-3.2.0-10.el7.x86_64, there are 3 problems:
> 
> 1. no memory backing + memAccess shared:
> 
> xml:
>     <numa>
>       <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
>       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
>     </numa>
> 
> cmdline:
> 
> -object
> memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,
> size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object
> memory-backend-ram,id=ram-node1,size=536870912 -numa
> node,nodeid=1,cpus=3-5,memdev=ram-node1
> 
> I didn't set the memory backend is file, but libvirt use the
> /var/lib/libvirt/qem/ram. I think this config is not right, libvirt should
> report error.

Why is that? I'm failing to see why this is a problem. This xml requests shared memAccess and the only way we can satisfy this is by using /var/lib/libvirt/qemu/ram. Or, in the future if we have -mem-shared argument in qemu (as discussed earlier) we don't have to use memory-backend-file. Anyway, I think this is okay.

> 
> 2. hugepage + dimm device:
> 
> xml:
> 
>   <memoryBacking>
>     <hugepages/>
>   </memoryBacking>
> 
>     <numa>
>       <cell id='0' cpus='0-2' memory='524288' unit='KiB'/>
>       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
>     </numa>
> 
>     <memory model='dimm' access='private'>
>       <target>
>         <size unit='KiB'>524287</size>
>         <node>0</node>
>       </target>
>       <address type='dimm' slot='0'/>
>     </memory>
> 
> 
> cmdline:
> 
> -mem-path /dev/hugepages/libvirt/qemu/2-r7 -numa
> node,nodeid=0,cpus=0-2,mem=512 -numa node,nodeid=1,cpus=3-5,mem=512 -object
> memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,
> size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0
> 
> You can see that libvirt use the /var/lib/libvirt/qemu/ram for the dimm
> device, but this is not right, should use the hugepage path.

Okay, file a different bug for this. This bug is growing out of hands. There are just to many places we allow users to configure all the knobs and expected result it not always clear. I'd like to get this one closed and address partial issues separately instead of covering everything in one bug.

> 
> 3. hugepages + memAccess shared:
> 
> xml:
> 
>   <memoryBacking>
>     <hugepages/>
>   </memoryBacking>
> 
>   <cpu mode='host-model' check='partial'>
>     <model fallback='allow'>qemu64</model>
>     <numa>
>       <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
>       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
>     </numa>
>   </cpu>
> 
> cmdline:
> 
>  -object
> memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,
> size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object
> memory-backend-ram,id=ram-node1,size=536870912 -numa
> node,nodeid=1,cpus=3-5,memdev=ram-node1
> 
> This just like the problem 2, libvirt use /var/lib/libvirt/qem/ram instead
> of hugepage path and node 1 didn't use the hugepage.

Yes, this is the very same problem as 2). It can be covered in the same bug as 2).
Comment 21 Luyao Huang 2017-06-15 09:56:36 UTC 
(In reply to Michal Privoznik from comment #20)
> (In reply to Luyao Huang from comment #19)
> > (In reply to Michal Privoznik from comment #17)
> > > (In reply to Luyao Huang from comment #16)
> > > > Hi Michal,
> > > > 
[...]
> > 
> > I didn't set the memory backend is file, but libvirt use the
> > /var/lib/libvirt/qem/ram. I think this config is not right, libvirt should
> > report error.
> 
> Why is that? I'm failing to see why this is a problem. This xml requests
> shared memAccess and the only way we can satisfy this is by using
> /var/lib/libvirt/qemu/ram. Or, in the future if we have -mem-shared argument
> in qemu (as discussed earlier) we don't have to use memory-backend-file.
> Anyway, I think this is okay.
> 

I mean, i didn't specify the memory backing source to file in xml, but libvirt will use the /var/lib/libvirt/qemu/ram. And another case is when i specify the 
source type to anonymous, libvirt still will use /var/lib/libvirt/qemu/ram:

  <memoryBacking>
    <source type='anonymous'/>
  </memoryBacking>

  <cpu mode='host-model' check='partial'>
    <model fallback='allow'>qemu64</model>
    <numa>
      <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
      <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
    </numa>
  </cpu>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0

And this looks not that right, isn't it ?

> > 
> > 2. hugepage + dimm device:
> > 
> > xml:
> > 
> >   <memoryBacking>
> >     <hugepages/>
> >   </memoryBacking>
> > 
> >     <numa>
> >       <cell id='0' cpus='0-2' memory='524288' unit='KiB'/>
> >       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
> >     </numa>
> > 
> >     <memory model='dimm' access='private'>
> >       <target>
> >         <size unit='KiB'>524287</size>
> >         <node>0</node>
> >       </target>
> >       <address type='dimm' slot='0'/>
> >     </memory>
> > 
> > 
> > cmdline:
> > 
> > -mem-path /dev/hugepages/libvirt/qemu/2-r7 -numa
> > node,nodeid=0,cpus=0-2,mem=512 -numa node,nodeid=1,cpus=3-5,mem=512 -object
> > memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,
> > size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0
> > 
> > You can see that libvirt use the /var/lib/libvirt/qemu/ram for the dimm
> > device, but this is not right, should use the hugepage path.
> 
> Okay, file a different bug for this. This bug is growing out of hands. There
> are just to many places we allow users to configure all the knobs and
> expected result it not always clear. I'd like to get this one closed and
> address partial issues separately instead of covering everything in one bug.
> 
> > 
> > 3. hugepages + memAccess shared:
[...]
> 
> Yes, this is the very same problem as 2). It can be covered in the same bug
> as 2).

Okay, i remember that there is already a bug (bug 1458638) use the same config, i will add a comment in that bug.

Thanks a lot for your reply.
Comment 22 Michal Privoznik 2017-06-15 11:54:38 UTC 
(In reply to Luyao Huang from comment #21)
> (In reply to Michal Privoznik from comment #20)
> > (In reply to Luyao Huang from comment #19)
> > > (In reply to Michal Privoznik from comment #17)
> > > > (In reply to Luyao Huang from comment #16)
> > > > > Hi Michal,
> > > > > 
> [...]
> > > 
> > > I didn't set the memory backend is file, but libvirt use the
> > > /var/lib/libvirt/qem/ram. I think this config is not right, libvirt should
> > > report error.
> > 
> > Why is that? I'm failing to see why this is a problem. This xml requests
> > shared memAccess and the only way we can satisfy this is by using
> > /var/lib/libvirt/qemu/ram. Or, in the future if we have -mem-shared argument
> > in qemu (as discussed earlier) we don't have to use memory-backend-file.
> > Anyway, I think this is okay.
> > 
> 
> I mean, i didn't specify the memory backing source to file in xml, but
> libvirt will use the /var/lib/libvirt/qemu/ram.

Well, libvirt chose to use memory-backend-file because only by choosing that it can meet memAccess='shared' requirement. So even though you haven't said source file, libvirt had no other option to choose from. It would if only qemu had -mem-shared argument (or whatever it would be called). But so far it doesn't.

> And another case is when i
> specify the 
> source type to anonymous, libvirt still will use /var/lib/libvirt/qemu/ram:
> 
>   <memoryBacking>
>     <source type='anonymous'/>
>   </memoryBacking>
> 
>   <cpu mode='host-model' check='partial'>
>     <model fallback='allow'>qemu64</model>
>     <numa>
>       <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
>       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
>     </numa>
>   </cpu>
> 
> cmdline:
> 
> -object
> memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,
> size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0
> 
> And this looks not that right, isn't it ?

Why not? The XML config says you want numa node to be shared. So it is. The numa cell configuration is more specific than generic overall configuration. And the rule is - the most specific configuration knobs have precedence over generic ones.

> 
> > > 
> > > 2. hugepage + dimm device:
> > > 
> > > xml:
> > > 
> > >   <memoryBacking>
> > >     <hugepages/>
> > >   </memoryBacking>
> > > 
> > >     <numa>
> > >       <cell id='0' cpus='0-2' memory='524288' unit='KiB'/>
> > >       <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
> > >     </numa>
> > > 
> > >     <memory model='dimm' access='private'>
> > >       <target>
> > >         <size unit='KiB'>524287</size>
> > >         <node>0</node>
> > >       </target>
> > >       <address type='dimm' slot='0'/>
> > >     </memory>
> > > 
> > > 
> > > cmdline:
> > > 
> > > -mem-path /dev/hugepages/libvirt/qemu/2-r7 -numa
> > > node,nodeid=0,cpus=0-2,mem=512 -numa node,nodeid=1,cpus=3-5,mem=512 -object
> > > memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,
> > > size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0
> > > 
> > > You can see that libvirt use the /var/lib/libvirt/qemu/ram for the dimm
> > > device, but this is not right, should use the hugepage path.
> > 
> > Okay, file a different bug for this. This bug is growing out of hands. There
> > are just to many places we allow users to configure all the knobs and
> > expected result it not always clear. I'd like to get this one closed and
> > address partial issues separately instead of covering everything in one bug.
> > 
> > > 
> > > 3. hugepages + memAccess shared:
> [...]
> > 
> > Yes, this is the very same problem as 2). It can be covered in the same bug
> > as 2).
> 
> Okay, i remember that there is already a bug (bug 1458638) use the same
> config, i will add a comment in that bug.

Thanks.
Comment 23 Luyao Huang 2017-06-16 02:33:17 UTC 
(In reply to Michal Privoznik from comment #22)
> (In reply to Luyao Huang from comment #21)
> > (In reply to Michal Privoznik from comment #20)
> > > (In reply to Luyao Huang from comment #19)
> > > > (In reply to Michal Privoznik from comment #17)
> > > > > (In reply to Luyao Huang from comment #16)
> > > > > > Hi Michal,
> > > > > > 
> > [...]
> > > > 
> > > > I didn't set the memory backend is file, but libvirt use the
> > > > /var/lib/libvirt/qem/ram. I think this config is not right, libvirt should
> > > > report error.
> > > 
> > > Why is that? I'm failing to see why this is a problem. This xml requests
> > > shared memAccess and the only way we can satisfy this is by using
> > > /var/lib/libvirt/qemu/ram. Or, in the future if we have -mem-shared argument
> > > in qemu (as discussed earlier) we don't have to use memory-backend-file.
> > > Anyway, I think this is okay.
> > > 
> > 
> > I mean, i didn't specify the memory backing source to file in xml, but
> > libvirt will use the /var/lib/libvirt/qemu/ram.
> 
> Well, libvirt chose to use memory-backend-file because only by choosing that
> it can meet memAccess='shared' requirement. So even though you haven't said
> source file, libvirt had no other option to choose from. It would if only
> qemu had -mem-shared argument (or whatever it would be called). But so far
> it doesn't.

Okay, i see, thanks for clearly explanation.

> 
> > And another case is when i
> > specify the 
> > source type to anonymous, libvirt still will use /var/lib/libvirt/qemu/ram:
> > 
[...]
> > -object
> > memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qem/ram,share=yes,
> > size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0
> > 
> > And this looks not that right, isn't it ?
> 
> Why not? The XML config says you want numa node to be shared. So it is. The
> numa cell configuration is more specific than generic overall configuration.
> And the rule is - the most specific configuration knobs have precedence over
> generic ones.
> 

Okay, i understand now, so the document looks need modify. 

> > 
> > > > 
> > > > 2. hugepage + dimm device:
> > > > 
> > > > xml:
> > > > 
[...]
> > 
> > Okay, i remember that there is already a bug (bug 1458638) use the same
> > config, i will add a comment in that bug.
> 
> Thanks.

Your are welcome, and thanks a lot for your quick reply
Comment 24 Luyao Huang 2017-06-16 03:44:15 UTC 
Verify this bug with libvirt-3.2.0-10.el7.x86_64:

prepare a guest have xml with different element which will effect by memory backing:

...
    <numa>
      <cell id='0' cpus='0-2' memory='524288' unit='KiB' memAccess='shared'/>
      <cell id='1' cpus='3-5' memory='524288' unit='KiB'/>
    </numa>
...
    <memory model='dimm' access='private'>
      <target>
        <size unit='KiB'>524287</size>
        <node>0</node>
      </target>
      <address type='dimm' slot='0'/>
    </memory>
    <memory model='dimm'>
      <target>
        <size unit='KiB'>524287</size>
        <node>0</node>
      </target>
      <address type='dimm' slot='1'/>
    </memory>
    <memory model='nvdimm' access='shared'>
      <source>
        <path>/tmp/nvdimm</path>
      </source>
      <target>
        <size unit='KiB'>524288</size>
        <node>1</node>
        <label>
          <size unit='KiB'>128</size>
        </label>
      </target>
      <address type='dimm' slot='2'/>
    </memory>
    <memory model='nvdimm'>
      <source>
        <path>/tmp/nvdimm</path>
      </source>
      <target>
        <size unit='KiB'>524288</size>
        <node>1</node>
        <label>
          <size unit='KiB'>128</size>
        </label>
      </target>
      <address type='dimm' slot='3'/>
    </memory>
...

S1: memory backing source type is file:

add xml:

  <memoryBacking>
    <source type='file'/>
  </memoryBacking>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1 -object memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0 -object memory-backend-file,id=memdimm1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -device pc-dimm,node=0,memdev=memdimm1,id=dimm1,slot=1 -object memory-backend-file,id=memnvdimm2,prealloc=yes,mem-path=/tmp/nvdimm,share=yes,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm2,id=nvdimm2,slot=2 -object memory-backend-file,id=memnvdimm3,prealloc=yes,mem-path=/tmp/nvdimm,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm3,id=nvdimm3,slot=3

S2: memory backing source type is anonymous:

add xml:

  <memoryBacking>
    <source type='anonymous'/>
  </memoryBacking>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-ram,id=ram-node1,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1 -object memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0 -object memory-backend-file,id=memdimm1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -device pc-dimm,node=0,memdev=memdimm1,id=dimm1,slot=1 -object memory-backend-file,id=memnvdimm2,prealloc=yes,mem-path=/tmp/nvdimm,share=yes,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm2,id=nvdimm2,slot=2 -object memory-backend-file,id=memnvdimm3,prealloc=yes,mem-path=/tmp/nvdimm,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm3,id=nvdimm3,slot=3


S3: hugepages + file backing:

Add xml:

  <memoryBacking>
    <hugepages/>
    <source type='file'/>
  </memoryBacking>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1 -object memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0 -object memory-backend-file,id=memdimm1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -device pc-dimm,node=0,memdev=memdimm1,id=dimm1,slot=1 -object memory-backend-file,id=memnvdimm2,prealloc=yes,mem-path=/tmp/nvdimm,share=yes,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm2,id=nvdimm2,slot=2 -object memory-backend-file,id=memnvdimm3,prealloc=yes,mem-path=/tmp/nvdimm,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm3,id=nvdimm3,slot=3

This is not right, but it will track in another bug.


S4: hugepages + anonymous backing:

# virsh edit r7
error: unsupported configuration: hugepages are not allowed with anonymous memory source
Failed. Try again? [y,n,i,f,?]:


S5: Test memory backend private:

Add xml:

  <memoryBacking>
    <access mode='private'/>
  </memoryBacking>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1 -object memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0 -object memory-backend-file,id=memdimm1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -device pc-dimm,node=0,memdev=memdimm1,id=dimm1,slot=1 -object memory-backend-file,id=memnvdimm2,prealloc=yes,mem-path=/tmp/nvdimm,share=yes,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm2,id=nvdimm2,slot=2 -object memory-backend-file,id=memnvdimm3,prealloc=yes,mem-path=/tmp/nvdimm,share=no,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm3,id=nvdimm3,slot=3

S6: Test memory backend shared:

xml:

  <memoryBacking>
    <access mode='shared'/>
  </memoryBacking>

cmdline:

-object memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -object memory-backend-file,id=ram-node1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -numa node,nodeid=1,cpus=3-5,memdev=ram-node1 -object memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=no,size=536870912 -device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0 -object memory-backend-file,id=memdimm1,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912 -device pc-dimm,node=0,memdev=memdimm1,id=dimm1,slot=1 -object memory-backend-file,id=memnvdimm2,prealloc=yes,mem-path=/tmp/nvdimm,share=yes,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm2,id=nvdimm2,slot=2 -object memory-backend-file,id=memnvdimm3,prealloc=yes,mem-path=/tmp/nvdimm,share=yes,size=536870912 -device nvdimm,node=1,label-size=131072,memdev=memnvdimm3,id=nvdimm3,slot=3


S7: allocation mode is immediate

xml:

  <memoryBacking>
    <allocation mode='immediate'/>
  </memoryBacking>

cmdline:

...-mem-prealloc...
Comment 25 errata-xmlrpc 2017-08-01 17:06:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 26 errata-xmlrpc 2017-08-01 23:48:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 27 errata-xmlrpc 2017-08-02 01:25:04 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846