Bug 1915216 - VM failed to start with pmem enabled.
Summary: VM failed to start with pmem enabled.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.4
Hardware: ppc64le
OS: Linux
medium
medium
Target Milestone: rc
: 8.5
Assignee: Virtualization Maintenance
QA Contact: Min Deng
URL:
Whiteboard:
Depends On: 1524652 1568223
Blocks: 1916117
TreeView+ depends on / blocked
 
Reported: 2021-01-12 09:30 UTC by Dan Zheng
Modified: 2021-11-16 08:05 UTC (History)
11 users (show)

Fixed In Version: qemu-kvm-6.0.0-16.module+el8.5.0+10848+2dccc46d
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-16 07:51:11 UTC
Type: Feature Request
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
IBM Linux Technology Center 190785 0 None None None 2021-01-13 10:28:23 UTC
Red Hat Product Errata RHBA-2021:4684 0 None None None 2021-11-16 07:51:34 UTC

Description Dan Zheng 2021-01-12 09:30:50 UTC
Description of problem:
VM failed to start with pmem enabled.

Version-Release number of selected component (if applicable):

libvirt-6.5.0-1.module+el8.3.0+7323+d54bb644.ppc64le
qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca.ppc64le
kernel-4.18.0-221.el8.ppc64le

How reproducible:
100%

Steps to Reproduce:
1.With alignsize + pmem
1.1 Define vm with below:
  <memory model='nvdimm'>
      <source>
        <path>/tmp/nvdimm</path>
        <alignsize unit='KiB'>2048</alignsize>
        <pmem/>
      </source>
      <target>
        <size unit='KiB'>524288</size>
        <node>1</node>
        <label>
          <size unit='KiB'>128</size>
        </label>
        <readonly/>
      </target>
      <address type='dimm' slot='1'/>
    </memory>

1.2 Start vm
# virsh start vm2
error: Failed to start domain vm2
error: internal error: process exited while connecting to monitor: 2020-07-20T12:09:31.096911Z qemu-kvm: -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/tmp/nvdimm-private-2,size=268697600,align=2097152,pmem=on,host-nodes=0,policy=preferred: Lack of libpmem support while setting the 'pmem=on' of memory-backend-file. We can't ensure data persistence.



2.
3.

Actual results:
See above

Expected results:
VM should be started correctly.




Additional info:

Comment 1 Dan Zheng 2021-01-12 09:42:35 UTC
This is spited from bz1568223. See https://bugzilla.redhat.com/show_bug.cgi?id=1568223#c44 for more details.

Comment 2 IBM Bug Proxy 2021-01-13 15:11:18 UTC
------- Comment From vajain21.com 2021-01-13 10:06 EDT-------
I think PMDK/libpmem support hasnt yet landed in RHEL8.3 so qemu is most probably compiled with libpmem=off. This means that support for '<pmem/>' tag in domain-xml is not in place yet.

Requesting you to please check removing the '<pmem/>' tag from the domain-xml and see if its working fine

Thanks,

Comment 3 Dan Zheng 2021-01-15 01:24:36 UTC
IBM,
Yes, just as you said, qemu has not ready to support <pmem/> which depends on bz1524652. Qemu will get re-compiled after 1524652 is fixed.

So this bz is just an enhancement to make sure we may have this supported in future (it is supported on x86_64.)

It works if '<pmem/>' is removed from domain xml.


Dan
Thanks

Comment 4 Michal Privoznik 2021-01-19 12:04:04 UTC
Libvirt checks whether QEMU supports pmem:

  https://gitlab.com/libvirt/libvirt/-/commit/55b4fc78b6397caf7eff73a0dc7fbb8892e98d49

And it also adds pmem=on only if supported by QEMU:

  https://gitlab.com/libvirt/libvirt/-/commit/82576d8f35e

If pmem is not supported then Libvirt errors out. The problem here is that QEMU reports 'pmem' attribute always and only at runtime it decides whether to error out (if compiled without libpmem) or no (compiled with libpmem). Therefore, I think this is a QEMU bug.

Comment 5 Michal Privoznik 2021-01-19 12:51:54 UTC
I think we need something like this:

https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg04558.html

Comment 6 Min Deng 2021-01-26 06:15:10 UTC
I tried the issue with build qemu-kvm-5.2.0-3.scrmod+el8.4.0+9533+d0097f55.wrb210120.ppc64le
cli,
1.boot up a guest with 
/usr/libexec/qemu-kvm -name avocado-vt-vm1 -sandbox on -machine pseries -nodefaults -device VGA,bus=pci.0,addr=0x2 -m 20480 -smp 2,maxcpus=2,cores=1,threads=1,sockets=2 -cpu host -chardev socket,id=chardev_serial0,server,path=/tmp/SS,nowait -device spapr-vty,id=serial0,reg=0x30000000,chardev=chardev_serial0 -device pci-ohci,id=usbtest,bus=pci.0,addr=0x3 -device usb-tablet,id=usb-tablet1,bus=usbtest.0,port=1 -device usb-hub,id=usb-testdev,bus=usbtest.0,port=2 -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4 -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=rhel840-ppc64le-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 -device scsi-hd,id=image1,drive=drive_image1,write-cache=on -vnc :1 -rtc base=utc,clock=host -boot menu=off,order=cdn,once=c,strict=off -enable-kvm -monitor stdio -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/tmp/nvdimm-private-2,size=268697600,align=2097152,pmem=on,host-nodes=0,policy=preferre

Actual results,
The guest can't boot up obviously since it's not supported.
qemu-kvm: -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/tmp/nvdimm-private-2,size=268697600,align=2097152,pmem=on,host-nodes=0,policy=preferred: Lack of libpmem support while setting the 'pmem=on' of memory-backend-file. We can't ensure data persistence.

Expected results,
From qemu-kvm QE's perspective, if it's not supported, the guest shouldn't boot up,it's obvious. what's the expected results for libvirt then ?

Thanks.

Comment 7 Michal Privoznik 2021-01-26 07:42:50 UTC
(In reply to Min Deng from comment #6)

> Expected results,
> From qemu-kvm QE's perspective, if it's not supported, the guest shouldn't
> boot up,it's obvious. what's the expected results for libvirt then ?
> 

Yeah, from libvirt's POV the guest shouldn't be able to boot because it can't unsure <pmem/>. We really need that patch.

Comment 8 Michal Privoznik 2021-01-26 08:03:47 UTC
I've resend the patch here:

https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg06561.html

Comment 10 David Gibson 2021-02-16 04:38:01 UTC
Igor, Eduardo,

I see that the patch from comment 8 isn't merged yet.  Is it in either of your queues?  If not, what should we do to expedite it?

Comment 11 Eduardo Habkost 2021-02-16 22:47:51 UTC
(In reply to David Gibson from comment #10)
> Igor, Eduardo,
> 
> I see that the patch from comment 8 isn't merged yet.  Is it in either of
> your queues?  If not, what should we do to expedite it?

Sorry for missing it.  Queued upstream:
https://lore.kernel.org/qemu-devel/20210216222357.GL4070@habkost.net

Comment 12 IBM Bug Proxy 2021-03-08 08:42:09 UTC
------- Comment From hannsj_uhl.com 2021-03-08 03:35 EDT-------
(In reply to comment #16)
> (In reply to David Gibson from comment #10)
> > Igor, Eduardo,
> >
> > I see that the patch from comment 8 isn't merged yet.  Is it in either of
> > your queues?  If not, what should we do to expedite it?
> Sorry for missing it.  Queued upstream:
> https://lore.kernel.org/qemu-devel/20210216222357.GL4070@habkost.net
.
... which I think is now upstream accepted as git commit
https://git.qemu.org/gitweb.cgi?p=qemu.git;a=commit;h=def835f0da0d153b397071e6bb8f2b46f51f96b4
("hostmem: Don't report pmem attribute if unsupported")
...

Comment 15 Min Deng 2021-05-12 08:27:08 UTC
Hi Danilo,
Can you help to check how to proceed this bz because it's in DevMissed status. Thanks.
By the way, bz1918555 has the same situation.
Min

Comment 16 Min Deng 2021-05-13 08:13:32 UTC
Verified the bug on the following builds,
qemu-kvm-6.0.0-16.module+el8.5.0+10848+2dccc46d.ppc64le
Steps please refer to comment 6,
Actual results,
qemu-kvm: -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/tmp/nvdimm-private-2,size=268697600,align=2097152,pmem=on,host-nodes=0,policy=preferred: Invalid parameter 'pmem'
Expected results,
Qemu will not report "pmem" any more.

Base on above comments, the bug should be fixed, thanks a lot.

Comment 19 errata-xmlrpc 2021-11-16 07:51:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4684


Note You need to log in before you can comment on or make changes to this bug.