Bug 1628098

Summary: [Intel 7.7 BUG][KVM][Crystal Ridge]object_get_canonical_path_component: assertion failed: (obj->parent != NULL)
Product: Red Hat Enterprise Linux 7 Reporter: Yumei Huang <yuhuang>
Component: qemu-kvm-rhevAssignee: Paul Lai (Intel) <plai>
Status: CLOSED ERRATA QA Contact: Yumei Huang <yuhuang>
Severity: high Docs Contact:
Priority: high    
Version: 7.6CC: chayang, jinzhao, juzhang, mrezanin, pagupta, plai, Robert.Hu, virt-maint, xuelian.guo, yi.z.zhang
Target Milestone: rcKeywords: Regression
Target Release: 7.7   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.12.0-22.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-22 09:18:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1649160    

Description Yumei Huang 2018-09-12 08:31:01 UTC
Description of problem:
Boot guest with nvdimm device backed by a regular file on host, set pmem=on, qemu core dumped.


Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.12.0-14.el7

How reproducible:
always

Steps to Reproduce:
1. Boot qemu with pmem=on for memory backend backed by a regular file on host

# /usr/libexec/qemu-kvm  -object memory-backend-file,id=mem0,mem-path=/tmp/nv0,size=1G,share,pmem=on


Actual results:
ERROR:qom/object.c:1650:object_get_canonical_path_component: assertion failed: (obj->parent != NULL)
Aborted (core dumped)


Expected results:
No core dump.

Additional info:

Comment 2 Yumei Huang 2018-09-12 08:42:34 UTC
It is introduced by commit "0b7c71b hostmem-file: add the 'pmem' option".

Comment 3 Yumei Huang 2018-09-12 10:07:47 UTC
Hit same error with pmem emulated by kernel line.

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-948.el7.x86_64 root=/dev/mapper/rhel_dhcp--8--214-root ro crashkernel=auto rd.lvm.lv=rhel_dhcp-8-214/root rd.lvm.lv=rhel_dhcp-8-214/swap rhgb quiet nouveau.modeset=0 rd.driver.blacklist=nouveau video=vesa:off LANG=en_US.UTF-8 memmap=4G!2G

# /usr/libexec/qemu-kvm  -object memory-backend-file,id=mem0,mem-path=/dev/pmem0,size=1G,share,pmem=on
**
ERROR:qom/object.c:1650:object_get_canonical_path_component: assertion failed: (obj->parent != NULL)
Aborted (core dumped)

Comment 4 pagupta 2018-09-12 11:33:58 UTC
(In reply to Yumei Huang from comment #3)
> Hit same error with pmem emulated by kernel line.
> 
> # cat /proc/cmdline 
> BOOT_IMAGE=/vmlinuz-3.10.0-948.el7.x86_64
> root=/dev/mapper/rhel_dhcp--8--214-root ro crashkernel=auto
> rd.lvm.lv=rhel_dhcp-8-214/root rd.lvm.lv=rhel_dhcp-8-214/swap rhgb quiet
> nouveau.modeset=0 rd.driver.blacklist=nouveau video=vesa:off
> LANG=en_US.UTF-8 memmap=4G!2G
> 
> # /usr/libexec/qemu-kvm  -object
> memory-backend-file,id=mem0,mem-path=/dev/pmem0,size=1G,share,pmem=on
> **
> ERROR:qom/object.c:1650:object_get_canonical_path_component: assertion
> failed: (obj->parent != NULL)
> Aborted (core dumped)

I think we need to provide a device as well and libvirtd or other user-space management interface should take of this?

-device nvdimm,memdev=mem0,id=nv1

Thanks,
Pankaj

Comment 5 Yumei Huang 2018-09-13 02:26:11 UTC
(In reply to pagupta from comment #4)
> (In reply to Yumei Huang from comment #3)
> > Hit same error with pmem emulated by kernel line.
> > 
> > # cat /proc/cmdline 
> > BOOT_IMAGE=/vmlinuz-3.10.0-948.el7.x86_64
> > root=/dev/mapper/rhel_dhcp--8--214-root ro crashkernel=auto
> > rd.lvm.lv=rhel_dhcp-8-214/root rd.lvm.lv=rhel_dhcp-8-214/swap rhgb quiet
> > nouveau.modeset=0 rd.driver.blacklist=nouveau video=vesa:off
> > LANG=en_US.UTF-8 memmap=4G!2G
> > 
> > # /usr/libexec/qemu-kvm  -object
> > memory-backend-file,id=mem0,mem-path=/dev/pmem0,size=1G,share,pmem=on
> > **
> > ERROR:qom/object.c:1650:object_get_canonical_path_component: assertion
> > failed: (obj->parent != NULL)
> > Aborted (core dumped)
> 
> I think we need to provide a device as well and libvirtd or other user-space
> management interface should take of this?
> 
> -device nvdimm,memdev=mem0,id=nv1

Yeah, that could reproduce too. I just put the shortest cli. 

A complete cli is:

# /usr/libexec/qemu-kvm -M pc,nvdimm \
-object memory-backend-file,id=mem0,mem-path=/tmp/nv0,size=1G,share,pmem=on \
-device nvdimm,id=dimm0,memdev=mem0 


> Thanks,
> Pankaj

Comment 6 Yumei Huang 2018-09-14 05:16:46 UTC
Add regression keyword since it's a newly introduced issue by commit 0b7c71b.

Comment 8 belinda 2018-10-29 06:41:36 UTC
Reproduced on RHEL7.6 RC.

Comment 9 Paul Lai (Intel) 2018-10-30 15:09:34 UTC
(In reply to Yumei Huang from comment #6)
> Add regression keyword since it's a newly introduced issue by commit 0b7c71b.

For clarity... the regression was introduced by....

0b7c71b8 hostmem-file: add the 'pmem' option
   (cherry picked from commit a4de8552b2580adf6fa4874439217b65d3bdd88b)
    Signed-off-by: Paul Lai <plai>

Yi Zhang has submitted upstream the following fix:
  https://patchwork.kernel.org/patch/10654299/

Comment 10 Zhang,Yi 2018-12-17 05:25:49 UTC
Patch already merged into qemu3.1 

commitid: 87dc3ce60a8a16b47aeb6c5f4dbc14ee975563df

Comment 11 Zhang,Yi 2018-12-17 05:35:20 UTC
(In reply to Paul Lai from comment #9)
> (In reply to Yumei Huang from comment #6)
> > Add regression keyword since it's a newly introduced issue by commit 0b7c71b.
> 
> For clarity... the regression was introduced by....
> 
> 0b7c71b8 hostmem-file: add the 'pmem' option
>    (cherry picked from commit a4de8552b2580adf6fa4874439217b65d3bdd88b)
>     Signed-off-by: Paul Lai <plai>
> 
> Yi Zhang has submitted upstream the following fix:
>   https://patchwork.kernel.org/patch/10654299/

Thanks Paul's clarify, since patch already merged into upstream, Paul, can u backbort it to RH7.6 RC to verify this fix?

Regards
Yi

Comment 14 Miroslav Rezanina 2019-02-05 17:08:31 UTC
Fix included in qemu-kvm-rhev-2.12.0-22.el7

Comment 15 Yumei Huang 2019-02-13 03:34:31 UTC
Verify:
qemu-kvm-rhev-2.12.0-23.el7

Instead of core dump, an error message is prompted.

# /usr/libexec/qemu-kvm  -object memory-backend-file,id=mem0,mem-path=/tmp/nv,size=1G,share,pmem=on
qemu-kvm: -object memory-backend-file,id=mem0,mem-path=/tmp/nv,size=1G,share,pmem=on: Lack of libpmem support while setting the 'pmem=on' of memory-backend-file. We can't ensure data persistence.

Comment 17 Paul Lai (Intel) 2019-04-01 14:57:09 UTC
@mrezanin
I don't see a question with the NeedInfo.  Maybe there's a field with it that I don't see.

Comment 18 Paul Lai (Intel) 2019-04-01 15:43:39 UTC
I see this in rhv/master-2.12.0

commit 111769e4a152fdb52f2b10fb957cdfec2fe7d4d2
Author: plai <plai>
Date:   Tue Jan 8 21:33:09 2019 +0100

    hostmem-file: remove object id from pmem error message

Comment 21 errata-xmlrpc 2019-08-22 09:18:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2553