Bug 1427466

Summary: [RHEV7.4] dump-guest-memory failed due to Python Exception <class 'gdb.error'> Attempt to extract a component of a value that is not a (null).
Product: Red Hat Enterprise Linux 7 Reporter: hachen <hachen>
Component: qemu-kvm-rhevAssignee: Marc-Andre Lureau <marcandre.lureau>
Status: CLOSED ERRATA QA Contact: hachen <hachen>
Severity: unspecified Docs Contact:
Priority: high    
Version: 7.4CC: aliang, chayang, choma, coli, drjones, hachen, juzhang, knoel, marcandre.lureau, michen, mrezanin, ngu, pingl, shuang, virt-maint, xfu, xutian, xuwei
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.9.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 23:44:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description hachen 2017-02-28 09:44:16 UTC
Description of problem:
 
Run dump-guest-memory failed when I am tryting to dump the core from qemu.

Version-Release number of selected component (if applicable):
HOST:
kernel-3.10.0-561.el7.x86_64
kernel-debuginfo-3.10.0-514.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-514.el7.x86_64

qemu-kvm-rhev-2.8.0-5.el7.x86_64
qemu-kvm-rhev-debuginfo-2.8.0-5.el7.x86_64

GUEST:
kernel-3.10.0-514.el7.x86_64


How reproducible: 6/6


Steps to Reproduce:
1. Set 'ulimit -c unlimited' and 'echo "/var/core.%p-%u-%g-%s-%t-%h-%e" >
/proc/sys/kernel/core_pattern' in the host terminal

2.1.Boot up a guest with dump-guest-core=on.
e.g:
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine pc,dump-guest-core=on \
    -nodefaults  \
    -vga cirrus  \
    -device nec-usb-xhci,id=usbtest,bus=pci.0,addr=03 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel73-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04 \
    -device virtio-net-pci,mac=9a:4d:4e:4f:50:51,id=id3DveCw,vectors=4,netdev=idgW5YRp,bus=pci.0,addr=05  \
    -netdev tap,id=idgW5YRp \
    -m 5457  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'SandyBridge',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usbtest.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \

3.Kill the qemu-kvm process with SIGSEGV, as root:
# pkill -SEGV -x qemu-kvm
(qemu) Segmentation fault (core dumped)

4.Open the "coredump" file with gdb, and load the extension.
gdb /var/core.21378-0-0-11-1488273759-intel-e31225-16-3.englab.nay.redhat.com-qemu-kvm
(gdb) source /usr/share/qemu-kvm/dump-guest-memory.py

5.extract the guest vmcore with the new extension
(gdb) set height 0
(gdb) dump-guest-memory /tmp/vmcore X86_64


Actual results:

guest RAM blocks:
target_start     target_end       host_addr        message count
---------------- ---------------- ---------------- ------- -----
Python Exception <class 'gdb.error'> Attempt to extract a component of a value that is not a (null).: 
Error occurred in Python command: Attempt to extract a component of a value that is not a (null).

Expected results:

guest RAM blocks:
target_start     target_end       host_addr        message count
---------------- ---------------- ---------------- ------- -----
0000000000000000 00000000000a0000 00007f4d3b600000 added       1
00000000000a0000 00000000000b0000 00007f4d3a000000 added       2
00000000000c0000 00000000000ca000 00007f4d3b6c0000 added       3
00000000000ca000 00000000000cd000 00007f4d3b6ca000 joined      3
00000000000cd000 00000000000ec000 00007f4d3b6cd000 joined      3
00000000000ec000 00000000000f0000 00007f4d3b6ec000 joined      3
00000000000f0000 0000000000100000 00007f4d3b6f0000 joined      3
0000000000100000 00000000c0000000 00007f4d3b700000 joined      3
00000000fc000000 00000000fd000000 00007f4d3a000000 added       4
00000000fffc0000 0000000100000000 00007f4d3b400000 added       5
0000000100000000 0000000195100000 00007f4dfb600000 added       6
dumping range at 00007f4d3b600000 for length 00000000000a0000
dumping range at 00007f4d3a000000 for length 0000000000010000
dumping range at 00007f4d3b6c0000 for length 00000000bff40000
dumping range at 00007f4d3a000000 for length 0000000001000000
dumping range at 00007f4d3b400000 for length 0000000000040000
dumping range at 00007f4dfb600000 for length 0000000095100000


Additional info:
Test on HOST:
kernel-3.10.0-561.el7.x86_64
kernel-debuginfo-3.10.0-514.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-514.el7.x86_64

qemu-kvm-rhev-debuginfo-2.6.0-27.el7.x86_64
qemu-kvm-rhev-2.6.0-27.el7.x86_64

GUEST:
kernel-3.10.0-514.el7.x86_64

It does not have this issue.

Comment 4 Marc-Andre Lureau 2017-03-09 10:59:20 UTC
reproduced, looking at it

Comment 5 Marc-Andre Lureau 2017-03-10 08:54:13 UTC
patch on qemu ML
"[PATCH] scripts/dump-guest-memory.py: fix int128_get64 on recent gcc"

Comment 6 hachen 2017-04-25 05:43:52 UTC
Test on:
qemu-kvm-rhev-2.9.0-1.el7.x86_64

Host:
kernel-debuginfo-3.10.0-656.el7.x86_64
kernel-3.10.0-656.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-656.el7.x86_64

Guest:
kernel-debuginfo-3.10.0-656.el7.x86_64
kernel-3.10.0-656.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-656.el7.x86_64


Got expected result, bug verified
(gdb) dump-guest-memory /tmp/vmcore X86_64
guest RAM blocks:
target_start     target_end       host_addr        message count
---------------- ---------------- ---------------- ------- -----
0000000000000000 00000000000a0000 00007fecafe00000 added       1
00000000000a0000 00000000000b0000 00007fecae800000 added       2
00000000000c0000 00000000000ca000 00007fecafec0000 added       3
00000000000ca000 00000000000cd000 00007fecafeca000 joined      3
00000000000cd000 00000000000ec000 00007fecafecd000 joined      3
00000000000ec000 00000000000f0000 00007fecafeec000 joined      3
00000000000f0000 0000000000100000 00007fecafef0000 joined      3
0000000000100000 00000000c0000000 00007fecaff00000 joined      3
00000000fc000000 00000000fd000000 00007fecae800000 added       4
00000000fffc0000 0000000100000000 00007fecafc00000 added       5
0000000100000000 0000000240000000 00007fed6fe00000 added       6
dumping range at 00007fecafe00000 for length 00000000000a0000
dumping range at 00007fecae800000 for length 0000000000010000
dumping range at 00007fecafec0000 for length 00000000bff40000

Comment 8 errata-xmlrpc 2017-08-01 23:44:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 9 errata-xmlrpc 2017-08-02 01:22:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 10 errata-xmlrpc 2017-08-02 02:14:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 11 errata-xmlrpc 2017-08-02 02:55:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 12 errata-xmlrpc 2017-08-02 03:19:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 13 errata-xmlrpc 2017-08-02 03:37:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 14 Michael 2019-07-17 05:21:33 UTC
Hi Lureau:

Can you help me have a look this? Maybe there is some thing wrong in my step. 

After core.xxx file was created, I was trying to load the extension.

(gdb) source /usr/share/qemu-kvm/dump-guest-memory.py
Traceback (most recent call last):
  File "/usr/share/qemu-kvm/dump-guest-memory.py", line 21, in <module>
    UINTPTR_T = gdb.lookup_type("uintptr_t")
gdb.error: No type named uintptr_t.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/qemu-kvm/dump-guest-memory.py", line 23, in <module>
    raise gdb.GdbError("Symbols must be loaded prior to sourcing dump-guest-memory.\n"
gdb.GdbError: Symbols must be loaded prior to sourcing dump-guest-memory.
Symbols may be loaded by 'attach'ing a QEMU process id or by 'load'ing a QEMU binary.
(gdb) 


Can you have a loop this issue. Thank you for your help.

Comment 15 Andrew Jones 2019-07-17 07:53:57 UTC
(In reply to Michael from comment #14)
> gdb.GdbError: Symbols must be loaded prior to sourcing dump-guest-memory.
> Symbols may be loaded by 'attach'ing a QEMU process id or by 'load'ing a
> QEMU binary.

Just do what the exception error text is telling you to do. For example, this should work:

$ gdb path/to/qemu
(gdb) source path/to/dump-guest-memory.py

Or, as the error text states, attach to a running QEMU's PID first.

Comment 16 Michael 2019-07-17 11:49:55 UTC
Hi (In reply to Andrew Jones from comment #15)
> 
> Just do what the exception error text is telling you to do. For example,
> this should work:
> 
> $ gdb path/to/qemu
> (gdb) source path/to/dump-guest-memory.py
> 
> Or, as the error text states, attach to a running QEMU's PID first.

Hi Andrew:

I can execute the command on the RHEL7. Fail on RHEL8. 

I open a new Bug to track it. Please see the detail in the new Bug 1730691. 

https://bugzilla.redhat.com/show_bug.cgi?id=1730691




Thanks.