Bug 1533410

Summary: Save full qemu core dump to provide maximum information about the crash (since RHEL 7.4 core dump doesn't contain guest memory dump)
Product: [oVirt] vdsm Reporter: Yaniv Bronhaim <ybronhei>
Component: CoreAssignee: Yaniv Bronhaim <ybronhei>
Status: CLOSED CURRENTRELEASE QA Contact: Pavol Brilla <pbrilla>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.20.4CC: bugs, lsvaty, lveyde, mperina
Target Milestone: ovirt-4.2.2Keywords: ZStream
Target Release: ---Flags: rule-engine: ovirt-4.2+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: vdsm v4.20.18 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1535431 (view as bug list) Environment:
Last Closed: 2018-03-29 10:54:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1535431    

Description Yaniv Bronhaim 2018-01-11 09:53:13 UTC
Description of problem:
without setting MakeCompatCore=yes (although CreateCoreBacktrace=yes is set) in /etc/abrt/plugins/CCpp.conf the abrt crash report includes the following files:

abrt_version
cgroup
count
event_log
hostname
limits
open_fds
package
pkg_epoch
pkg_release
proc_pid_status
runlevel
uid
var_log_messages
analyzer
cmdline
dso_list
executable
kernel
machineid
os_info
pid
pkg_fingerprint
pkg_vendor
pwd
time
username
architecture
component
environ 
global_pid
last_occurrence
maps
os_release
pkg_arch
pkg_name 
pkg_version
reason
type 
uuid

with compact core config the output includes also the core_backtrace file with the following output:

# cat core_backtrace 
{   "signal": 11
,   "executable": "/usr/libexec/qemu-kvm"
,   "stacktrace":
      [ {   "crash_thread": true
        ,   "frames":
              [ {   "address": 140000363764479
                ,   "build_id": "c3f28802314af4ee866bf8d2e1b506b7bbf34cf6"
                ,   "build_id_offset": 973567
                ,   "function_name": "ppoll"
                ,   "file_name": "/usr/lib64/libc-2.17.so"
                }
              , {   "address": 93963107712921
                ,   "build_id": "250731da86353c7dda890a58874c5eb9137f95fc"
                ,   "build_id_offset": 6033305
                ,   "function_name": "qemu_poll_ns"
                ,   "file_name": "/usr/libexec/qemu-kvm"
                }
              , {   "address": 93963107716520
                ,   "build_id": "250731da86353c7dda890a58874c5eb9137f95fc"
                ,   "build_id_offset": 6036904
                ,   "function_name": "main_loop_wait"
                ,   "file_name": "/usr/libexec/qemu-kvm"
                }
              , {   "address": 93963104510348
                ,   "build_id": "250731da86353c7dda890a58874c5eb9137f95fc"
                ,   "build_id_offset": 2830732
                ,   "function_name": "main"
                ,   "file_name": "/usr/libexec/qemu-kvm"
                } ]
        } ]
}


Version-Release number of selected component (if applicable):
vdsm-4-20-12

How reproducible:
100%

Steps to Reproduce:
1.kill -SIGSEGV [vm-pid]
2.check /var/tmp/abrt/ output

Comment 1 Martin Perina 2018-01-17 10:32:24 UTC
After a discussion with qemu team even properly generated backtrace doesn't contain enough information about the crash. Also since RHEL 7.4 the generated crash dump doesn't contain guest memory dump, so the size should keep around 500MB. That's why we are going to change abrt configuration again to save complete core dump.

Comment 3 Pavol Brilla 2018-03-07 09:38:54 UTC
# ls -l */*core*
-rw-r-----. 1 root abrt 387616768 Mar  7 10:36 ccpp-2018-03-07-10:36:04-5041/coredump

# ls -l ccpp-2018-03-07-10\:36\:04-5041/
total 52592
-rw-r-----. 1 root abrt         6 Mar  7 10:36 abrt_version
-rw-r-----. 1 root abrt         4 Mar  7 10:36 analyzer
-rw-r-----. 1 root abrt         6 Mar  7 10:36 architecture
-rw-r-----. 1 root abrt       769 Mar  7 10:36 cgroup
-rw-r-----. 1 root abrt      3495 Mar  7 10:36 cmdline
-rw-r-----. 1 root abrt        13 Mar  7 10:36 component
-rw-r-----. 1 root abrt      7597 Mar  7 10:37 core_backtrace
-rw-r-----. 1 root abrt 387616768 Mar  7 10:36 coredump
-rw-r-----. 1 root abrt         1 Mar  7 10:37 count
-rw-r-----. 1 root abrt      8274 Mar  7 10:37 dso_list
-rw-r-----. 1 root abrt        85 Mar  7 10:36 environ
-rw-r-----. 1 root abrt         0 Mar  7 10:37 event_log
-rw-r-----. 1 root abrt        21 Mar  7 10:36 executable
-rw-r-----. 1 root abrt         4 Mar  7 10:36 global_pid
-rw-r-----. 1 root abrt        35 Mar  7 10:36 hostname
-rw-r-----. 1 root abrt        26 Mar  7 10:36 kernel
-rw-r-----. 1 root abrt        10 Mar  7 10:36 last_occurrence
-rw-r-----. 1 root abrt      1323 Mar  7 10:36 limits
-rw-r-----. 1 root abrt       135 Mar  7 10:37 machineid
-rw-r-----. 1 root abrt     61983 Mar  7 10:36 maps
-rw-r-----. 1 root abrt      6063 Mar  7 10:36 open_fds
-rw-r-----. 1 root abrt       532 Mar  7 10:36 os_info
-rw-r-----. 1 root abrt        51 Mar  7 10:36 os_release
-rw-r-----. 1 root abrt        27 Mar  7 10:36 package
-rw-r-----. 1 root abrt         4 Mar  7 10:36 pid
-rw-r-----. 1 root abrt         6 Mar  7 10:36 pkg_arch
-rw-r-----. 1 root abrt         2 Mar  7 10:36 pkg_epoch
-rw-r-----. 1 root abrt        13 Mar  7 10:36 pkg_name
-rw-r-----. 1 root abrt         6 Mar  7 10:36 pkg_release
-rw-r-----. 1 root abrt        13 Mar  7 10:36 pkg_vendor
-rw-r-----. 1 root abrt         6 Mar  7 10:36 pkg_version
-rw-r-----. 1 root abrt      1194 Mar  7 10:36 proc_pid_status
-rw-r-----. 1 root abrt         1 Mar  7 10:36 pwd
-rw-r-----. 1 root abrt        26 Mar  7 10:36 reason
-rw-r-----. 1 root abrt         4 Mar  7 10:36 runlevel
-rw-r-----. 1 root abrt  20613548 Mar  7 10:37 sosreport.tar.xz
-rw-r-----. 1 root abrt        10 Mar  7 10:36 time
-rw-r-----. 1 root abrt         4 Mar  7 10:36 type
-rw-r-----. 1 root abrt         3 Mar  7 10:36 uid
-rw-r-----. 1 root abrt         5 Mar  7 10:36 username
-rw-r-----. 1 root abrt        40 Mar  7 10:37 uuid
-rw-r-----. 1 root abrt      2400 Mar  7 10:37 var_log_messages

# yum list vdsm
Loaded plugins: package_upload, product-id, search-disabled-repos
Installed Packages
vdsm.x86_64                 4.20.19-1.el7ev            @rhv-4.2.2

Comment 4 Sandro Bonazzola 2018-03-29 10:54:30 UTC
This bugzilla is included in oVirt 4.2.2 release, published on March 28th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.