Bug 1220993 - No VM's core dumps after kill vm EL7
Summary: No VM's core dumps after kill vm EL7
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.5.1
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ovirt-3.6.0-rc
: 3.6.0
Assignee: Yaniv Bronhaim
QA Contact: Jiri Belka
URL:
Whiteboard:
Depends On:
Blocks: 1215554 1229312
TreeView+ depends on / blocked
 
Reported: 2015-05-13 04:18 UTC by Douglas Schilling Landgraf
Modified: 2016-03-09 19:39 UTC (History)
13 users (show)

Fixed In Version: ovirt-3.6.0-2.1
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1229312 (view as bug list)
Environment:
Last Closed: 2016-03-09 19:39:51 UTC
oVirt Team: Infra
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1215554 medium CLOSED [7.1_3.5.1] No VM's core dumps after kill vm in rhevh7.1 2020-10-14 00:28:05 UTC
Red Hat Product Errata RHBA-2016:0362 normal SHIPPED_LIVE vdsm 3.6.0 bug fix and enhancement update 2016-03-09 23:49:32 UTC
oVirt gerrit 41072 master MERGED Configure libvirt to generate VM's core dumps 2020-04-24 08:58:12 UTC
oVirt gerrit 41474 ovirt-3.5 MERGED Configure libvirt to generate VM's core dumps 2020-04-24 08:58:12 UTC

Internal Links: 1215554

Description Douglas Schilling Landgraf 2015-05-13 04:18:43 UTC
Description of problem:

No VM's core dumps after kill vm.

Red Hat Enterprise Virtualization Hypervisor 7.1 (20150420.0.el7ev)
ovirt-node-3.2.2-3.el7.noarch
vdsm-4.16.13.1-1.el7ev.x86_64
ovirt-node-selinux-3.2.2-3.el7.noarch
selinux-policy-3.13.1-23.el7.noarch
selinux-policy-targeted-3.13.1-23.el7.noarch

# rpm -qf /var/log/core/
vdsm-4.16.13.1-1.el7ev.x86_64

Red Hat Enterprise Virtualization Manager Version: 3.5.1-0.4.el6ev

Steps to Reproduce:
1. Install RHEVH-7.1-20150420.0
2. Register to RHEVM3.5.1.
3. Create VM.
4. Run command "pkill -ABRT -f 'qemu-kvm'" on host.
5. Run  ll /var/log/core/ on host.

Additional info:

There is a vdsm patch that starts the coredump configuration [1] with systemd, however it's required one additional step, which is add into libvirtd.service LimitCORE=infinity as the libvirt will trigger qemu-kvm.

i.e: /usr/lib/systemd/system/libvirtd.service
# cat ./usr/lib/systemd/system/libvirtd.service
[Unit]
Description=Virtualization daemon
Before=libvirt-guests.service
After=network.target
After=dbus.service
After=iscsid.service
After=apparmor.service
Documentation=man:libvirtd(8)
Documentation=http://libvirt.org

[Service]
Type=notify
LimitCORE=infinity <-------------------  Added
EnvironmentFile=-/etc/sysconfig/libvirtd
ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
# Override the maximum number of opened files
#LimitNOFILE=2048

[Install]
WantedBy=multi-user.target


[1]supervdsm: enable coredumping
https://gerrit.ovirt.org/#/c/35745/

Comment 2 Yaniv Bronhaim 2015-05-17 11:56:13 UTC
I don't think vdsm shouldn't touch libvirtd.service. I prefer to hear more views - ccing Danken to hear his opinion and Jiri to understand why the default is with coredump limitation and if its more reasonable to change that default.

Comment 3 Yaniv Bronhaim 2015-05-17 15:47:19 UTC
s/shouldn't/should ^

Comment 4 Jiri Denemark 2015-05-18 08:42:47 UTC
Normal systems (RHEL, for example) run abrt daemon which will handle the crash and process or store the core dump depending on its configuration regardless on LimitCORE settings. On hosts without abrt daemon, setting LimitCore to unlimited by default is not a very good idea since core files may be really big, especially those of qemu-kvm processes running big guests. And I believe even vdsm should not enable them by default. You should rather start using abrt which should be even able to process the core file without storing it on a disk first (although this functionality is not available in RHEL 7.1, AFAIK).

However, if you still want to set LimitCORE=unlimited (either by default or on demand), you should not mess with /usr/lib/systemd/system/libvirtd.service. You should do this by creating a new file in /etc/systemd/system/libvirtd.service.d/. For example,

$ cat /etc/systemd/system/libvirtd.service.d/unlimited-core.conf
[Service]
LimitCORE=unlimited

Comment 5 Dan Kenigsberg 2015-05-18 11:55:22 UTC
Yaniv, as Jiri hints, the best solution is to implement bug 917062. But until this happens, we should solve our el7 regression, and add the suggested libvirt.conf to our libvirt configurator.

Comment 6 Yaniv Bronhaim 2015-05-20 09:20:48 UTC
Thanks for the replies! Moving to 3.6 and we'll consider to add it soon. I prefer not to introduce this change in 3.5

Comment 7 Jiri Belka 2015-06-05 08:35:05 UTC
FYI I reported this as well and got closed by mskrivan@, he told me it has never been intended to have core files. See https://bugzilla.redhat.com/show_bug.cgi?id=1220860

Comment 9 Jiri Belka 2015-06-25 08:51:29 UTC
ok, vdsm-4.17.0-1028.git7f5e1f2.el7.noarch

# rpm -qf /etc/systemd/system/libvirtd.service.d/unlimited-core.conf 
vdsm-4.17.0-1028.git7f5e1f2.el7.noarch

# cat /etc/systemd/system/libvirtd.service.d/unlimited-core.conf 
## beginning of configuration section by vdsm
[Service]
LimitCORE=infinity
## end of configuration section by vdsm

# file /var/log/core/core.519.1435222212.dump 
/var/log/core/core.519.1435222212.dump: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from '/usr/libexec/qemu-kvm -name test -S -machine rhel6.5.0,accel=kvm,usb=off -cpu S'

Comment 11 errata-xmlrpc 2016-03-09 19:39:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0362.html


Note You need to log in before you can comment on or make changes to this bug.