Bug 1433394 - kdump could fill up /var filesystem while writing to /var/crash
Summary: kdump could fill up /var filesystem while writing to /var/crash
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhev-hypervisor-ng
Version: 4.0.7
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ovirt-4.2.0
: 4.2.0
Assignee: Yuval Turgeman
QA Contact: Qin Yuan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-17 14:42 UTC by Andreas Bleischwitz
Modified: 2019-05-16 13:03 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2018-05-15 17:57:40 UTC
oVirt Team: Node
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:1524 0 None None None 2018-05-15 17:58:49 UTC
oVirt gerrit 79500 0 master MERGED Added /var/crash to the volumes list 2021-01-04 08:50:41 UTC

Description Andreas Bleischwitz 2017-03-17 14:42:14 UTC
Description of problem:
Default installation of RHV-H 4.0-20170308.0 image does not separate extra filesystem for kdump. This could lead to a filled-up /var filesystem after crash.

Version-Release number of selected component (if applicable):
RHV-H 4.0-20170308.0 ISO 

How reproducible:
Always following default installation.

Steps to Reproduce:
1. Install host using RHV-H 4.0-20170308.0 ISO
2. find /var created as LVM, but /var/crash lacking
3.

Actual results:
/var/crash is a subdirectory of /var.

Expected results:
/var/crash should be on it's own filesystem to avoid issues with filled /var filesystem.

Additional info:

Comment 1 Ryan Barry 2017-03-20 23:51:15 UTC
I'll see if we can roll this into the NIST changes for 4.1.2, since partitioning will be modified there.

Comment 2 Ryan Barry 2017-04-03 19:35:47 UTC
We can add this without too many problems, though sizing is a definite problem.

We (Red Hat) recommend ${mem_size}+2% with the default kdump config, which could be very large...

With -c, it ends up being somewhat smaller. but this isn't part of the default kdump options.

This causes several problems:

1) Any attempt to add /var/crash will need to add it dynamically based on the memory size

2) The size of this partition may be huge, especially on systems with large amount of memory.

We really need a documentation update for this. But there's a risk of disk size in general. We can add a 10GB partition (which should be safe with -c), but docs will need to be update to reflect the new total space, as well as recommending "-c".

In general, though, it would be best to also recommend a network crash target...

Comment 5 Qin Yuan 2017-11-08 06:38:03 UTC
According to comment #2, and https://gerrit.ovirt.org/#/c/79500/, /var/crash should be added as a separated partition with 10GiB. 

The verification versions, steps, and results are as following:

Versions:
RHVH-4.2-20171105.2-RHVH-x86_64-dvd1.iso
imgbased-1.0.1-0.1.el7ev.noarch

Steps:
1. Install RHVH-4.2-20171105.2-RHVH-x86_64-dvd1.iso
2. Choose auto partitioning
3. Finish installation, reboot and log into RHVH system
4. Check `lvs -a |grep crash`, `mount |grep crash`

Results:
[root@dell-per515-01 ~]# lvs -a |grep crash
  var_crash               rhvh Vwi-aotz--  10.00g pool00                       2.86 

[root@dell-per515-01 ~]# mount |grep crash
/dev/mapper/rhvh-var_crash on /var/crash type ext4 (rw,relatime,seclabel,discard,stripe=16,data=ordered)

As you can see, var_crash was created as a thin volume, the size was 10.00g, and the file system was mounted on /var/crash, the discard option was present.

The results are consistent with the designing, so change the bug status to VERIFIED.

Comment 9 errata-xmlrpc 2018-05-15 17:57:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1524

Comment 10 Franta Kust 2019-05-16 13:03:17 UTC
BZ<2>Jira Resync


Note You need to log in before you can comment on or make changes to this bug.