Bug 1433394

Summary: kdump could fill up /var filesystem while writing to /var/crash
Product: Red Hat Enterprise Virtualization Manager Reporter: Andreas Bleischwitz <ableisch>
Component: rhev-hypervisor-ngAssignee: Yuval Turgeman <yturgema>
Status: CLOSED ERRATA QA Contact: Qin Yuan <qiyuan>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.0.7CC: cshao, dfediuck, huzhao, qiyuan, sbonazzo, weiwang, yaniwang, ycui, yzhao
Target Milestone: ovirt-4.2.0   
Target Release: 4.2.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-15 17:57:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Andreas Bleischwitz 2017-03-17 14:42:14 UTC
Description of problem:
Default installation of RHV-H 4.0-20170308.0 image does not separate extra filesystem for kdump. This could lead to a filled-up /var filesystem after crash.

Version-Release number of selected component (if applicable):
RHV-H 4.0-20170308.0 ISO 

How reproducible:
Always following default installation.

Steps to Reproduce:
1. Install host using RHV-H 4.0-20170308.0 ISO
2. find /var created as LVM, but /var/crash lacking
3.

Actual results:
/var/crash is a subdirectory of /var.

Expected results:
/var/crash should be on it's own filesystem to avoid issues with filled /var filesystem.

Additional info:

Comment 1 Ryan Barry 2017-03-20 23:51:15 UTC
I'll see if we can roll this into the NIST changes for 4.1.2, since partitioning will be modified there.

Comment 2 Ryan Barry 2017-04-03 19:35:47 UTC
We can add this without too many problems, though sizing is a definite problem.

We (Red Hat) recommend ${mem_size}+2% with the default kdump config, which could be very large...

With -c, it ends up being somewhat smaller. but this isn't part of the default kdump options.

This causes several problems:

1) Any attempt to add /var/crash will need to add it dynamically based on the memory size

2) The size of this partition may be huge, especially on systems with large amount of memory.

We really need a documentation update for this. But there's a risk of disk size in general. We can add a 10GB partition (which should be safe with -c), but docs will need to be update to reflect the new total space, as well as recommending "-c".

In general, though, it would be best to also recommend a network crash target...

Comment 5 Qin Yuan 2017-11-08 06:38:03 UTC
According to comment #2, and https://gerrit.ovirt.org/#/c/79500/, /var/crash should be added as a separated partition with 10GiB. 

The verification versions, steps, and results are as following:

Versions:
RHVH-4.2-20171105.2-RHVH-x86_64-dvd1.iso
imgbased-1.0.1-0.1.el7ev.noarch

Steps:
1. Install RHVH-4.2-20171105.2-RHVH-x86_64-dvd1.iso
2. Choose auto partitioning
3. Finish installation, reboot and log into RHVH system
4. Check `lvs -a |grep crash`, `mount |grep crash`

Results:
[root@dell-per515-01 ~]# lvs -a |grep crash
  var_crash               rhvh Vwi-aotz--  10.00g pool00                       2.86 

[root@dell-per515-01 ~]# mount |grep crash
/dev/mapper/rhvh-var_crash on /var/crash type ext4 (rw,relatime,seclabel,discard,stripe=16,data=ordered)

As you can see, var_crash was created as a thin volume, the size was 10.00g, and the file system was mounted on /var/crash, the discard option was present.

The results are consistent with the designing, so change the bug status to VERIFIED.

Comment 9 errata-xmlrpc 2018-05-15 17:57:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1524

Comment 10 Franta Kust 2019-05-16 13:03:17 UTC
BZ<2>Jira Resync