Bug 1883753 - Makedumpfile gets estimated vmcore size with specified options
Summary: Makedumpfile gets estimated vmcore size with specified options
Keywords:
Status: ASSIGNED
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: kexec-tools
Version: 8.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: ltao
QA Contact: xiaoying yan
Sujata Kurup
URL:
Whiteboard:
Depends On:
Blocks: 1958452 1674330 1868698
TreeView+ depends on / blocked
 
Reported: 2020-09-30 07:05 UTC by lijiang
Modified: 2021-07-22 09:20 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1958452 (view as bug list)
Environment:
Last Closed:
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)

Description lijiang 2020-09-30 07:05:21 UTC
Description of problem:

Currently, makedumpfile only gets the following descriptions:

# makedumpfile /proc/kcore  --mem-usage
The kernel version is not supported.
The makedumpfile operation may be incomplete.

TYPE		PAGES			EXCLUDABLE	DESCRIPTION
----------------------------------------------------------------------
ZERO		48385           	yes		Pages filled with zero
NON_PRI_CACHE	204744          	yes		Cache pages without private flag
PRI_CACHE	153             	yes		Cache pages with private flag
USER		23905           	yes		User process pages
FREE		7689759         	yes		Free pages
KERN_DATA	291027          	no		Dumpable kernel data 

page size:		4096            
Total pages on system:	8257973         
Total size on system:	33824657408      Byte

It doesn't provide an estimated vmcore size, which is not helpful to estimate if there is enough free space to save the vmcore in a target disk(bz1868698).

In view of this, we would like to get an estimated vmcore size according to the specified options as below:

#makedumpfile -l --message-level 1 -d 31 --mem-usage
...
...
The estimated vmcore size: xxxM(or xxxG)

#makedumpfile -l --message-level 7 -d 15 --mem-usage
...
...
The estimated vmcore size: xxxM(or xxxG)


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 lijiang 2020-09-30 09:13:28 UTC
As we discussed in the meeting, the '--mem-usage' option has a conflict with other options and it costs time as below:

# time makedumpfile /proc/kcore  --mem-usage
The kernel version is not supported.
The makedumpfile operation may be incomplete.
TYPE		PAGES			EXCLUDABLE	DESCRIPTION
----------------------------------------------------------------------
ZERO		48065           	yes		Pages filled with zero
NON_PRI_CACHE	204748          	yes		Cache pages without private flag
PRI_CACHE	153             	yes		Cache pages with private flag
USER		23906           	yes		User process pages
FREE		7689812         	yes		Free pages
KERN_DATA	291289          	no		Dumpable kernel data 

page size:		4096            
Total pages on system:	8257973         
Total size on system:	33824657408      Byte

real	0m9.535s
user	0m1.450s
sys	0m8.074s


Therefore, it could be good to add a new option '--vmcore-size', which will get a value of estimated vmcore size and do not need to actually save the vmcore(only compute the size of vmcore):

#makedumpfile -l --message-level 1 -d 31 /proc/kcore vmcore --vmcore-size

# time makedumpfile -l --message-level 1 -d 31 /proc/kcore vmcore --vmcore-size
Copying data                                      : [ 67.1 %] -           eta: 1s

real	0m2.801s
user	0m1.851s
sys	0m0.943s

The performance could be improved because the vmcore is not actually saved.

Comment 3 Dave Young 2020-10-01 03:40:02 UTC
> Therefore, it could be good to add a new option '--vmcore-size', which will
> get a value of estimated vmcore size and do not need to actually save the
> vmcore(only compute the size of vmcore):
> 

The --mem-usage can not estimate the compressed size, so the idea is to add a new option to do go through the saving process but do not save data only return the final saved size, something like save to /dev/null etc.

It is still not enough to estimate a vmcore size since vmcore size depends on the system memory use and workload,  people may need to periodicity monitor the vmcore size and the remain space of dump target.  Anyway it is still a best effort but it is better than none like before.

Comment 7 ltao 2021-04-22 05:56:34 UTC
The patch [1][2][3] implemented option --dry-run and --show-stats:

# makedumpfile -l --message-level 1 -d 31 /proc/kcore vmcore --dry-run --show-stats
Checking for memory holes                         : [100.0 %] |                  STEP [Checking for memory holes  ] : 0.000071 seconds
Excluding unnecessary pages                       : [100.0 %] \                  STEP [Excluding unnecessary pages] : 0.268348 seconds
Checking for memory holes                         : [100.0 %] -                  STEP [Checking for memory holes  ] : 0.000372 seconds
Checking for memory holes                         : [100.0 %] /                  STEP [Checking for memory holes  ] : 0.000032 seconds
Excluding unnecessary pages                       : [100.0 %] |                  STEP [Excluding unnecessary pages] : 0.264383 seconds
Copying data                                      : [ 84.2 %] |           eta: 0sSTEP [Copying data               ] : 3.926611 seconds
STEP [Copying data               ] : 0.000000 seconds


Original pages  : 0x00000000007f32ba
  Excluded pages   : 0x00000000007520c7
    Pages filled with zero  : 0x0000000000011c49
    Non-private cache pages : 0x00000000000b9cd5
    Private cache pages     : 0x0000000000086f4e
    User process data pages : 0x000000000011cd44
    Free pages              : 0x00000000004e2b17
    Hwpoison pages          : 0x0000000000000000
    Offline pages           : 0x0000000000000000
  Remaining pages  : 0x00000000000a11f3
  (The number of pages is reduced to 7%.)
Memory Hole     : 0x000000000005cd46
--------------------------------------------------
Total pages     : 0x0000000000850000
Write bytes     : 1273508345

Cache hit: 738293, miss: 967362, hit rate: 43.3%

--dry-run asks makedumpfile don't write data into disk, --show-stats will print report messages.
the "Write bytes" provide an value of the vmcore size. The value may have slight differences
between each run. So it can estimate the vmcore size when dry-run.

[1] https://www.mail-archive.com/kexec@lists.infradead.org/msg25485.html
[2] https://www.mail-archive.com/kexec@lists.infradead.org/msg25484.html
[3] https://github.com/makedumpfile/makedumpfile/commit/0ef2ca6c9fa2f61f217a4bf5d7fd70f24e12b2eb


Note You need to log in before you can comment on or make changes to this bug.