Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1447512

Summary: Optimize kdump initramfs build time
Product: Red Hat Enterprise Linux 7 Reporter: Xunlei Pang <xlpang>
Component: kexec-toolsAssignee: Kairui Song <kasong>
Status: CLOSED DUPLICATE QA Contact: Ziqian SUN (Zamir) <zsun>
Severity: low Docs Contact:
Priority: unspecified    
Version: 7.4CC: bhsharma, dyoung, kdump-bugs, ruyang, xiawu, xlpang
Target Milestone: rc   
Target Release: 7.7   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1645942 (view as bug list) Environment:
Last Closed: 2019-03-18 08:58:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1484701    
Bug Blocks: 1473055, 1549423, 1645942, 1647768    

Description Xunlei Pang 2017-05-03 03:37:17 UTC
Description of problem:

Here is an example, kdump takes about 13 seconds to finish the initramfs build:
Rebuilding /boot/initramfs-4.8.6-300.fc25.x86_64kdump.img
kexec: loaded kdump kernel
Starting kdump: [OK]

real	0m12.462s
user	0m16.381s
sys	0m3.733s

On some slow kvm machines, it takes even longer, 40 seconds according to the test result on Amazone EC2 C5 Cloud.

Kdump initramfs building relies on dracut, do some investigation to see if we can speed up the initramfs building process a little.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
It is targeted on RHEL7.5

Comment 3 Ziyue Yang 2017-05-03 06:03:08 UTC
As I mentioned in

https://bugzilla.redhat.com/show_bug.cgi?id=1446698

Disable list_modules and list_files in lsinitrd will significantly reduce dracut build time (in my case ~30% reduce in dracut and ~15% reduce in kdumpctl start with/without rebuild). There are mainly 4 sources of overhead caused by lsinitrd:



1. In dracut script (with rebuild)

if (( maxloglvl >= 5 )); then
    if [[ $allowlocal ]]; then
	"$dracutbasedir/lsinitrd.sh" "${DRACUT_TMPDIR}/initramfs.img"| ddebug
    else
        lsinitrd "${DRACUT_TMPDIR}/initramfs.img"| ddebug
    fi
fi

These verbose statements will be triggered by default, by kdumpctl calling dracut. Removing these leads to a reduce from 18s to 13s in my QEMU machine.



2. In kdumpctl script,  (with/without rebuild)

_dracut_args=$(lsinitrd $TARGET_INITRD | grep "^Arguments:" | head -1)
if [[ -z "$_dracut_args" ]];then
	echo "Warning: No dracut arguments found in initrd"
	return 0
fi

This lsinitrd call needs only Arguments lines but all the information are dumped by default.



3. In kdumpctl script, check_wdt_modified() (with/without rebuild)

wdtdrv=$(< "$dir/device/modalias")
wdtdrv=$(modinfo $wdtdrv | grep filename | awk -F"kernel/" '{print $2}')
active=$(< "$dir/state")
# rebuild when:
# module for this watchdog is not found and watchdog is active
# module for this watchdog is found and watchdog is inactive
lsinitrd $TARGET_INITRD | grep $wdtdrv &> /dev/null

This call of lsinitrd needs only module info.
BTW, the wdtdrv variable seems not getting a valid module's name but its alias instead, and at least on my PC they are not the same thing.



4. In kdumpctl script, check_wdt_modified() (with/without rebuild)

# check if watchdog kernel module unloaded.
loaded_mods=$(lsinitrd $TARGET_INITRD -f etc/cmdline.d/00-watchdog.conf)

This call of lsinitrd seems to be dead code because the 00-watchdog.conf file semms not an always valid path.

Comment 5 Ziyue Yang 2017-05-03 07:31:01 UTC
(In reply to Ziyue Yang from comment #3)

> 1. In dracut script (with rebuild)
> 
> if (( maxloglvl >= 5 )); then
>     if [[ $allowlocal ]]; then
> 	"$dracutbasedir/lsinitrd.sh" "${DRACUT_TMPDIR}/initramfs.img"| ddebug
>     else
>         lsinitrd "${DRACUT_TMPDIR}/initramfs.img"| ddebug
>     fi
> fi
> 
> These verbose statements will be triggered by default, by kdumpctl calling
> dracut. Removing these leads to a reduce from 18s to 13s in my QEMU machine.
> 

Confirmed that maxloglvl will be 5, set by sysloglvl, in root mode, causing lsinitrd overhead.

Comment 6 Xunlei Pang 2017-06-30 04:20:12 UTC
*** Bug 1459622 has been marked as a duplicate of this bug. ***

Comment 8 Xunlei Pang 2017-08-24 02:38:21 UTC
Hi Ziyue, you should create a dedicated bug for RHEL7 dracut, rather than depending on a Fedora bug.

Comment 14 Kairui Song 2019-02-26 09:38:33 UTC
After the backport of bz1484701, an estimation of time consume of each component in my VM is as below:
It's a rough classification of each stage but could be a good reference.

Kdumpclt Check feasibility:	14ms
Kdumpclt Check Config:		15ms
Kdumpclt Check Status:		17ms
Mkdumprd Prepare:		254ms
Mkdumprd parse config:		415ms
Dracut Parse Args:		361ms
Dracut Select Modules:		454ms
Dracut Prepare Install Modules:	29ms
Dracut Install Modules:		4694ms
Dracut Prepare Compress Image:	748ms
Dracut Compress Image:		4479ms
Dracut Cleanup:			12ms

kexec load and unload in kernel is not counted, took around 0.3s. 

11492 (11.5s) in total.

Before backport of bz1484701 it will use about 14s in my VM.

Most time is now consumed by install dracut modules and compress image.
Mkdumprd parse config could be optimzied to cost only ~10ms by backporting a not merged upstream patch:
[PATCH] mkdumprd: Improve the config reading logic

Dracut module installation related part should be optimized to speed up about ~1000ms in RHEL-7 by backport:
https://github.com/dracutdevs/dracut/pull/545

But it's a bit late for dracut backport, and will cause conflict, and also changes too many codes. So would be better to only keep optimizing the time consumption in RHEL-8, and leave RHEL-7.7 with current time consumption for now.

Comment 15 Kairui Song 2019-03-18 08:58:33 UTC
As commented in comment 14, closing this bug.

*** This bug has been marked as a duplicate of bug 1484701 ***