Bug 1626245
| Summary: | dracut-033-553 performs recursive find of /home, causing huge performance lag for patch | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Marc Richter <mrichter> |
| Component: | microcode_ctl | Assignee: | Eugene Syromiatnikov <esyr> |
| Status: | CLOSED ERRATA | QA Contact: | Jeff Bastian <jbastian> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.6 | CC: | dracut-maint-list, mrichter, skozina |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | microcode_ctl-2.1-46.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-10-30 09:45:55 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Can you please run dracut again with --debug and post here the output. It should be enough just to run dracut --debug /tmp/x.img on the upgraded system. Based on the information provided by the customer the find was called by //usr/lib/dracut/modules.d/99microcode_ctl-fw_dir_override/module-setup.sh@114(install) which is not in the dracut package. Looking at the microcode_ctl I believe that this issue is already fixed in the latest version of the package, but anyway let's switch it to them to get a confirmation. Yes, unfortunately, the original search command was sloppy, that should be fixed in the latest versions of the microcode_ctl package: * microcode_ctl-2.1-47.el7 (RHEL 7.6); * microcode_ctl-2.1-29.16.el7_5 (RHEL 7.5.z); * microcode_ctl-2.1-22.18.el7_4 (RHEL 7.4.z); * microcode_ctl-2.1-16.19.el7_3 (RHEL 7.3.z); * microcode_ctl-2.1-12.16.el7_2 (RHEL 7.2.z). Verified on RHEL-7.6-20180920.0 (Snapshot 5)
::::::::::::::::::::::::::::::::::
:: Before (7.6 High Touch Beta) ::
::::::::::::::::::::::::::::::::::
[root@dell-pet3420-01 ~]# rpm -q microcode_ctl
microcode_ctl-2.1-44.el7.x86_64
[root@dell-pet3420-01 ~]# cd /
[root@dell-pet3420-01 /]# mkdir -p /home/YOU/SHALL/NOT/PASS
[root@dell-pet3420-01 /]# strace -f -e trace=newfstatat \
dracut --force /tmp/initramfs.img $(uname -r) 2>&1 |
grep -C3 SHALL
[pid 29086] newfstatat(7, "test", {st_mode=S_IFDIR|0700, st_size=93, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(7, "YOU", {st_mode=S_IFDIR|0755, st_size=19, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(7, "YOU", {st_mode=S_IFDIR|0755, st_size=19, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(8, "SHALL", {st_mode=S_IFDIR|0755, st_size=17, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(8, "SHALL", {st_mode=S_IFDIR|0755, st_size=17, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(9, "NOT", {st_mode=S_IFDIR|0755, st_size=18, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(9, "NOT", {st_mode=S_IFDIR|0755, st_size=18, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 29086] newfstatat(10, "PASS", {st_mode=S_IFDIR|0755, st_size=6, ...}, AT_SYMLINK_NOFOLLOW) = 0
::::::::::::::::::::::::::::
:: After (7.6 Snapshot 5) ::
::::::::::::::::::::::::::::
[root@dell-pet3420-01 /]# yum -y update microcode_ctl
...
[root@dell-pet3420-01 /]# rpm -q microcode_ctl
microcode_ctl-2.1-47.el7.x86_64
[root@dell-pet3420-01 /]# strace -f -e trace=newfstatat \
dracut --force /tmp/initramfs.img $(uname -r) 2>&1 |
grep -C3 SHALL
[root@dell-pet3420-01 /]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:3097 |
Created attachment 1481426 [details] tarball of find process Description of problem: When updating an existing system to RHEL 7.6 HTB, dracut takes 30-60 minutes to run. The issue appears to be a recursive 'find' starting at '/' looking for specific intel microcode files/directories. Version-Release number of selected component (if applicable): dracut-033-553.el7.x86_64 with findutils-4.5.11-6.el7.x86_64 (RHEL 7.6 High-touch Beta version) How reproducible: RHEL 7.5 systems updating to RHEL 7.6 HTB via "yum update" with large NFS filesystem mounted home directories Actual results: upgrade takes up to an hour to run Expected results: much shorter upgrade cycle Additional info: attaching files "ps -ef" output while dracut was running, showing the 'dracut' process and its child 'find' process; a tarball of /proc/24409, which was the 'find' process in question; and 'find-output-tmpjHlZ1y', which was a file generated in /tmp/ containing stderr output from the 'find' process (and showing clearly that 'find' was traversing a mounted NFS file system of user home directories that dracut has no business getting anywhere near)