Bug 1775790

Summary: Fix and speed up /usr/lib/rpm/kmod.prov
Product: [Fedora] Fedora Reporter: Denys Vlasenko <dvlasenk>
Component: redhat-rpm-configAssignee: Florian Festi <ffesti>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: ajax, ffesti, fweimer, igor.raits, john.j5live, jonathan, j, pmatilai, praiskup
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-12-06 14:06:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Denys Vlasenko 2019-11-22 19:54:09 UTC
For kernel builds, /usr/lib/rpm/kmod.prov is fork+execed by rpmbuild in "Processing files:" step about 8000 times, single-threaded, with cumulative run time of ~2 minutes.

Ideally rpmbuild should be changed to parallelize this, but it's not trivial to implement.

I propose meanwhile to speed up the script, by avoiding additional fork+execing.
    Current one:

IFS=$'\n'

for i in $(grep -E '(/lib/modules/.*\.ko|/lib/modules/.*/modules.builtin)');
do
        kmod=$(basename $i | sed -e 's/.[xg]z//');

        if [ $kmod == "modules.builtin" ]; then
                for j in $(cat $i); do
                        j=$(basename $j);
                        echo "kmod($j)"
                done
        else
                echo "kmod($kmod)"
        fi
done

    Proposed replacement:

IFS=$'\n'

read -r fname || exit

# Only process files from .../lib/modules/... subtree
[ "${fname#*/lib/modules/*}" != "$fname" ] || exit 0

kmod=${fname##*/}  # like basename, but faster

if [ "$kmod" = "modules.builtin" ]; then
        for j in $(cat -- "$fname"); do
                echo "kmod(${j##*/})"
        done
        exit 0
fi

kmod=${kmod%.gz}
kmod=${kmod%.xz}
if [ "${kmod%.ko}" != "$kmod" ]; then
        echo "kmod($kmod)"
fi


Tested to work, observed speedup: almost exactly 2 times faster.

While verifying correctness, noticed that old script was buggy - it was generating a bogus "Provides:" item - kmod(modules.builtin.modinfo),
because the logic in script was filtering for */*.ko files and for */modules.builtin* files, and wasn't prepared for the existence of */modules.builtin.modinfo file.

Comment 1 Panu Matilainen 2019-12-03 14:44:41 UTC
Please submit a pull-request on redhat-rpm-config, that's much more productive than speculating changes in bugzilla.