Bug 1695763

It applies to RHEL7 as well, just you do not have there kernel-modules-extra.
The reason is in "grouping algorithm".

Created attachment 1555492 [details]
Handle independent modules in one run

Having kernel-core-4.18.0-80.el8.x86_64 and kernel-modules-extra-4.18.0-80.el8.x86_64 installed.

Installation times:
Without the patch:

5:05.37 kernel-debug-core-4.18.0-80.el8.x86_64.rpm
5:16.58 kernel-debug-core-4.18.0-80.2.el8.x86_64.rpm
2:28.61 kernel-core-4.18.0-80.1.el8.x86_64.rpm
2:27.07 kernel-core-4.18.0-80.2.el8.x86_64.rpm

With the patch:

52.621 kernel-debug-core-4.18.0-80.el8.x86_64.rpm
52.880 kernel-debug-core-4.18.0-80.2.el8.x86_64.rpm
35.863 kernel-core-4.18.0-80.1.el8.x86_64.rpm
36.577 kernel-core-4.18.0-80.2.el8.x86_64.rpm

6x times faster for the content of kernel-modules-extra-4.18.0-80.el8.x86_64 (it depends of amount of modules and their group configuration).

non-debug versions are compatible so do not perform rollback. Looks like a place for optimization as well.

https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=21181358 scratch build with the patch

> Looks like a place for optimization as well.

Actually, not a lot. That is because of second run of depmod after removing the incompatible link.

So the only possible optimization would be get rid of depmod and implement its logic in the script (may be rewriting it in C).

If I remember correctly, rewriting on python is not an option because some of dependencies. But I'm wondering would tiny lua work here.

Hi,

IMHO the real bottleneck is calling depmod so many times here

http://pkgs.devel.redhat.com/cgit/rpms/kmod/tree/weak-modules?h=rhel-8.1.0#n733

Why is the validation needed for each group and not only once after all the symlinks are created?

This also rises a question: Why are the groups needed? There is an associative array "groups", which is a slightly modified depmod output in format
[module]="module depmod1 depmod2 ..."
for example:
[/lib/modules/4.18.0-80.el8.x86_64/extra/net/sctp/sctp_diag.ko.xz]="/lib/modules/4.18.0-80.el8.x86_64/extra/net/sctp/sctp_diag.ko.xz /lib/modules/4.18.0-80.el8.x86_64/extra/net/sctp/sctp.ko.xz"

But before calling read_modules_list the entry from "groups" is stored in a tmp file using this printf
http://pkgs.devel.redhat.com/cgit/rpms/kmod/tree/weak-modules?h=rhel-8.1.0#n1008

Since the entry from "groups" is not quoted, each line in the tmp file contains just one module.

printf '%s\n' $g > $tmp
vs
printf '%s\n' "$g" > $tmp

From this point on the script works with just a simple index array "modules", where each entry contains just one module, so the whole group info is lost. Based on the fix for this bug and how the add_weak_links works now, I think that if all modules are processed with a single update_modules_for_krel call, the result will be the same.

Am I missing something here?

Thanks

(In reply to Frantisek Hrbata from comment #7)
> Hi,
> 
> IMHO the real bottleneck is calling depmod so many times here
> 
> http://pkgs.devel.redhat.com/cgit/rpms/kmod/tree/weak-modules?h=rhel-8.1.
> 0#n733
> 
> Why is the validation needed for each group and not only once after all the
> symlinks are created?
> 
> This also rises a question: Why are the groups needed? There is an
> associative array "groups", which is a slightly modified depmod output in
> format
> [module]="module depmod1 depmod2 ..."
> for example:
> [/lib/modules/4.18.0-80.el8.x86_64/extra/net/sctp/sctp_diag.ko.xz]="/lib/
> modules/4.18.0-80.el8.x86_64/extra/net/sctp/sctp_diag.ko.xz
> /lib/modules/4.18.0-80.el8.x86_64/extra/net/sctp/sctp.ko.xz"
> 
> But before calling read_modules_list the entry from "groups" is stored in a
> tmp file using this printf
> http://pkgs.devel.redhat.com/cgit/rpms/kmod/tree/weak-modules?h=rhel-8.1.
> 0#n1008
> 
> Since the entry from "groups" is not quoted, each line in the tmp file
> contains just one module.
> 
> printf '%s\n' $g > $tmp
> vs
> printf '%s\n' "$g" > $tmp
> 
> From this point on the script works with just a simple index array
> "modules", where each entry contains just one module, so the whole group
> info is lost. Based on the fix for this bug and how the add_weak_links works
> now, I think that if all modules are processed with a single
> update_modules_for_krel call, the result will be the same.
> 
> Am I missing something here?
> 
> Thanks

you can check git log for the reasoning. Quoting is not a problem there, the modules are read one by one to an array.

If you have a better solution, patches are really welcomed.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3532