Bug 1140812

Summary: Bug in DKMS inter-modules dependencies feature
Product: [Fedora] Fedora Reporter: Bruno Faccini <bruno.faccini>
Component: dkmsAssignee: Simone Caronni <negativo17>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 20CC: bob.glossman, charles_rose, dyoung, gcase, lwang, mario_limonciello, negativo17, peter.a.jones, tcallawa
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: dkms-2.2.0.3-28.git.7c3e7c5.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1143051 (view as bug list) Environment:
Last Closed: 2014-10-08 18:58:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1143051    

Description Bruno Faccini 2014-09-11 18:21:10 UTC
Description of problem:
DKMS inter-module dependencies feature is broken and does not handle simple scenarios like spl->zfs->lustre required build order discovery when both zfs and lustre DKMS RPMs have correctly provided a dkms.conf config-file with respectively BUILD_DEPENDS[0]="spl" and BUILD_DEPENDS[0]="zfs" statements.

Version-Release number of selected component (if applicable):
2.2.0.3-20.fc20

How reproducible:
on demand

Steps to Reproduce:
1. install lustre-dkms, spl-dkms, zfs-dkms RPMs along with a new kernel RPM and its kernel-devel companion.
2. reboot. lustre modules rebuild will fail (missing zfs sources/headers), when zfs and spl will succesfully.
3. a 2nd reboot should fix/work now zfs has builded.

you can also force it to reproduce without any reboot with the following steps :
1. dkms remove [lustre,zfs,spl] modules
2. dkms add [lustre,zfs,spl] modules
3. dkms autoinstall, will fail like during a reboot. And a second autoinstall should fix/work.

Actual results:
Automated DKMS rebuild fails. Need manual/human intervention to fix!

Expected results:
Automated DKMS rebuild success.

Additional info:

The problem is that upon 1st reboot/autoinstall, DKMS seems to correctly detect that all 3 modules need to be rebuilt, but then lustre-dkms is rebuilt without regards to its zfs dependency and fails, when in the same time spl-dkms is correctly rebuilt before zfs-dkms to satisfy their own inter-dependency. Upon 2nd reboot/autoinstall, lustre-dkms rebuild is again attempted and can succeed.

Having a look to the dkms source-code/script, I think this problem comes from the fact that the inter-modules dependencies detection code is buggy and that the following patch will fix it :
=============================================================================
--- ./dkms.orig 2014-09-10 08:05:43.000000000 -0700
+++ ./dkms.bfi  2014-09-11 11:15:01.000000000 -0700
@@ -3181,9 +3181,20 @@
 
     # Walk through our list of installed and built modules, and create
     # a list of modules that need to be reinstalled.
+    # Assuming all versions of same module to be parsed sequentially.
     while read status mvka; do
         IFS='/' read m v k a <<< "$mvka"
         [[ ! $last_m ]] && last_m="$m"
+        # autoinstall previous module's latest version after its deps.
+        if [[ $last_m != $m ]]; then
+            if [[ $tenative ]]; then
+                to_install[${#to_install[@]}]="$tenative"
+                build_depends["$last_m"]="${BUILD_DEPENDS[@]}"
+            fi
+            last_m="$m"
+            last_v='0'
+            tenative=''
+       fi
         # If the module is already installed or weak-installed, skip it.
         if _is_module_installed "$m" "$v" "$kernelver" "$arch"; then
                installed_modules[${#installed_modules[@]}]="$m"
@@ -3197,27 +3208,21 @@
         read_conf_or_die "$k" "$a" "$dkms_tree/$m/$v/source/dkms.conf"
         if [[ ! $AUTOINSTALL ]]; then
             continue
-        # otherwise, only autoinstall the latest version we have hanging around.
-        elif [[ $last_m != $m ]]; then
-            last_m="$m"
-            last_v='0'
-            if [[ $tenative ]]; then
-                to_install[${#to_install[@]}]="$tenative"
-                build_depends["$m"]="${BUILD_DEPENDS[@]}"
-            fi
-            tenative=''
         fi
+       # otherwise, only autoinstall the latest version we have hanging around.
         if [[ ($(VER $v) > $(VER $last_v)) ]]; then
             last_v="$v"
             tenative="$m/$v"
         fi
     done < <(module_status)
+
     # We may have exited the loop with $tenative set.  If it is,
     # it contains something that should be updated.
     if [[ $tenative ]]; then
         to_install[${#to_install[@]}]="$tenative"
         build_depends["$m"]="${BUILD_DEPENDS[@]}"
     fi
+
     [[ $to_install ]] || return 0
 
        while true; do
@@ -3257,6 +3262,11 @@
                # Step 4: Remove modules that were installed during Step 2 from
                # the job queue.
                to_install=( "${next_install[@]}" )
+
+       done
+       for mv in "${to_install[@]}"; do
+               IFS=/ read m v <<< "$mv"
+               echo "$m/$v autoinstall failed due to missing dependencies: ${build_depends[$m]}"
        done
 } 

=============================================================================
At least it fixes the spl->zfs->lustre scenario!
It will also report module re-build failures due to missing dependencies.

BTW, what is the current git repo for DKMS and where I can submit this patch proposal ??

Comment 1 Simone Caronni 2014-09-17 17:26:15 UTC
I'm trying to push all the patches we have added to the DKMS package in Fedora to the upstream repository. It was abandoned until a few weeks ago, when it was revived with a couple of commits.

I'll try to add this patch to the package asap.

Comment 2 Simone Caronni 2014-09-24 09:05:31 UTC
All the current RHEL/CentOS/Fedora patches have been merged upstream along with additional patches. This patch is next:

http://pkgs.fedoraproject.org/cgit/dkms.git/log/

I'm waiting on a reply from the upstream maintainer before building the package for all branches; for now I'm building it for rawhide only.

Comment 3 Fedora Update System 2014-09-25 08:13:12 UTC
dkms-2.2.0.3-28.git.7c3e7c5.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/dkms-2.2.0.3-28.git.7c3e7c5.fc21

Comment 4 Fedora Update System 2014-09-25 08:13:26 UTC
dkms-2.2.0.3-28.git.7c3e7c5.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/dkms-2.2.0.3-28.git.7c3e7c5.fc20

Comment 5 Fedora Update System 2014-09-25 08:13:41 UTC
dkms-2.2.0.3-28.git.7c3e7c5.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/dkms-2.2.0.3-28.git.7c3e7c5.fc19

Comment 6 Fedora Update System 2014-09-25 08:14:00 UTC
dkms-2.2.0.3-28.git.7c3e7c5.el7 has been submitted as an update for Fedora EPEL 7.
https://admin.fedoraproject.org/updates/dkms-2.2.0.3-28.git.7c3e7c5.el7

Comment 7 Fedora Update System 2014-09-25 08:14:12 UTC
dkms-2.2.0.3-28.git.7c3e7c5.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/dkms-2.2.0.3-28.git.7c3e7c5.el6

Comment 8 Fedora Update System 2014-09-25 08:14:24 UTC
dkms-2.2.0.3-28.git.7c3e7c5.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/dkms-2.2.0.3-28.git.7c3e7c5.el5

Comment 9 Simone Caronni 2014-09-25 08:23:31 UTC
Please test and leave some feedback, as the source tarball now includes all the patches previously added to the Fedora package.

Comment 10 Fedora Update System 2014-09-26 09:00:26 UTC
Package dkms-2.2.0.3-28.git.7c3e7c5.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dkms-2.2.0.3-28.git.7c3e7c5.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-11510/dkms-2.2.0.3-28.git.7c3e7c5.fc19
then log in and leave karma (feedback).

Comment 11 Fedora Update System 2014-10-08 18:58:13 UTC
dkms-2.2.0.3-28.git.7c3e7c5.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 12 Fedora Update System 2014-10-08 19:05:06 UTC
dkms-2.2.0.3-28.git.7c3e7c5.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 13 Fedora Update System 2014-10-08 19:08:03 UTC
dkms-2.2.0.3-28.git.7c3e7c5.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 14 Fedora Update System 2014-10-17 17:35:51 UTC
dkms-2.2.0.3-28.git.7c3e7c5.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2014-10-17 17:40:45 UTC
dkms-2.2.0.3-28.git.7c3e7c5.el7 has been pushed to the Fedora EPEL 7 stable repository.  If problems still persist, please make note of it in this bug report.