Bug 2000926

Summary:	Latest kmod-kvdo does not work with the latest RHEL-9.0.0 kernel
Product:	Red Hat Enterprise Linux 9	Reporter:	RHEL Program Management Team <pgm-rhel-tools>
Component:	kmod-kvdo	Assignee:	Andy Walsh <awalsh>
Status:	CLOSED ERRATA	QA Contact:	Filip Suba <fsuba>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	CentOS Stream	CC:	awalsh, bstinson, cwei, fsuba, jstodola, jwboyer, osci-admins+bakery, pvlasin
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1997635
Clones:	2060771 (view as bug list)		Environment:
Last Closed:	2022-05-17 15:49:27 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2060771

Comment 2 Andy Walsh 2021-10-13 23:00:12 UTC

kmod-kvdo-8.1.0.316-3.el9 has been found to not work with kernel-5.14.0-6.el9:

:: [ 17:33:37 ] :: [  BEGIN   ] :: Running 'uname -a'
Linux localhost.localdomain 5.14.0-6.el9.x86_64 #1 SMP Fri Oct 8 17:48:27 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
:: [ 17:33:37 ] :: [   PASS   ] :: Command 'uname -a' (Expected 0, got 0)
:: [ 17:33:37 ] :: [  BEGIN   ] :: Running 'find /lib/modules -name kvdo.ko'
/lib/modules/5.14.0-4.el9.x86_64/extra/kmod-kvdo/vdo/kvdo.ko
:: [ 17:33:37 ] :: [   PASS   ] :: Command 'find /lib/modules -name kvdo.ko' (Expected 0, got 0)
:: [ 17:33:37 ] :: [  BEGIN   ] :: Running 'modinfo uds'
modinfo: ERROR: Module uds not found.
:: [ 17:33:37 ] :: [   FAIL   ] :: Command 'modinfo uds' (Expected 0, got 1)
:: [ 17:33:37 ] :: [  BEGIN   ] :: Running 'dmesg -c > /dev/null'
:: [ 17:33:37 ] :: [   PASS   ] :: Command 'dmesg -c > /dev/null' (Expected 0, got 0)
:: [ 17:33:37 ] :: [  BEGIN   ] :: Running 'insmod /lib/modules/5.14.0-4.el9.x86_64/extra/kmod-kvdo/uds/uds.ko'
insmod: ERROR: could not insert module /lib/modules/5.14.0-4.el9.x86_64/extra/kmod-kvdo/uds/uds.ko: Unknown symbol in module
:: [ 17:33:37 ] :: [   FAIL   ] :: Command 'insmod /lib/modules/5.14.0-4.el9.x86_64/extra/kmod-kvdo/uds/uds.ko' (Expected 0, got 1)
:: [ 17:33:37 ] :: [  BEGIN   ] :: Running 'dmesg'
[   88.017555] uds: loading out-of-tree module taints kernel.
[   88.019724] uds: module verification failed: signature and/or required key missing - tainting kernel
[   88.021500] uds: Unknown symbol dm_bufio_prefetch (err -2)
[   88.022597] uds: Unknown symbol dm_bufio_mark_buffer_dirty (err -2)
[   88.023837] uds: Unknown symbol dm_bufio_read (err -2)
[   88.024814] uds: disagrees about version of symbol wake_up_process
[   88.025929] uds: Unknown symbol wake_up_process (err -22)
[   88.026908] uds: disagrees about version of symbol blkdev_put
[   88.027942] uds: Unknown symbol blkdev_put (err -22)
[   88.028836] uds: Unknown symbol dm_bufio_client_destroy (err -2)
[   88.029908] uds: Unknown symbol dm_bufio_get_block_data (err -2)
[   88.030994] uds: disagrees about version of symbol blkdev_get_by_dev
[   88.032121] uds: Unknown symbol blkdev_get_by_dev (err -22)
[   88.033133] uds: disagrees about version of symbol blkdev_get_by_path
[   88.034270] uds: Unknown symbol blkdev_get_by_path (err -22)
[   88.035283] uds: Unknown symbol dm_bufio_client_create (err -2)
[   88.036325] uds: Unknown symbol dm_bufio_release (err -2)
[   88.037309] uds: disagrees about version of symbol pv_ops
[   88.038308] uds: Unknown symbol pv_ops (err -22)
[   88.039205] uds: Unknown symbol dm_bufio_write_dirty_buffers (err -2)
[   88.040357] uds: Unknown symbol dm_bufio_new (err -2)
[   88.041259] uds: Unknown symbol dm_bufio_set_sector_offset (err -2)
:: [ 17:33:37 ] :: [   PASS   ] :: Command 'dmesg' (Expected 0, got 0)
:: [ 17:33:37 ] :: [   FAIL   ] :: Module/Kernel incompatibility detected 

A rebuild is in the works.

Comment 3 Andy Walsh 2021-11-11 19:24:49 UTC

kmod-kvdo-8.1.0.316-4.el9 has been found to not work with kernel-5.14.0-14.el9.

:: [ 09:08:08 ] :: [  BEGIN   ] :: Running 'uname -a'
Linux localhost.localdomain 5.14.0-14.el9.x86_64 #1 SMP Wed Nov 10 21:42:06 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
:: [ 09:08:08 ] :: [   PASS   ] :: Command 'uname -a' (Expected 0, got 0)
:: [ 09:08:08 ] :: [  BEGIN   ] :: Running 'find /lib/modules -name kvdo.ko'
/lib/modules/5.14.0-7.el9.x86_64/extra/kmod-kvdo/vdo/kvdo.ko
:: [ 09:08:08 ] :: [   PASS   ] :: Command 'find /lib/modules -name kvdo.ko' (Expected 0, got 0)
:: [ 09:08:08 ] :: [  BEGIN   ] :: Running 'modinfo uds'
modinfo: ERROR: Module uds not found.
:: [ 09:08:08 ] :: [   FAIL   ] :: Command 'modinfo uds' (Expected 0, got 1)
:: [ 09:08:08 ] :: [  BEGIN   ] :: Running 'dmesg -c > /dev/null'
:: [ 09:08:08 ] :: [   PASS   ] :: Command 'dmesg -c > /dev/null' (Expected 0, got 0)
:: [ 09:08:08 ] :: [  BEGIN   ] :: Running 'insmod /lib/modules/5.14.0-7.el9.x86_64/extra/kmod-kvdo/uds/uds.ko'
insmod: ERROR: could not insert module /lib/modules/5.14.0-7.el9.x86_64/extra/kmod-kvdo/uds/uds.ko: Invalid module format
:: [ 09:08:08 ] :: [   FAIL   ] :: Command 'insmod /lib/modules/5.14.0-7.el9.x86_64/extra/kmod-kvdo/uds/uds.ko' (Expected 0, got 1)
:: [ 09:08:08 ] :: [  BEGIN   ] :: Running 'dmesg'
[  131.234502] uds: disagrees about version of symbol module_layout
:: [ 09:08:08 ] :: [   PASS   ] :: Command 'dmesg' (Expected 0, got 0)
:: [ 09:08:08 ] :: [   FAIL   ] :: Module/Kernel incompatibility detected 

A rebuild has been completed. kmod-kvdo-8.1.0.316-5.el9 addresses the issue introduced with kernel-5.14.0-14.el9.

Comment 4 Andy Walsh 2021-12-08 14:37:59 UTC

kmod-kvdo-8.1.0.316-5.el9 has been found to not work with kernel-5.14.0-22.el9.

:: [ 17:44:18 ] :: [  BEGIN   ] :: Running 'uname -a'
Linux localhost.localdomain 5.14.0-22.el9.x86_64 #1 SMP Fri Nov 26 19:34:42 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
:: [ 17:44:18 ] :: [   PASS   ] :: Command 'uname -a' (Expected 0, got 0)
:: [ 17:44:18 ] :: [  BEGIN   ] :: Running 'find /lib/modules -name kvdo.ko'
/lib/modules/5.14.0-14.el9.x86_64/extra/kmod-kvdo/vdo/kvdo.ko
:: [ 17:44:18 ] :: [   PASS   ] :: Command 'find /lib/modules -name kvdo.ko' (Expected 0, got 0)
:: [ 17:44:18 ] :: [  BEGIN   ] :: Running 'modinfo uds'
modinfo: ERROR: Module uds not found.
:: [ 17:44:18 ] :: [   FAIL   ] :: Command 'modinfo uds' (Expected 0, got 1)
:: [ 17:44:18 ] :: [  BEGIN   ] :: Running 'dmesg -c > /dev/null'
:: [ 17:44:18 ] :: [   PASS   ] :: Command 'dmesg -c > /dev/null' (Expected 0, got 0)
:: [ 17:44:18 ] :: [  BEGIN   ] :: Running 'insmod /lib/modules/5.14.0-14.el9.x86_64/extra/kmod-kvdo/uds/uds.ko'
insmod: ERROR: could not insert module /lib/modules/5.14.0-14.el9.x86_64/extra/kmod-kvdo/uds/uds.ko: Invalid module format
:: [ 17:44:18 ] :: [   FAIL   ] :: Command 'insmod /lib/modules/5.14.0-14.el9.x86_64/extra/kmod-kvdo/uds/uds.ko' (Expected 0, got 1)
:: [ 17:44:18 ] :: [  BEGIN   ] :: Running 'dmesg'
[  101.302135] uds: disagrees about version of symbol module_layout
:: [ 17:44:19 ] :: [   PASS   ] :: Command 'dmesg' (Expected 0, got 0)
:: [ 17:44:19 ] :: [   FAIL   ] :: Module/Kernel incompatibility detected 

A rebuild has been completed.  kmod-kvdo-8.1.0.316-6.el9 addresses the issue introduced with kernel-5.14.0-22.el9.

Comment 5 Andy Walsh 2021-12-09 19:00:45 UTC

kmod-kvdo-8.1.0.316-6.el9 has been found to not work with kernel-5.14.0-27.el9 on x86_64.

:: [ 13:22:28 ] :: [  BEGIN   ] :: Running 'uname -a'
Linux localhost.localdomain 5.14.0-27.el9.x86_64 #1 SMP PREEMPT Tue Dec 7 22:00:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
:: [ 13:22:28 ] :: [   PASS   ] :: Command 'uname -a' (Expected 0, got 0)
:: [ 13:22:28 ] :: [  BEGIN   ] :: Running 'find /lib/modules -name kvdo.ko'
/lib/modules/5.14.0-26.el9.x86_64/weak-updates/kmod-kvdo/vdo/kvdo.ko
/lib/modules/5.14.0-25.el9.x86_64/extra/kmod-kvdo/vdo/kvdo.ko
:: [ 13:22:28 ] :: [   PASS   ] :: Command 'find /lib/modules -name kvdo.ko' (Expected 0, got 0)
:: [ 13:22:28 ] :: [  BEGIN   ] :: Running 'modinfo uds'
modinfo: ERROR: Module uds not found.
:: [ 13:22:28 ] :: [   FAIL   ] :: Command 'modinfo uds' (Expected 0, got 1)
:: [ 13:22:28 ] :: [  BEGIN   ] :: Running 'dmesg -c > /dev/null'
:: [ 13:22:28 ] :: [   PASS   ] :: Command 'dmesg -c > /dev/null' (Expected 0, got 0)
:: [ 13:22:28 ] :: [  BEGIN   ] :: Running 'insmod /lib/modules/5.14.0-25.el9.x86_64/extra/kmod-kvdo/uds/uds.ko'
insmod: ERROR: could not insert module /lib/modules/5.14.0-25.el9.x86_64/extra/kmod-kvdo/uds/uds.ko: Invalid module format
:: [ 13:22:28 ] :: [   FAIL   ] :: Command 'insmod /lib/modules/5.14.0-25.el9.x86_64/extra/kmod-kvdo/uds/uds.ko' (Expected 0, got 1)
:: [ 13:22:28 ] :: [  BEGIN   ] :: Running 'dmesg'
[  149.494481] uds: disagrees about version of symbol module_layout
:: [ 13:22:28 ] :: [   PASS   ] :: Command 'dmesg' (Expected 0, got 0)
:: [ 13:22:28 ] :: [   FAIL   ] :: Module/Kernel incompatibility detected

A rebuild is in process.  kmod-kvdo-8.1.0.316-7.el9 will be created to address this issue.

Comment 6 Andy Walsh 2021-12-15 14:15:07 UTC

kmod-kvdo-8.1.0.316-7.el9 has been found to not work with kernel-5.14.0-31.el9 on all arches.

:: [ 09:11:21 ] :: [  BEGIN   ] :: Running 'uname -a'
Linux localhost.localdomain 5.14.0-31.el9.x86_64 #1 SMP PREEMPT Tue Dec 14 16:03:54 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
:: [ 09:11:21 ] :: [   PASS   ] :: Command 'uname -a' (Expected 0, got 0)
:: [ 09:11:21 ] :: [  BEGIN   ] :: Running 'find /lib/modules -name kvdo.ko'
/lib/modules/5.14.0-30.el9.x86_64/weak-updates/kmod-kvdo/vdo/kvdo.ko
/lib/modules/5.14.0-28.el9.x86_64/extra/kmod-kvdo/vdo/kvdo.ko
:: [ 09:11:21 ] :: [   PASS   ] :: Command 'find /lib/modules -name kvdo.ko' (Expected 0, got 0)
:: [ 09:11:21 ] :: [  BEGIN   ] :: Running 'modinfo uds'
modinfo: ERROR: Module uds not found.
:: [ 09:11:21 ] :: [   FAIL   ] :: Command 'modinfo uds' (Expected 0, got 1)
:: [ 09:11:21 ] :: [  BEGIN   ] :: Running 'dmesg -c > /dev/null'
:: [ 09:11:21 ] :: [   PASS   ] :: Command 'dmesg -c > /dev/null' (Expected 0, got 0)
:: [ 09:11:21 ] :: [  BEGIN   ] :: Running 'insmod /lib/modules/5.14.0-28.el9.x86_64/extra/kmod-kvdo/uds/uds.ko'
insmod: ERROR: could not insert module /lib/modules/5.14.0-28.el9.x86_64/extra/kmod-kvdo/uds/uds.ko: Invalid module format
:: [ 09:11:21 ] :: [   FAIL   ] :: Command 'insmod /lib/modules/5.14.0-28.el9.x86_64/extra/kmod-kvdo/uds/uds.ko' (Expected 0, got 1)
:: [ 09:11:21 ] :: [  BEGIN   ] :: Running 'dmesg'
[  142.819463] uds: disagrees about version of symbol module_layout
:: [ 09:11:21 ] :: [   PASS   ] :: Command 'dmesg' (Expected 0, got 0)
:: [ 09:11:21 ] :: [   FAIL   ] :: Module/Kernel incompatibility detected

A rebuild is in process.  kmod-kvdo-8.1.0.316-8.el9 will be created to address this issue.

Comment 7 Andy Walsh 2021-12-20 03:43:54 UTC

kmod-kvdo-8.1.0.316-8.el9 has been found to not work with kernel-5.14.0-33.el9 on all arches.

:: [ 16:06:32 ] :: [  BEGIN   ] :: Running 'uname -a'
Linux localhost.localdomain 5.14.0-33.el9.x86_64 #1 SMP PREEMPT Thu Dec 16 20:53:55 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
:: [ 16:06:32 ] :: [   PASS   ] :: Command 'uname -a' (Expected 0, got 0)
:: [ 16:06:32 ] :: [  BEGIN   ] :: Running 'find /lib/modules -name kvdo.ko'
/lib/modules/5.14.0-31.el9.x86_64/extra/kmod-kvdo/vdo/kvdo.ko
:: [ 16:06:32 ] :: [   PASS   ] :: Command 'find /lib/modules -name kvdo.ko' (Expected 0, got 0)
:: [ 16:06:32 ] :: [  BEGIN   ] :: Running 'modinfo uds'
modinfo: ERROR: Module uds not found.
:: [ 16:06:32 ] :: [   FAIL   ] :: Command 'modinfo uds' (Expected 0, got 1)
:: [ 16:06:32 ] :: [  BEGIN   ] :: Running 'dmesg -c > /dev/null'
:: [ 16:06:32 ] :: [   PASS   ] :: Command 'dmesg -c > /dev/null' (Expected 0, got 0)
:: [ 16:06:32 ] :: [  BEGIN   ] :: Running 'insmod /lib/modules/5.14.0-31.el9.x86_64/extra/kmod-kvdo/uds/uds.ko'
insmod: ERROR: could not insert module /lib/modules/5.14.0-31.el9.x86_64/extra/kmod-kvdo/uds/uds.ko: Invalid module format
:: [ 16:06:32 ] :: [   FAIL   ] :: Command 'insmod /lib/modules/5.14.0-31.el9.x86_64/extra/kmod-kvdo/uds/uds.ko' (Expected 0, got 1)
:: [ 16:06:32 ] :: [  BEGIN   ] :: Running 'dmesg'
[  100.316971] uds: disagrees about version of symbol module_layout
:: [ 16:06:32 ] :: [   PASS   ] :: Command 'dmesg' (Expected 0, got 0)
:: [ 16:06:32 ] :: [   FAIL   ] :: Module/Kernel incompatibility detected

A rebuild is in process.  kmod-kvdo-8.1.0.316-9.el9 will be created to address this issue.

Comment 8 Jan Stodola 2022-01-31 12:40:31 UTC

"modprobe kvdo" is failing with kernel-5.14.0-48.el9:

[root@localhost ~]# modprobe kvdo
modprobe: FATAL: Module kvdo not found in directory /lib/modules/5.14.0-48.el9.x86_64
[root@localhost ~]# rpm -q kernel kmod-kvdo
kernel-5.14.0-48.el9.x86_64
kmod-kvdo-8.1.0.316-10.el9.x86_64
[root@localhost ~]#

Comment 9 Andy Walsh 2022-02-01 02:46:08 UTC

Hello.  kmod-kvdo-8.1.1.287-11.el9 is making its way through the build process.  That will work with kernel-5.14.0-50.el9 and up.

Comment 18 Andy Walsh 2022-03-29 17:39:49 UTC

*** Bug 2056692 has been marked as a duplicate of this bug. ***

Comment 20 Filip Suba 2022-04-07 13:42:44 UTC

Verified with kmod-kvdo-8.1.1.371-24.el9_0.

Comment 22 errata-xmlrpc 2022-05-17 15:49:27 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: kmod-kvdo), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:3919