Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1931546

Summary: oracleasm fails to load in kernel-4.18.0-240.15.1.el8_3
Product: Red Hat Enterprise Linux 8 Reporter: John Pittman <jpittman>
Component: kmod-redhat-oracleasmAssignee: Eugene Syromiatnikov <esyr>
Status: CLOSED CURRENTRELEASE QA Contact: Ziqian SUN (Zamir) <zsun>
Severity: urgent Docs Contact: Jaroslav Klech <jklech>
Priority: urgent    
Version: 8.3CC: bhull, esyr, jmagrini, lmiksik, loberman, nyewale, shivgupt, skozina
Target Milestone: rcKeywords: Regression, Triaged, ZStream
Target Release: ---Flags: pm-rhel: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1933087 1933234 (view as bug list) Environment:
Last Closed: 2021-03-26 08:55:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1933087, 1933234    

Description John Pittman 2021-02-22 16:34:11 UTC 
Description of problem:

oracleasm fails to load in 4.18.0-240.15.1.el8_3

Version-Release number of selected component (if applicable):

kernel-4.18.0-240.15.1.el8_3
kmod-redhat-oracleasm-8:2.0.8-1.el8_3

How reproducible:

Every time

Steps to Reproduce:

Install latest RHEL8 kernel
Boot to kernel
Install kmod-redhat-oracleasm-8:2.0.8-1.el8_3
Execute 'modprobe oracleasm'

Actual results:

[root@localhost ~]# modprobe oracleasm
modprobe: FATAL: Module oracleasm not found in directory /lib/modules/4.18.0-240.15.1.el8_3.x86_64

Expected results:

Module should load

Additional Info:

kernel-4.18.0-240.10.1.el8_3 works, so this is a regression
Comment 2 loberman 2021-02-22 17:57:30 UTC 
I built this to test it with kernel 240.15.1 booted

Source Package - kmod-redhat-oracleasm-2.0.8-1.el8_3.src.rpm
SHA-256: f844a80f149f18356d3813e01c96c4a976efe87ee7b8d6fa385d8c82bcb50d05

My system
Linux segstorage3.cee.lab.eng.bos.redhat.com 4.18.0-240.15.1.el8_3.x86_64 #1 SMP Wed Feb 3 03:12:15 EST 2021 x86_64 x86_64 x86_64 GNU/Linux

It built clean, but required the GA kernel-devel which is normal for kmods on a minor release

Recommends: kmod-redhat-oracleasm-debugsource(x86-64) = 8:2.0.8-1.el8
Checking for unpackaged file(s): /usr/lib/rpm/check-files /root/upstream/linux/BUILDROOT/kmod-redhat-oracleasm-2.0.8-1.el8.x86_64
Wrote: /root/upstream/linux/RPMS/x86_64/kmod-redhat-oracleasm-2.0.8-1.el8.x86_64.rpm
Wrote: /root/upstream/linux/RPMS/x86_64/kmod-redhat-oracleasm-debugsource-2.0.8-1.el8.x86_64.rpm
Wrote: /root/upstream/linux/RPMS/x86_64/kmod-redhat-oracleasm-debuginfo-2.0.8-1.el8.x86_64.rpm

Tried to load it

[root@segstorage3 x86_64]# rpm -Uvh kmod-redhat-oracleasm-2.0.8-1.el8.x86_64.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:kmod-redhat-oracleasm-8:2.0.8-1.e################################# [100%]

[root@segstorage3 modules]# find . -name oracleasm.ko
./4.18.0-240.1.1.el8_3.x86_64/weak-updates/kmod-redhat-oracleasm/oracleasm.ko
./4.18.0-240.el8.x86_64/extra/kmod-redhat-oracleasm/oracleasm.ko
[root@segstorage3 modules]# ls -lt ./4.18.0-240.1.1.el8_3.x86_64/weak-updates/kmod-redhat-oracleasm/oracleasm.ko ./4.18.0-240.el8.x86_64/extra/kmod-redhat-oracleasm/oracleasm.ko
lrwxrwxrwx 1 root root     75 Feb 22 12:53 ./4.18.0-240.1.1.el8_3.x86_64/weak-updates/kmod-redhat-oracleasm/oracleasm.ko -> /lib/modules/4.18.0-240.el8.x86_64/extra/kmod-redhat-oracleasm/oracleasm.ko
-rw-r--r-- 1 root root 139424 Feb 22 12:49 ./4.18.0-240.el8.x86_64/extra/kmod-redhat-oracleasm/oracleasm.ko

[root@segstorage3 kmod-redhat-oracleasm]# insmod ./oracleasm.ko
insmod: ERROR: could not insert module ./oracleasm.ko: Invalid parameters

[ 1381.690562] oracleasm: disagrees about version of symbol perf_trace_run_bpf_submit
[ 1381.733840] oracleasm: Unknown symbol perf_trace_run_bpf_submit (err -22)
[ 1416.158146] oracleasm: disagrees about version of symbol perf_trace_run_bpf_submit
[ 1416.199771] oracleasm: Unknown symbol perf_trace_run_bpf_submit (err -22)
Comment 3 loberman 2021-02-22 18:32:53 UTC 
Symbol is in all these kernels already

System.map-4.18.0-193.14.3.el8_2.x86_64:ffffffff81200bb0 T perf_trace_run_bpf_submit
System.map-4.18.0-193.14.3.el8_2.x86_64:ffffffff82181370 r __ksymtab_perf_trace_run_bpf_submit
System.map-4.18.0-193.14.3.el8_2.x86_64:ffffffff82198ac6 r __kstrtab_perf_trace_run_bpf_submit

System.map-4.18.0-240.1.1.el8_3.x86_64:ffffffff81211780 T perf_trace_run_bpf_submit
System.map-4.18.0-240.1.1.el8_3.x86_64:ffffffff8219bde0 r __ksymtab_perf_trace_run_bpf_submit
System.map-4.18.0-240.1.1.el8_3.x86_64:ffffffff821b3b52 r __kstrtab_perf_trace_run_bpf_submit

System.map-4.18.0-240.15.1.el8_3.x86_64:ffffffff81211840 T perf_trace_run_bpf_submit
System.map-4.18.0-240.15.1.el8_3.x86_64:ffffffff8219be30 r __ksymtab_perf_trace_run_bpf_submit
System.map-4.18.0-240.15.1.el8_3.x86_64:ffffffff821b3c1c r __kstrtab_perf_trace_run_bpf_submit
Comment 4 loberman 2021-02-22 19:01:20 UTC 
I force built against 240.15.1.el8_3 and its working now
Changed the SPEC file from 240.el8 to 240.15.1.el8_3

So there is some issue here with upward compatibility when building against GA

[root@segstorage3 x86_64]# lsmod | grep oracle
oracleasm              69632  0
[root@segstorage3 x86_64]# rmmod ^C
[root@segstorage3 x86_64]# modinfo oracleasm
filename:       /lib/modules/4.18.0-240.15.1.el8_3.x86_64/extra/kmod-redhat-oracleasm/oracleasm.ko
description:    Kernel driver backing the Generic Linux ASM Library.
author:         Joel Becker, Martin K. Petersen <martin.petersen>
version:        2.0.8
license:        GPL
rhelversion:    8.3
srcversion:     9027FD4020AB5E374115218
depends:        
name:           oracleasm
vermagic:       4.18.0-240.15.1.el8_3.x86_64 SMP mod_unload modversions 
parm:           use_logical_block_size:Prefer logical block size over physical (Y=logical, N=physical [default]) (bool)
Comment 5 loberman 2021-02-22 19:34:43 UTC 
Please note that the source for the module I used was kmod-redhat-oracleasm-2.0.8-1.el8.x86_64

However I believe Eugene had a new version for 8.3
kmod-redhat-oracleasm-2.0.8-3.el8_3.x86_64.rpm

I did have a copy of that version but it fails the same way for me.

Just notes for Eugene when he sees this.

Regards
Laurence
Comment 8 Eugene Syromiatnikov 2021-02-23 22:03:54 UTC 
A rebuild won't suffice, as it won't work on systems that have both pre-4.18.0-240.14.1 and post-4.18.0-240-14.1 kernels installed. There is some attempt to handle this with a following change:
http://pkgs.devel.redhat.com/cgit/rpms/oracleasm/commit/?h=private-esyromia-rhel-8.3.0-multi-kernel-hack
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=35124663
Comment 12 Eugene Syromiatnikov 2021-02-24 20:37:24 UTC 
Here's an updated package with some minor fixes in scripts:
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=35144689

Ziqian, may I ask to try to fiddle with it?  I tried some combinations of kmod/kernels installation/removal, but I'm afraid I wasn't thorough enough.
Comment 32 Ziqian SUN (Zamir) 2021-03-26 06:20:02 UTC 
The RHEL-8.3 fix has been released
https://access.redhat.com/errata/RHBA-2021:0983
Comment 35 Eugene Syromiatnikov 2021-03-26 08:55:05 UTC 
The updated packages, that contain the fix, have been released[1].

[1] https://access.redhat.com/errata/RHBA-2021:0983