Bug 690224

Summary: Veritas SF 5.1 disagrees about version of symbol aio_complete
Product: Red Hat Enterprise Linux 6 Reporter: Daniel Yeisley <dyeisley>
Component: kernelAssignee: Jeff Moyer <jmoyer>
Status: CLOSED ERRATA QA Contact: Chao Yang <chyang>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.1CC: borgan, chyang, jburke, jmoyer, mgahagan, qcai
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: kernel-2.6.32-128.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 692647 (view as bug list) Environment:
Last Closed: 2011-05-19 12:47:07 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 692647    
Attachments:
Description Flags
fix the abi breakage none

Description Daniel Yeisley 2011-03-23 16:13:27 UTC
Description of problem:
Just last Thursday I was able to get the Veritas Storage Foundation and High Availability 5.1 SP1PR2 test to run on RHEL 6.0.  The test sets up a cluster between two systems and does some failover and filesystem checks.  

After it ran successfully I upgraded the kernel to the 6.1-beta version and saw the following during the test run: 

vxfs: module license 'Proprietary' taints kernel.
vxfs: disagrees about version of symbol aio_complete
vxfs: Unknown symbol aio_complete
vxfs: disagrees about version of symbol aio_put_req
vxfs: Unknown symbol aio_put_req

Shortly thereafter I lost my connection to one of the nodes in the cluster and had to reboot it.  

Version-Release number of selected component (if applicable):
kernel-2.6.32-122.el6.x86_64

How reproducible:
Always.

Steps to Reproduce:
1. Verify that the test works on RHEL6
2. Upgrade kernel to 6.1-beta
3. Run the test again.
  
Actual results:
One of the nodes in the cluster dies.

Expected results:
Test should complete as it does on RHEL 6.

Additional info:
I was able to verify that the test works with kernel-2.6.32-71.18.2.el6.x86_64.rpm.

Comment 2 Jeff Moyer 2011-03-23 17:52:14 UTC
aio_complete and aio_put_req should be on the whitelist.

We'll need two bugs for this, one for the abi breakage (we can use this one) and another (that I'll file) for the whitelist addition.

Comment 4 Brock Organ 2011-03-23 18:09:18 UTC
Hi,

Talking with Dan (the reporter), this regression seems serious, could I request
that we consider this for  snapshot 1 or 2 ...? (adding "snapshot 2" to devel
whiteboard to get it on the radar)

Regards,

Brock

Comment 5 RHEL Program Management 2011-03-23 18:09:53 UTC
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.

Comment 6 Jeff Moyer 2011-03-24 12:54:38 UTC
Created attachment 487317 [details]
fix the abi breakage

Comment 8 Aristeu Rozanski 2011-03-30 14:33:53 UTC
Patch(es) available on kernel-2.6.32-128.el6

Comment 10 Daniel Yeisley 2011-03-30 17:23:00 UTC
I had a chance to test on kernel-2.6.32-128.el6.  The vxfs complaints that I saw before have gone away.  

Instead I now see:
vxdmp: disagrees about version of symbol __invalidate_device
vxdmp: Unknown symbol __invalidate_device

Comment 13 Jeff Moyer 2011-03-30 17:49:42 UTC
(In reply to comment #10)
> I had a chance to test on kernel-2.6.32-128.el6.  The vxfs complaints that I
> saw before have gone away.  
> 
> Instead I now see:
> vxdmp: disagrees about version of symbol __invalidate_device
> vxdmp: Unknown symbol __invalidate_device

Right, that symbol is not on the whitelist, and will not be added to the white list.  We'll need to engage Veritas about that one and see if we can get them using more standard interfaces.

Comment 14 Daniel Yeisley 2011-03-31 14:40:28 UTC
It looks like they've been using this call for a while.  I installed Veritas Storage Foundation 5.0MP3, which was the version run for RHEL 5.x ISV-qe testing.  

[root@dell-pe6800-01 certification]# nm -g /lib/modules/2.6.18-238.el5/veritas/vxvm/vxdmp.ko | grep __
                 U __bdevname
                 U __bitmap_weight
000000006f6ea93d A __crc_vxvm_imc_cleanup
00000000cf454acc A __crc_vxvm_imc_deport
000000008231343d A __crc_vxvm_imc_export
000000000af88ef8 A __crc_vxvm_imc_import
                 w __crc_vxvm_imc_setup
0000000006439a4d A __crc_vxvm_imc_unexport
                 U __elv_add_request
                 U __generic_unplug_device
                 U __get_free_pages
                 U __init_rwsem
                 U __invalidate_device
                 U __kmalloc
                 U __list_add
                 U __memcpy
                 U __mod_timer
0000000000000000 D __this_module
                 U __wake_up
[root@dell-pe6800-01 certification]# uname -a
Linux dell-pe6800-01.rhts.eng.bos.redhat.com 2.6.18-238.el5 #1 SMP Sun Dec 19 14:22:44 EST 2010 x86_64 x86_64 x86_64 GNU/Linux

What's the more standard interface that they should be using?

Comment 15 Jeff Moyer 2011-03-31 15:44:01 UTC
> What's the more standard interface that they should be using?

That depends on what they're trying to do.  It requires their developers to explain that to us.  This bug is not really a good place for that discussion, though.

Comment 16 Daniel Yeisley 2011-03-31 19:19:39 UTC
Ok, marking this verified since the original vxfs symbol problem has gone away.

Comment 17 errata-xmlrpc 2011-05-19 12:47:07 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0542.html