Bug 584901

Summary: Using VFS IMA with OpenAFS generates lots of log noise
Product: Red Hat Enterprise Linux 6 Reporter: Bryn M. Reeves <bmr>
Component: kernelAssignee: Eric Paris <eparis>
Status: CLOSED CURRENTRELEASE QA Contact: Filesystem QE <fs-qe>
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: arozansk, bhinson, bkozdemb, borgan, botsch, dejohnso, eguan, eparis, eric.williams, james.leddy, jasonmc, jedgecombe, ktdreyer, kzhang, marc.c.dionne, ralston, sbandodk, simon, stephan.wiesand, tao, yi.zou
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-11 15:48:41 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Patch which fixes IMA object lifetime thus making it work with AFS
none
Same basic fix, hopefully applies to older RHEL6 kernels none

Description Bryn M. Reeves 2010-04-22 17:06:01 UTC
Description of problem:
IMA seems to cause a ton of kernel messages on every file access due to not understanding the openafs cache. This slows down the client and generates large volumes of log traffic, e.g.:

ima_file_free: V9356 open/free imbalance (r:0 w:-21 o:-21 f:0)

The rate at which these messages are logged when running OpenAFS is excessive leading to disk space utilisation and performance problems.

Version-Release number of selected component (if applicable):
2.6.32-*.el6

How reproducible:
100%

Steps to Reproduce:
1. Configure OpenAFS with a RHEL6 kernel that enables IMA
2. Access some files
  
Actual results:
Log spamming of the form:

  ima_file_free: V9356 open/free imbalance (r:0 w:-21 o:-21 f:0)

On every file access.

Expected results:
Upstream has already modified this to limit the volume of log messages.

Additional info:
author	Mimi Zohar <zohar.com>	
committer Eric Paris <eparis>	
	 Wed, 9 Dec 2009 20:58:05 +0000 (15:58 -0500)
commit	632eb15fdd4f87886138ab3511f0b651abffe9df

ima: limit imbalance msg

Limit the number of imbalance messages to once per filesystem type instead of
once per system boot.  (it's actually slightly racy and could give you a
couple per fs, but this isn't a real issue)

Signed-off-by: Mimi Zohar <zohar.com>
Acked-by: Mimi Zohar <zohar.ibm.com>

Comment 2 RHEL Program Management 2010-04-22 18:23:53 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 5 Eric Paris 2010-04-23 14:31:26 UTC
Created attachment 408627 [details]
Patch which fixes IMA object lifetime thus making it work with AFS

Comment 6 Eric Paris 2010-04-23 18:14:20 UTC
*** Bug 585286 has been marked as a duplicate of this bug. ***

Comment 13 Marc Dionne 2010-04-28 01:22:34 UTC
I'm trying to test the attached patch on the kernel source from the 2.6.32-19.el6 SRPM (from ftp.redhat.com/pub/redhat/rhel/beta/6/source/SRPMS/), but it doesn't apply cleanly.  A few hunks fail in fs/namei.c, and it looks like it's based on a different tree than the current SRPM - for instance the context lines have handle_truncate(), which doesn't exist in the version I'm looking at.
I guess you may be working from something more recent than the published SRPM.

Comment 14 Eric Paris 2010-04-28 19:37:06 UTC
Yes I am.  Some of the patches that are needed to make this work are not in the -19 kernel  :(.   I'm going to try to find one of the old versions of the patch for you to test.

Comment 15 Eric Paris 2010-04-28 20:12:14 UTC
Created attachment 409960 [details]
Same basic fix, hopefully applies to older RHEL6 kernels

Comment 16 Marc Dionne 2010-04-29 12:38:48 UTC
That patch does apply cleanly on the -19 kernel, thanks.  I haven't been able to complete the build and test yet, I'll post an update when I have.  It's pretty clear from the code that this should take care of the issue for OpenAFS.

Comment 18 Marc Dionne 2010-04-29 17:13:40 UTC
BTW it would also be very nice to see this fixed in Fedora 12, where it has also been a problem, or a move to 2.6.33.

Comment 19 Marc Dionne 2010-04-30 01:08:04 UTC
I completed some tests and can confirm that as expected, IMA warnings are no longer an issue with the patch applied, tested with the current OpenAFS.

Thanks

Comment 20 Stephan Wiesand 2010-04-30 14:04:39 UTC
I also had success with this patch and OpenAFS-1.5.74 (the current development release) + http://git.openafs.org/?p=openafs.git;a=commit;h=14195f0f48d52dd3a81c52c4a3bc2078857d0f86 . It even works with SELinux in enforcing mode once all the labels are right.

This will be the basic setup for all further work with the beta here.

Thanks for the patch, and for making this BZ public.

Comment 21 Jason McCormick 2010-05-06 21:08:08 UTC
Is it known yet if this patch will be the released version of EL6?  I want to file a support request through our support account if not.

Comment 22 Eric Paris 2010-05-14 19:55:50 UTC
The patch has been posted for review by the internal Red Hat kernel team.  We have not yet committed to inclusion but we are pursuing the process for inclusion in EL6.  I believe that comment #2 is still the 'official' position.

Comment 24 Marc Dionne 2010-07-01 13:28:25 UTC
Is there any update on the 'official' position on this patch?
Had a look at the beta 2 kernel source and it doesn't look like this has made it in yet.

Comment 25 Aristeu Rozanski 2010-07-01 16:11:25 UTC
Patch(es) available on kernel-2.6.32-42.el6

Comment 28 Eric Paris 2010-07-19 15:16:44 UTC
*** Bug 534113 has been marked as a duplicate of this bug. ***

Comment 29 Raghu Udiyar 2010-08-13 15:24:32 UTC
*** Bug 593329 has been marked as a duplicate of this bug. ***

Comment 30 Marc Dionne 2010-08-28 20:24:11 UTC
Since nobody has confirmed here, I'll just mention that I got a chance to test openafs with kernel-2.6.32-44.2 and I can confirm that the issue is indeed resolved.

Comment 32 releng-rhel@redhat.com 2010-11-11 15:48:41 UTC
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.

Comment 33 John Feeney 2011-05-04 22:26:57 UTC
*** Bug 589760 has been marked as a duplicate of this bug. ***