Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 497257

Summary: The tmpfs filesystem goes on readonly mode.
Product: Red Hat Enterprise Linux 5 Reporter: Lachlan McIlroy <lmcilroy>
Component: kernelAssignee: Larry Woodman <lwoodman>
Status: CLOSED ERRATA QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: urgent    
Version: 5.2CC: cward, dzickus, emcnabb, jtluka, qcai, sardella, sbeal, tao, vgaikwad
Target Milestone: rcKeywords: OtherQA
Target Release: 5.3   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-30 06:55:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 525215, 526775, 533192    
Attachments:
Description Flags
Patch to fix this problem none

Description Lachlan McIlroy 2009-04-23 03:09:20 UTC 
Description of problem:
Root filesystem went read-only after do_get_write_access() encountered a memory failure.

do_get_write_access: OOM for frozen_buffer
ext3_reserve_inode_write: aborting transaction: Out of memory in __ext3_journal_get_write_access
EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Out of memory
Aborting journal on device dm-0.
EXT3-fs error (device dm-0) in ext3_dirty_inode: Out of memory
ext3_abort called.
EXT3-fs error (device dm-0): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only 

There was a lot of oomkiller activity around the time this happened.

Version-Release number of selected component (if applicable):
kernel-2.6.18-92.1.13.el5PAE

How reproducible:
Customer has seen this a few times.

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

I did some digging through the code and found this suspect bit in __alloc_pages() added by patch linux-2.6-mm-oom-prevent-from-killing-several-processes.patch from BZ 392351.

--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -994,6 +994,8 @@ nofail_alloc:
 		goto nopage;
 
 rebalance:
+	if (test_thread_flag(TIF_MEMDIE))
+		goto nopage;
 	cond_resched();
 
 	/* We now go into synchronous reclaim */

I'm pretty sure that if a thread tagged with TIF_MEMDIE (which means it has been targetted by the oomkiller) enters this code with __GFP_NOFAIL it can fail to honour that flag and return with NULL.

Maybe the original problem from BZ 392351 could instead be fixed with:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=b43a57bb4dae72e8f7232e7c821a8799eda30022
which wont have this new bug.
Comment 2 Lachlan McIlroy 2009-05-12 01:12:25 UTC 
Updating priority to reflect issue status.
Comment 5 Larry Woodman 2009-09-29 15:54:16 UTC 
Created attachment 363038 [details]
Patch to fix this problem

The attached patch fixes this problem by not bailing out of __alloc_pages if __GFP_NOFAIL was passed in.
Comment 7 RHEL Program Management 2009-10-01 16:16:13 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 13 Don Zickus 2009-11-17 21:55:44 UTC 
in kernel-2.6.18-174.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Please do NOT transition this bugzilla state to VERIFIED until our QE team
has sent specific instructions indicating when to do so.  However feel free
to provide a comment indicating that this fix has been verified.
Comment 16 Chris Ward 2010-02-11 10:32:39 UTC 
~~ Attention Customers and Partners - RHEL 5.5 Beta is now available on RHN ~~

RHEL 5.5 Beta has been released! There should be a fix present in this 
release that addresses your request. Please test and report back results 
here, by March 3rd 2010 (2010-03-03) or sooner.

Upon successful verification of this request, post your results and update 
the Verified field in Bugzilla with the appropriate value.

If you encounter any issues while testing, please describe them and set 
this bug into NEED_INFO. If you encounter new defects or have additional 
patch(es) to request for inclusion, please clone this bug per each request
and escalate through your support representative.
Comment 20 errata-xmlrpc 2010-03-30 06:55:34 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0178.html