Red Hat Bugzilla – Bug 165989
The msync(MS_SYNC) call should fail after cable pulled from scsi disk
Last modified: 2007-11-30 17:07:08 EST
Description of problem:
Customer expects the fix added into bugzilla 116900 can also solve this issue
and has tried out 2.4.21-34.EL without success. Look like the msync(MS_SYNC)
code path needs to get re-examined after the pull-cable-from-the-scsi-disk test.
The problem can be demonstrated by running the uploaded test program which does
msyncs in a loop and then pulling the cables (or disabling the SAN ports). It is
observed that the -34 kernel remount the FS RO and msyncs get read only FS error
(so far so good, until..) then the for loop continues and subsequent msyncs
succeed. Shouldn't they all get read only FS errors ?
Version-Release number of selected component (if applicable):
Steps to Reproduce:
Use the uploaded test program (run as "mmapwrite <filename> <iterations>").
Created attachment 117753 [details]
Test program to demonstrate the problem.
I've been looking at the test program. What is the typical number of iterations
passed through the second command line argument?
1) Thanx for the information!
2) Makes sense to me. Once we figure out what needs to be done, then we can
figure out where to put it.
Yes, Stephen's comment had to do with ways of reproducing problems associated
with issues in the storage subsystem, _without_ having to physically change the
I have prototyped a solution and am currently discussing it with some of the
engineers to get some feedback on it.
Created attachment 117974 [details]
The msync(2) code in RHEL-3 works by walking a list of dirty pages and
arranging to have them written out to storage. When the i/o is done on
each page, they are moved to a clean_pages list. However, when ext3
finds that the file system is readonly, it sets the PG_dirty bit in
the page struct again. Thus, the page is marked as dirty, ie. needs to
be written to storage, but is on the clean_pages list. Having the
dirty bit already set prevents the page from moving from the clean_pages
to the dirty_pages list, thus preventing msync from finding the page
again. Since ext3 is not called into again, the readonly file system
error is not returned again.
The solution is to place the page on the dirty_pages list instead of
the clean_pages list if PG_dirty is set in the page struct. Thus,
msync will continue to find the page and attempt to write it out. This
will call into ext3 and ext3 can then return the readonly file system
Please note that RHEL-4 did not suffer from this issue.
A fix for this problem has just been committed to the RHEL3 U7
patch pool this evening (in kernel version 2.4.21-37.1.EL).
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.