Bug 436811 - RHEL5: Some files with inherit_jdata flag will not allow reads from apache
RHEL5: Some files with inherit_jdata flag will not allow reads from apache
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: gfs-kmod (Show other bugs)
All Linux
medium Severity medium
: rc
: ---
Assigned To: Robert Peterson
Cluster QE
Depends On:
  Show dependency treegraph
Reported: 2008-03-10 12:14 EDT by Shane Bradley
Modified: 2010-10-22 19:08 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-09-02 07:03:10 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Proposed patch (2.97 KB, patch)
2008-12-04 13:08 EST, Robert Peterson
no flags Details | Diff

  None (edit)
Description Shane Bradley 2008-03-10 12:14:19 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20080208 Fedora/ Firefox/

Description of problem:
If the inherit_jdata is set on a directory. The client application
cannot read the file. However, the file is readable if you view
locally from console. It appears that apache knows the file exists,
however the contents is empty.  NFS mounts will give read errors on
the file.

It appears that some files work with apache and some do not.


I have been able to reproduce this issue.  It appears that some files
that are copied over do not display data via apache web server.

For example, crontab does show data whereas fstab does not.

With the directory mounted with nfs you can write to the file, but you
cannot read from it.

# echo "Hello world" >> jdata/fstab
# cat jdata/fstab
cat: jdata/fstab: Input/output error

However, i can run "stat" against the file, but cannot run "file"
against the file.

# stat jdata/fstab
 File: `jdata/fstab'
 Size: 627             Blocks: 8          IO Block: 32768  regular file
Device: 13h/19d Inode: 98544       Links: 1
Access: (0666/-rw-rw-rw-)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2008-03-06 13:57:23.000000000 -0500
Modify: 2008-03-06 14:09:09.000000000 -0500
Change: 2008-03-06 14:09:09.000000000 -0500

# file jdata/fstab

jdata/fstab: ERROR: cannot read `jdata/fstab' (Input/output error)

It appears that inode is readable, however the file contents are not
from the client side(nfs, apache).

Another item to note is that I can view some files in apache but not
nfs.  For example /etc/crontab is displayed in apache, but not in nfs.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
File Edit Options Buffers Tools Help
1. Create gfs fs
2. Mount GFS fs
3. mkdir /gfs/mountpoint/jdata; cp /etc/* /gfs/mountpoint/jdata
4. rm /var/www/html -rf
5. ln -s /gfs/mountpoint /var/www/html
6. gfs_tool setflag inherit_jdata /var/www/html/jdata
7. service httpd restart
8. lynx http://localhost/jdata/fstab
9. lynx http://localhost/jdata/crontab

Actual Results:
The contents of "fstab" are not displayed, however contents of "crontab" are displayed with apache.

Expected Results:
Contents should be displayed by apache.

Additional info:
Comment 1 Robert Peterson 2008-04-16 18:07:30 EDT
I recreated this on roth-01, but I haven't started debugging it yet.
Comment 2 Robert Peterson 2008-04-17 16:20:32 EDT
The problem is caused by this section of code in gfs_sendfile
(in ops_file.c):

        if (gfs_is_jdata(ip))
                retval = -ENOSYS;
                retval = generic_file_sendfile(in_file, offset, count, actor,

This section of code has been in GFS since June 2004.
Comment 4 Robert Peterson 2008-12-04 13:08:40 EST
Created attachment 325714 [details]
Proposed patch

I didn't think it was possible to solve this problem short of writing
a huge new section of code to enable the sendfile operation for gfs
files with the jdata flag.  That would be pointless anyway, because
newer kernels have gotten rid of "sendfile" in favor of "splice".
It would be an even bigger effort to write "splice" for GFS.  That's
why there's been so little activity on this problem.

However, I talked it over with some NFS developers today and we figured
out a way to circumvent the problem that is not too complex or involved.
Basically, this patch fools the file system caller into thinking there
is no "sendfile" operation available for jdata files.  So rather than
using the "sendfile" request and having it rejected, it falls back
on reading the file through an operation that does not involved
"sendfile".  This seems to do the trick.

If the customer is willing to try this patch out, I can get the
appropriate flags set for inclusion into 5.4.  Unfortunately, we are
now past the cutoff for 5.3.
Comment 5 Robert Peterson 2008-12-08 17:01:59 EST
I'd like to know if the user is willing to try out the patch, and if
so, does it fix the problem?  Setting the NEEDINFO flag awaiting their
Comment 8 Robert Peterson 2008-12-15 17:33:29 EST
The patch was committed to the RHEL5 branch of the cluster git tree
for inclusion into RHEL5.4.  The upstream splice_read function has the
same issue, so I ported the patch and pushed it to the master and
STABLE2 branches of the cluster git tree as well.
Comment 9 Robert Peterson 2008-12-15 17:34:00 EST
Changing status to Modified.
Comment 11 Fedora Update System 2009-01-23 21:36:16 EST
gfs2-utils-2.03.11-1.fc9, cman-2.03.11-1.fc9, rgmanager-2.03.11-1.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 13 Chris Feist 2009-04-07 16:29:05 EDT
I've built a package which can be used as a hotfix.  I'd recommend that some testing be done as this package has not gone through our normal QE process.

Comment 18 Chris Ward 2009-07-03 14:01:02 EDT
~~ Attention - RHEL 5.4 Beta Released! ~~

RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!

If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.

Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.

Questions can be posted to this bug or your customer or partner representative.
Comment 20 errata-xmlrpc 2009-09-02 07:03:10 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.