Red Hat Bugzilla – Bug 436811
RHEL5: Some files with inherit_jdata flag will not allow reads from apache
Last modified: 2010-10-22 19:08:17 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:126.96.36.199) Gecko/20080208 Fedora/188.8.131.52-1.fc8 Firefox/184.108.40.206
Description of problem:
If the inherit_jdata is set on a directory. The client application
cannot read the file. However, the file is readable if you view
locally from console. It appears that apache knows the file exists,
however the contents is empty. NFS mounts will give read errors on
It appears that some files work with apache and some do not.
I have been able to reproduce this issue. It appears that some files
that are copied over do not display data via apache web server.
For example, crontab does show data whereas fstab does not.
With the directory mounted with nfs you can write to the file, but you
cannot read from it.
# echo "Hello world" >> jdata/fstab
# cat jdata/fstab
cat: jdata/fstab: Input/output error
However, i can run "stat" against the file, but cannot run "file"
against the file.
# stat jdata/fstab
Size: 627 Blocks: 8 IO Block: 32768 regular file
Device: 13h/19d Inode: 98544 Links: 1
Access: (0666/-rw-rw-rw-) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2008-03-06 13:57:23.000000000 -0500
Modify: 2008-03-06 14:09:09.000000000 -0500
Change: 2008-03-06 14:09:09.000000000 -0500
# file jdata/fstab
jdata/fstab: ERROR: cannot read `jdata/fstab' (Input/output error)
It appears that inode is readable, however the file contents are not
from the client side(nfs, apache).
Another item to note is that I can view some files in apache but not
nfs. For example /etc/crontab is displayed in apache, but not in nfs.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
File Edit Options Buffers Tools Help
1. Create gfs fs
2. Mount GFS fs
3. mkdir /gfs/mountpoint/jdata; cp /etc/* /gfs/mountpoint/jdata
4. rm /var/www/html -rf
5. ln -s /gfs/mountpoint /var/www/html
6. gfs_tool setflag inherit_jdata /var/www/html/jdata
7. service httpd restart
8. lynx http://localhost/jdata/fstab
9. lynx http://localhost/jdata/crontab
The contents of "fstab" are not displayed, however contents of "crontab" are displayed with apache.
Contents should be displayed by apache.
I recreated this on roth-01, but I haven't started debugging it yet.
The problem is caused by this section of code in gfs_sendfile
retval = -ENOSYS;
retval = generic_file_sendfile(in_file, offset, count, actor,
This section of code has been in GFS since June 2004.
Created attachment 325714 [details]
I didn't think it was possible to solve this problem short of writing
a huge new section of code to enable the sendfile operation for gfs
files with the jdata flag. That would be pointless anyway, because
newer kernels have gotten rid of "sendfile" in favor of "splice".
It would be an even bigger effort to write "splice" for GFS. That's
why there's been so little activity on this problem.
However, I talked it over with some NFS developers today and we figured
out a way to circumvent the problem that is not too complex or involved.
Basically, this patch fools the file system caller into thinking there
is no "sendfile" operation available for jdata files. So rather than
using the "sendfile" request and having it rejected, it falls back
on reading the file through an operation that does not involved
"sendfile". This seems to do the trick.
If the customer is willing to try this patch out, I can get the
appropriate flags set for inclusion into 5.4. Unfortunately, we are
now past the cutoff for 5.3.
I'd like to know if the user is willing to try out the patch, and if
so, does it fix the problem? Setting the NEEDINFO flag awaiting their
The patch was committed to the RHEL5 branch of the cluster git tree
for inclusion into RHEL5.4. The upstream splice_read function has the
same issue, so I ported the patch and pushed it to the master and
STABLE2 branches of the cluster git tree as well.
Changing status to Modified.
gfs2-utils-2.03.11-1.fc9, cman-2.03.11-1.fc9, rgmanager-2.03.11-1.fc9 has been pushed to the Fedora 9 stable repository. If problems still persist, please make note of it in this bug report.
I've built a package which can be used as a hotfix. I'd recommend that some testing be done as this package has not gone through our normal QE process.
~~ Attention - RHEL 5.4 Beta Released! ~~
RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!
If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.
Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.
Questions can be posted to this bug or your customer or partner representative.
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.