Bug 206096

Summary: Downloading File(s) From Apache Directory Listing CIFS Share Mounted In /var/www/html Always Stops at 33.9KB
Product: [Fedora] Fedora Reporter: Ryan Wagoner <ryan>
Component: kernelAssignee: Jeff Layton <jlayton>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Brian Brock <bbrock>
Severity: high Docs Contact:
Priority: medium    
Version: 5CC: cebbert, davej, smfrench, steved, wtogami
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-01-15 11:45:47 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
cifs dmesg output none

Description Ryan Wagoner 2006-09-12 01:39:00 UTC
Description of problem:
I have a CIFS share from a Windows 2003 Server mounted in /var/www/html. When
browsing the apache directory listing from a remote machine I see the files.
However when I click on a file it will only download 33.9KB of the file. I used
FC4 before with SMBFS and the same setup worked fine. Now since the switch to
CIFS it does not work. I can copy the file from the cifs share to the
/var/www/html folder and can dowload it remotely just fine.

Version-Release number of selected component (if applicable):
Apache 2.2.2

How reproducible:
Occurs everytime. I have done a clean install of FC5 and its always reproducable.

Steps to Reproduce:
1. mount -t cifs //server/share /var/www/html/share -o username=blah,password=blah
2. browse directory listing from remote machine
3. download file and it will stop at 33.9KB.
  
Additional info:
If you switch to FC4 and repeat the above steps but with -t smbfs the files on
the remote share will download just fine.

Comment 1 Ryan Wagoner 2006-09-12 03:04:02 UTC
I did some additional testing on FC4. Reguardless if I mount the share CIFS or
SMBFS in the /var/www/html/share folder I can browse the apache directory
listing remotely and download the file. This is Apache version 2.0.54 on FC4. 

What I am going to try next is to install the FC5 rpm for apache 2.2.2 on FC4
and see if I can still download the file. If not then it appears the bug is in
Apache.

Comment 2 Joe Orton 2006-09-12 08:33:19 UTC
Try adding to httpd.conf:

  EnableSendfile off
  EnableMMap off


Comment 3 Ryan Wagoner 2006-09-12 21:45:40 UTC
Problem resolved!!! I would have never found those by myself. I read up on those
commands and probably should've been using them all along. I googled my problem
before posting this bug report, but couldn't find anybody with the same issue.
Thanks alot and sorry for posting a useless bug report.

Comment 4 Joe Orton 2006-09-13 08:59:27 UTC
This is a bug in the sendfile support in kernel cifs fs.

Comment 5 Dave Jones 2006-10-16 18:21:37 UTC
A new kernel update has been released (Version: 2.6.18-1.2200.fc5)
based upon a new upstream kernel release.

Please retest against this new kernel, as a large number of patches
go into each upstream release, possibly including changes that
may address this problem.

This bug has been placed in NEEDINFO state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

In the last few updates, some users upgrading from FC4->FC5
have reported that installing a kernel update has left their
systems unbootable. If you have been affected by this problem
please check you only have one version of device-mapper & lvm2
installed.  See bug 207474 for further details.

If this bug is a problem preventing you from installing the
release this version is filed against, please see bug 169613.

If this bug has been fixed, but you are now experiencing a different
problem, please file a separate bug for the new problem.

Thank you.

Comment 6 Ryan Wagoner 2006-10-17 01:44:57 UTC
I updated to the 2.6.18-1.2200.fc5 kernel version and restarted the box.
Commented out the EnableSendfile off and EnableMMap off lines and restarted
httpd. Problem still exists. File transfer stop at 33.9KB. Uncommented the
lines, restarted httpd, and the problem goes away.

Comment 7 Steve French 2006-10-17 02:07:07 UTC
Any chance you could attach or send a network capture of this ie an ethereal
trace (since it appears to be a small amount of data sent before the failure).

Comment 8 Ryan Wagoner 2006-10-17 02:21:40 UTC
Yeah I can get an ethereal trace. Not only does the new kernel have the same
problem with the file transfers stopping at 33.9KB, but now I can't even browse
a directoy with more than 100 or so files in a apache directory listing or at
the console with ls in a CIFS network share. Should I report this as a new bug
in the 2.6.18-1.2200.fc5 kernel? 

Comment 9 Steve French 2006-10-18 03:06:35 UTC
One other thing that could help that is easy (while I struggle to reserve one of
our fc5 test machines long enough to upgrade to this particular version of
kernel) would be to get the dmesg output from this (if it is a short enough
scenario as it seems to be if only 33.9KB is transferred).
  "echo 1 > /proc/fs/cifs/cifsFYI"
as late as possible before the failure, and then attach the data returned by
"dmesg" (or equivalent)


By the way it looks like the new FC5 kernel is easily at recent enough level of
cifs, cifs 1.45, but probably does not have a key unlock fix which fixes a
problem introduced on unlock (to Windows servers which is why I mention it here)
by 2.6.18 posix locking code.  See 

http://kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=6b70c9559bcf381a6521e38b0dd5d3d4d905868a

but seems unlikely to relate to write and our other tests (not including
sendfile test unfortunately) against Windows 2003 did not show such an error (we
have transferred many orders of magnitude larger files in our testing to Windows
2003 from cifs) - perhaps this is related to something sendfile unique.

The readdir (ls) issue is also new and we are looking into trying to reproduce
that but the same considerations apply (ethereal trace would be great, dmesg
output would also be helpful as we narrow this down).  Thanks.

Comment 10 Ryan Wagoner 2006-10-18 04:59:59 UTC
Created attachment 138749 [details]
cifs dmesg output

Comment 11 Ryan Wagoner 2006-10-18 05:09:45 UTC
I attached the dmesg output. Where am I suspossed to run the ethereal trace?
Between the fedora server and windows server?

I also filled a bug report on the ls issue yesterday. Are you saying that the
unlock issue was introduced in the 2.6.18-1.2200.fc5 and wasn't around in the
previous 2.6.17-1.2187_FC5 kernel. As when I updated thats when the problem
occured with the ls command. The ls issue is also present in the latest Fedora
Core 6 Test 3 and Developemnt releases. This is a showstopper for the latest FC5
kernel and FC6.

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=211070

Comment 12 Steve French 2006-10-18 12:15:26 UTC
the unlock issue does not appear to be the problem here, but in any case is
fixed in 2.6.19 (the bug was introduced with the change that enabled much better
posix locking emulation to windows in 2.6.18).  I will request a push of that
tiny patch up to the 2.6.18.point-release (it has already been sent to RedHat)

The ethereal trace would be run on either the fedora server or the windows
server whichever is easier for you.  It is best if the capture is started just
before the problem occurs if that is possible (so we don't have to wade through
too much network trace data).

Comment 13 Jeff Layton 2007-09-20 20:03:06 UTC
Been a while since this BZ was touched. Is this still an issue on more recent
kernels? Also, were you ever able to get the network traces that Steve requested?


Comment 14 Jeff Layton 2008-01-15 11:45:47 UTC
No response in several months. Closing this case with a resolution of
INSUFFICIENT_DATA. Please reopen if you're still able to reproduce this and can
provide the requested info.