Red Hat Bugzilla – Bug 187295
CIFS under heavy load crashes system
Last modified: 2007-11-30 17:07:24 EST
Description of problem:
When put under heavy load, the CIFS driver crashes bringing down the rest of the
system with it.
Version-Release number of selected component (if applicable):
Always (with sufficient load)
Steps to Reproduce:
1. Mount a CIFS share.
2. Put the mount under heavy load.
I have tried using both Windows 2003 Server and a TeraStation NAS (old samba) as
the server for the mount. I have also tried using both the CIFS driver and the
(deprecated) SMBFS driver. What I did find strange, though was the following:
cp -a /40 /GBs /of /data /mnt/cifsShare
This takes a long time, but does NOT result in a crash. However:
for a in /\40 /\GBs /\of /\data; do
rsync -rtv --delete $a /mnt/cifsShare
for a in /\40 /\GBs /\of /\data; do
rsync -av --delete $a /mnt/cifsShare
both results in a crash.
Unfortunately, this is on a production server, so when the box goes down I have
to reboot it immediately and haven't been able to grab any of the console info.
//server/share /mnt/cifsShare cifs
credentials=/etc/cifs.creds,gid=users,dir_mode=0770,file_mode=0660 0 0
This seems similar to closed bug #174982.
is this a regression from U2 cifs behavior?
I experienced the same behavior in U2 and was waiting to test in U3 due to the
looks like we need another update :(
Then I won't report the "processes become deadlocked when cifs server goes away"
bug that I'm trying to figure out as well... :)
Is there an estimate on the next kernel rollout?
Do you know offhand if the e1000 issues have been resolved in the U3? This
machine uses the e1000 driver with an onboard card as well as a dual-port PCI-X
card. I'd hate to blame the wrong module... =)
Guess I need to do some more test...
Okay, I ran stress tests on the e1000 driver and was unable to make it crash, so
I believe this is actually a CIFS issue.
I'm seeing the same behavior as well on a HP DL380 g3 (tg3 based nic's).
Luckily (?) I seem to be able to reporduce this at will with rsync, as it
generally locks when rsync'ing a specific user's data from one system to the
CIFS filesystem. Though the load is not heavy at all (on both the system as a
whole and the filesystem).
Please LMK if's there any specific debugging steps you'd like me to take.
Created attachment 130880 [details]
OOPS from kernel
OOPS from serial console when running an rsync against user data that triggers
Hurm.. Looking at the OOPS, the top of the call trace is cifs_rename.
Oddly, the directory with the bad data contains two files "Buddy.jpg" and
"buddy.jpg" The first one syncs just duckily, the latter one is the last file
rsync spits out before the crash. Could this be a case sensitivity issue
tickling a deadlock?
I can not reporduce this with vi, so rsync must do domething funky when copying?
ok, here is a pointer to a changeset which might resolve this issue:
I've examined the backtrace in comment #9, and it is indeed falling over on
exactly the same code that is fixed in the above patch. Therefore, i think it is
likely that this patch will solve your issue.
I've crated a test kernel with this patch (it doesn't quite apply cleanly to
rhel4), based on latest beta kernel: http://people.redhat.com/~jbaron/bz187295/.
I could build it on top of 34.0.1 if you want...
Also, i've contacted the upstream CIFS maintainer, and we are in discussion as
to what further CIFS improvements are appropriate for rhel4.
anybody had a chance to test this? thanks.
That kernel wouldn't boot for me. Hangs on PCI probing. *shrug*
hmmm...sounds like a different issue...if possible could you post the boot log
up to the point where it fails...i'm also curious if the latest U4 beta kernels
work for you, located at: http://people.redhat.com/~jbaron/rhel4/ thanks.
2.6.9-40 has the same cifs module as 2.6.9-34(.0.1), is there any reason to test
As far as the kernel you built goes... it's an i686 and I'm running an x86_64
ok, i've place test kernels for x86 and x86_64 at:
Please let us know if these resolve the issue. thanks.
This issues appears to be resolved in 2.6.9-40.1.EL.cifs.1smp.
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release. Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products. This request is not yet committed for inclusion in an Update
committed in stream U5 build 42.10. A test kernel with this patch is available
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.