Bug 591229

Summary: oops when unmounting CIFS filesystem
Product: Red Hat Enterprise Linux 6 Reporter: Jeff Layton <jlayton>
Component: kernelAssignee: Jeff Layton <jlayton>
Status: CLOSED CURRENTRELEASE QA Contact: yanfu,wang <yanwang>
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: rwheeler, steved, yanwang
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-11 15:53:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jeff Layton 2010-05-11 17:27:14 UTC
Upstream bug here that RHEL6 is also susceptible to:

https://bugzilla.samba.org/show_bug.cgi?id=7407

Basically, we were trusting that the uniqueid sent by the server is actually unique. Samba has a bug where that is often not the case, and it leads to the oops shown in the above samba bz at umount. We'll definitely want this patch in RHEL6. RHEL4/5 shouldn't be vulnerable as they don't have the patches that lead to the problem.

Comment 1 Jeff Layton 2010-05-11 17:30:09 UTC
Steve F. is pushing the patch for this to Linus today and cc'ing stable. It should hopefully make 2.6.34.

Comment 3 RHEL Program Management 2010-05-11 19:21:13 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 4 Jeff Layton 2010-05-13 11:30:46 UTC
Actually, we've decided to hold off on the patch for now until I have a chance to look over some related problems in this code.

Comment 5 Jeff Layton 2010-05-18 19:00:20 UTC
A patch was finally taken in upstream just before 2.6.34 shipped.

Comment 7 Aristeu Rozanski 2010-05-28 20:37:51 UTC
Patch(es) available on kernel-2.6.32-31.el6

Comment 10 yanfu,wang 2010-07-23 07:31:50 UTC
I can't reproduce the problem on rhel6, umount is successful.

# uname -a
Linux dell-pe2950-01.rhts.eng.bos.redhat.com 2.6.32-19.el6.x86_64 #1 SMP Tue Mar 9 17:48:46 EST 2010 x86_64 x86_64 x86_64 GNU/Linux

# rpm -qa|grep samba
samba-winbind-clients-3.4.4-50.el6.x86_64
samba-3.4.4-50.el6.x86_64
samba-common-3.4.4-50.el6.x86_64
samba-client-3.4.4-50.el6.x86_64

# cat /proc/mounts |grep cifs
//localhost/autofs /mnt cifs rw,mand,relatime,unc=\\localhost\autofs,username=root,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,posixpaths,serverino,acl,rsize=16384,wsize=57344 0 0

# umount /mnt


is any specific steps should I follow? pls provide clear reproducer, thanks.

Comment 11 Jeff Layton 2010-07-23 11:09:04 UTC
Suresh J. summarized the reproducer this way:

I have a nice way of reliably reproducing the problem.

- create two small partitions and format (ext3)
- mount first partition as /vol1 and mount the second as /vol1/vol2
  inside the first
- create files on both partitions (say 10 files)
- export them
- mount from the client, access them
- the inode numbers will easily collide
- umount them and the client will crash..

...the key is that you need to export a share that contains the root of both filesystems so you can ensure that their inode numbers collide.

Since this is really a server-side problem, you'll also need to ensure that the samba version you're using doesn't contain the fix for this. I'll have to double check whether that's the case...

Comment 12 Jeff Layton 2010-07-23 11:12:10 UTC
3.4.4 should still have the bug, so you should be ok using that version for testing.

Comment 13 yanfu,wang 2010-07-27 06:10:22 UTC
(In reply to comment #11)
> Suresh J. summarized the reproducer this way:
> 
> I have a nice way of reliably reproducing the problem.
> 
> - create two small partitions and format (ext3)
> - mount first partition as /vol1 and mount the second as /vol1/vol2
>   inside the first
> - create files on both partitions (say 10 files)
> - export them
> - mount from the client, access them
> - the inode numbers will easily collide
> - umount them and the client will crash..
> 
> ...the key is that you need to export a share that contains the root of both
> filesystems so you can ensure that their inode numbers collide.
> 
> Since this is really a server-side problem, you'll also need to ensure that the
> samba version you're using doesn't contain the fix for this. I'll have to
> double check whether that's the case...    

thanks Jeff, I can reproduce using the above steps now.

reproduce on RHEL6.0-Beta1-5.0:
# uname -a
Linux dell-pem710-01.rhts.eng.bos.redhat.com 2.6.32-19.el6.i686 #1 SMP Tue Mar 9 18:10:40 EST 2010 i686 i686 i386 GNU/Linux

# rpm -qa|grep samba
samba-common-3.4.4-50.el6.i686
samba-3.4.4-50.el6.i686
samba-winbind-clients-3.4.4-50.el6.i686

# df -T
/dev/sda3     ext3     2064208   1014912    944440  52% /
tmpfs        tmpfs     3109140         0   3109140   0% /dev/shm
/dev/sda1     ext3      198337     25240    162857  14% /boot
/dev/sda2     ext3     5039616    141220   4642396   3% /test1
/dev/sda5     ext3    54427084    184276  51478060   1% /test2

# mount
/dev/sda3 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext3 (rw)
/dev/sda2 on /test1 type ext3 (rw)
/dev/sda5 on /test2 type ext3 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/sda2 on /vol1 type ext3 (rw)
/dev/sda5 on /vol1/vol2 type ext3 (rw)

then create files on both partitions (10 files).

# cat /etc/samba/smb.conf 
        security = share

        [vol1]
        comment = vol1
        path = /vol1
        public = yes
        writable = yes
        printable = no


        [vol2]
        comment = vol2
        path = /vol1/vol2
        public = yes
        writable = yes
        printable = no

start samba service and mounted from client.
# mount -t cifs //samba_server/vol1 /mnt/v1
# mount -t cifs //samba_server/vol2 /mnt/v2
# ls /mnt/v1
# ls /mnt/v2
# umount /mnt/v1
kernel crash and console down:
Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:------------[ cut here ]------------

Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:invalid opcode: 0000 [#1] SMP 

Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:last sysfs file: /sys/devices/system/cpu/cpu7/cache/index1/shared_cpu_map

Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:Stack:

Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:Call Trace:

Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:Code: 50 30 4c 8b 0a 31 d2 48 85 f6 74 04 48 8b 56 40 48 05 70 02 00 00 48 89 de 48 c7 c7 10 18 66 81 48 89 04 24 31 c0 e8 50 ae 34 00 <0f> 0b eb fe 0f 0b eb fe 55 48 89 e5 53 48 83 ec 08 0f 1f 44 00 

Message from syslogd@dell-pe2950-01 at Jul 27 01:08:51 ...
 kernel:Kernel panic - not syncing: Fatal exception

verified on RHEL6.0-Snapshot-7 with the same steps:
# uname -a
Linux dell-per805-01.rhts.eng.bos.redhat.com 2.6.32-44.el6.x86_64 #1 SMP Wed Jul 7 15:47:50 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
# rpm -qa|grep samba
samba-3.5.4-65.el6.x86_64
samba-common-3.5.4-65.el6.x86_64
samba-winbind-clients-3.5.4-65.el6.x86_64

[root@dell-pe2950-01 ~]# mount
...
//10.16.45.161/vol1 on /mnt/v1 type cifs (rw,mand)
//10.16.45.161/vol2 on /mnt/v2 type cifs (rw,mand)

[root@dell-pe2950-01 ~]# umount /mnt/v1
[root@dell-pe2950-01 ~]# umount /mnt/v2
[root@dell-pe2950-01 ~]# mount
/dev/mapper/vg_dellpe295001-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg_dellpe295001-lv_home on /home type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

umount successfully without kernel oops.

Comment 14 releng-rhel@redhat.com 2010-11-11 15:53:33 UTC
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.