Bug 1163927 - CIFS mounts fail with kernel-3.17.2-200.fc20.x86_64
Summary: CIFS mounts fail with kernel-3.17.2-200.fc20.x86_64
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 20
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Sachin Prabhu
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-11-13 17:27 UTC by Jerry
Modified: 2015-01-13 11:13 UTC (History)
13 users (show)

Fixed In Version: kernel-3.17.8-200.fc20
Clone Of:
Environment:
Last Closed: 2015-01-11 02:57:47 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Proposed patch (1.70 KB, application/mbox)
2014-11-25 15:35 UTC, Sachin Prabhu
no flags Details

Description Jerry 2014-11-13 17:27:54 UTC
CIFS mounts fail with kernel-3.17.2-200.fc20.x86_64.

A switch back to kernel-3.16.7-200.fc20.x86_64 solves the problem.

Steps to reproduce:

[root@myserver ~]# mount -v -t cifs -o rw,uid=myuser,gid=myuser,credentials=/etc/cred.txt //otherserver/mountdir /mnt/mymount
mount.cifs kernel mount options: ip=192.168.1.1,unc=\\otherserver\mountdir,uid=1000,gid=1000,user=nas,pass=********
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
[root@myserver ~]#

/var/log/messages:
Nov 13 12:15:43 myserver su: (to root) myuser on pts/0
Nov 13 12:15:56 myserver kernel: [   67.957537] FS-Cache: Loaded
Nov 13 12:15:56 myserver kernel: FS-Cache: Loaded
Nov 13 12:15:56 myserver kernel: [   67.959309] Key type dns_resolver registered
Nov 13 12:15:56 myserver kernel: Key type dns_resolver registered
Nov 13 12:15:57 myserver kernel: [   67.971944] FS-Cache: Netfs 'cifs' registered for caching
Nov 13 12:15:57 myserver kernel: [   67.972126] Key type cifs.spnego registered
Nov 13 12:15:57 myserver kernel: [   67.972136] Key type cifs.idmap registered
Nov 13 12:15:57 myserver kernel: FS-Cache: Netfs 'cifs' registered for caching
Nov 13 12:15:57 myserver kernel: Key type cifs.spnego registered
Nov 13 12:15:57 myserver kernel: Key type cifs.idmap registered
Nov 13 12:15:57 myserver kernel: [   68.005605] CIFS VFS: cifs_mount failed w/return code = -13
Nov 13 12:15:57 myserver kernel: CIFS VFS: cifs_mount failed w/return code = -13


Again, booting to the previous kernel gives a valid mount.  I did an rsync with about 6 files and they are all fine.

Thanks!

Comment 1 Thomas Jarosch 2014-11-19 09:56:24 UTC
Same here after the upgrade.

Samba version on the server side:
samba-3.0.33-3.39.el5_8.x86_64

Comment 2 Jerry 2014-11-21 16:21:27 UTC
Just tried this with kernel-3.17.3-200.fc20.x86_64 and got the same results.

[root@myserver ~]# mount -v -t cifs -o rw,uid=myuser,gid=myuser,credentials=/etc/cred.txt //otherserver/mountdir /mnt/mymount/
mount.cifs kernel mount options: ip=192.168.1.1,unc=\\otherserver\mountdir,uid=1000,gid=1000,user=nas,pass=********
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

/var/log/messages:
Nov 21 09:37:59 myserver su: (to root) myuser on pts/0
Nov 21 09:38:17 myserver fprintd: ** Message: No devices in use, exit
Nov 21 09:40:16 myserver kernel: [  262.544559] FS-Cache: Loaded
Nov 21 09:40:16 myserver kernel: FS-Cache: Loaded
Nov 21 09:40:16 myserver kernel: [  262.546073] Key type dns_resolver registered
Nov 21 09:40:16 myserver kernel: Key type dns_resolver registered
Nov 21 09:40:16 myserver kernel: [  262.555287] FS-Cache: Netfs 'cifs' registered for caching
Nov 21 09:40:16 myserver kernel: [  262.555483] Key type cifs.spnego registered
Nov 21 09:40:16 myserver kernel: [  262.555492] Key type cifs.idmap registered
Nov 21 09:40:16 myserver kernel: FS-Cache: Netfs 'cifs' registered for caching
Nov 21 09:40:16 myserver kernel: Key type cifs.spnego registered
Nov 21 09:40:16 myserver kernel: Key type cifs.idmap registered
Nov 21 09:40:16 myserver kernel: [  262.588450] CIFS VFS: cifs_mount failed w/return code = -13
Nov 21 09:40:16 myserver kernel: CIFS VFS: cifs_mount failed w/return code = -13

This is working in kernel-3.16.7-200.fc20.x86_64.

Samba is samba-4.1.12-5.fc20.x86_64.

Comment 3 Sachin Prabhu 2014-11-21 18:12:05 UTC
I have tried to reproduce the problem but am unable to do so. Can you please provide me with a tcpdump each of the working and not working cases so that I can compare to try and determine what changed.

Also cifs debug messages for the failed case could be useful. To generate those, run
# echo 7 >/proc/fs/cifs/cifsFYI
run the mount command
The debug messages will be available in the dmesg. Simply provide me with the output of the dmesg command.

Sachin Prabhu

Comment 4 Jerry 2014-11-21 18:54:25 UTC
(In reply to Sachin Prabhu from comment #3)
> I have tried to reproduce the problem but am unable to do so. Can you please
> provide me with a tcpdump each of the working and not working cases so that
> I can compare to try and determine what changed.
> 
I am not familiar with tcpdump.  Can you be more specific, i.e, tell me which commands you wish me to run?

Comment 5 Jerry 2014-11-21 22:10:22 UTC
For 3.17.3-200:

# echo 7 >/proc/fs/cifs/cifsFYI
-bash: /proc/fs/cifs/cifsFYI: No such file or directory

Comment 6 Jerry 2014-11-21 22:53:22 UTC
Created attachment 960002 [details]
Requested tcpdump pcap files and dmesg log

I think this is what you want.  I can try again if this is insufficient.

I used this page to get the tcpdump information: https://wiki.samba.org/index.php/LinuxCIFS_troubleshooting

Comment 7 Sachin Prabhu 2014-11-25 14:18:07 UTC
Hello Jerry,

Thanks for the tcpdumps. Looking at the traces, there appears to be a difference in the UIDs set in the header. This change was introduced by code to cleanup and simplify the authentication code.

I am trying to reproduce this on my test machines following which I'll write a patch to fix the issue.

Sachin Prabhu

Comment 8 Sachin Prabhu 2014-11-25 15:35:51 UTC
Created attachment 961260 [details]
Proposed patch

From edc0216ca23ed651ecfecd02d864e832b3f75e41 Mon Sep 17 00:00:00 2001
From: Sachin Prabhu <sprabhu>
Date: Tue, 25 Nov 2014 15:11:36 +0000
Subject: [PATCH] Set UID in sess_auth_rawntlmssp_authenticate too

A user complained that they were unable to login to their cifs share
after a kernel update. From the wiretrace we can see that the server
erturns different UIDs as response to NTLMSSP_NEGOTIATE and NTLMSSP_AUTH
phases.

With changes in the authentication code, we no longer set the
cifs_sess->Suid returned in response to the NTLM_AUTH phase. We continue
to use the UID sent in response to the NTLMSSP_NEGOTIATE phase.

This results in an access being denied when connecting to a tree.

Signed-off-by: Sachin Prabhu <sprabhu>

Comment 9 Sachin Prabhu 2014-11-25 20:52:13 UTC
Hello Jerry,

I was unable to reproduce it internally as you have on your setup. Can you please test this new kernel at
http://people.redhat.com/sprabhu/bz1167970/

This is the standard kernel-3.17.2-200 with the patch from c#8.

Can you give me a new tcpdump irrespective if it is working or no.
If it doesn't work, I can use it to determine if something else is needed.
If it does, I can use it to confirm that the UID issue was indeed the problem.

Sachin Prabhu

Comment 10 Jerry 2014-12-01 00:59:29 UTC
Created attachment 963059 [details]
tcpdump with patched kernel (successful mount)

Mounting my CIFS share was successful with this patched kernel.  Attached is the requested tcpdump.

Comment 11 Sachin Prabhu 2014-12-03 10:52:22 UTC
Thanks Jerry,

From the tcpdump I can confirm that the server first returns UID=100 in response to NTLMSSP_NEGOTIATE. Subsequently as a response to NTLMSSP_CHALLENGE, the UID is set to 101. The patched kernel uses this new UID and starts using it in the subsequent operations as expected. 

My next step is to push a patch upstream with this fix.

Sachin Prabhu

Comment 12 Sachin Prabhu 2014-12-05 12:16:49 UTC
Update:

1) Set tcpdumps to private.

2) Patch containing fix has been sent upstream
http://thread.gmane.org/gmane.linux.kernel.cifs/10392

Comment 13 Josh Boyer 2014-12-05 13:13:19 UTC
(In reply to Sachin Prabhu from comment #12)
> Update:
> 
> 1) Set tcpdumps to private.
> 
> 2) Patch containing fix has been sent upstream
> http://thread.gmane.org/gmane.linux.kernel.cifs/10392

We can carry this in Fedora until it works its way into upstream.  Is there a reason you didn't include it as a stable submission?

Comment 14 Sachin Prabhu 2014-12-05 15:39:56 UTC
Waiting for upstream to accept the patch. The patch sent upstream has been slightly modified.

Comment 15 Thomas Jarosch 2014-12-23 16:20:19 UTC
+1 for having it in the Fedora kernel until it's upstream :)

Comment 16 Josh Boyer 2015-01-06 13:58:45 UTC
Fixed in Fedora git.

Comment 17 Fedora Update System 2015-01-09 13:09:52 UTC
kernel-3.17.8-300.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/kernel-3.17.8-300.fc21

Comment 18 Fedora Update System 2015-01-09 13:10:46 UTC
kernel-3.17.8-200.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/kernel-3.17.8-200.fc20

Comment 19 Fedora Update System 2015-01-10 03:00:09 UTC
Package kernel-3.17.8-200.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-3.17.8-200.fc20'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-0515/kernel-3.17.8-200.fc20
then log in and leave karma (feedback).

Comment 20 Fedora Update System 2015-01-11 02:57:47 UTC
kernel-3.17.8-300.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2015-01-13 00:05:30 UTC
kernel-3.17.8-200.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 22 Gerd v. Egidy 2015-01-13 11:13:45 UTC
3.17.8-200.fc20 has fixed the problem for me. Thank you.


Note You need to log in before you can comment on or make changes to this bug.