Bug 919084 - cifs fails to properly parse SPNEGO blob from Windows 8 CIFS server
Summary: cifs fails to properly parse SPNEGO blob from Windows 8 CIFS server
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 18
Hardware: i686
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Jeff Layton
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-07 15:05 UTC by Jason Burgess
Modified: 2014-06-18 07:43 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-04-10 20:46:48 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
cifs: ignore everything in SPNEGO blob after mechTypes (3.29 KB, patch)
2013-03-11 13:50 UTC, Jeff Layton
no flags Details | Diff

Description Jason Burgess 2013-03-07 15:05:58 UTC
Description of problem:

Cannot mount CIFS share. Was working a week ago and nothing has changed on the target server. I run a nightly cronjob that executes "mount.cifs" with options to process files on an SMB share. One night last week, this suddenly stopped working but I am not sure what YUM update broke it.


Version-Release number of selected component (if applicable):


How reproducible:

Every time.

Steps to Reproduce:
1. /sbin/mount.cifs //server01/d /mnt/server01 -o user=processor,password=<removed>
  
Actual results:

mount.cifs kernel mount options: ip=192.168.1.10,unc=\\server01\d,user=processor,pass=********
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

Expected results:

Mount success. Stats available of remote file system with "df".

Additional info:

I've tried all variations of the mount.cifs command...interactive, etc.. no dice. Issue seems to be with the "password" option. If I remove it, at least I get prompted for the password... but typing it in and hitting enter gives the same result.

Comment 1 Jeff Layton 2013-03-07 15:44:16 UTC
If you run "dmesg" are there any helpful error messages there? Also, there was a problem a while ago where certain versions of /bin/mount would be confused by the user= syntax. If you change that option to read "username=processor" does it work?

Comment 2 Yan Li 2013-03-07 20:03:17 UTC
Same here. A freshly installed Fedora 18, but x86-64, installed all latest updates. I'm trying to connect to a directory shared from Windows 8 Pro (64-bit). I can rule out network problems since I can browse the shared directory from the Files GUI.

The only related dmesg output is like:
[   17.944445] fuse init (API version 7.20)
[   50.505722] FS-Cache: Loaded
[   50.508450] Key type dns_resolver registered
[   50.522786] FS-Cache: Netfs 'cifs' registered for caching
[   50.522915] Key type cifs.spnego registered
[   50.522921] Key type cifs.idmap registered
[   50.539209] CIFS VFS: cifs_mount failed w/return code = -22
[  318.192131] hrtimer: interrupt took 3450661 ns
[  480.670668] CIFS VFS: cifs_mount failed w/return code = -22
[  779.624553] CIFS VFS: cifs_mount failed w/return code = -22

I've tried both user= and username=. Same command line works immediately in RHEL6.

Comment 3 Jeff Layton 2013-03-07 20:08:17 UTC
Ok, can you turn up the debugging on the cifs kernel module using these instructions:

https://wiki.samba.org/index.php/LinuxCIFS_troubleshooting#Enabling_Debugging

...and then attempt the mount, and disable debugging after it fails. Then please post the dmesg output here.

Comment 4 Yan Li 2013-03-07 22:07:20 UTC
Thanks! Here's the output:

[ 8575.485703] fs/cifs/cifsfs.c: Devname: \\redacted\vb428 flags: 0
[ 8575.485720] fs/cifs/connect.c: Username: redacted
[ 8575.485724] fs/cifs/connect.c: file mode: 0x1ed  dir mode: 0x1ed
[ 8575.485799] fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 6 with uid: 0
[ 8575.485801] fs/cifs/connect.c: UNC: \\redacted\vb428
[ 8575.485812] fs/cifs/connect.c: Socket created
[ 8575.485813] fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x1b58
[ 8575.486665] fs/cifs/fscache.c: cifs_fscache_get_client_cookie: (0xffff880210f5f800/0xffff8801f4c370a0)
[ 8575.486669] fs/cifs/connect.c: CIFS VFS: in cifs_get_smb_ses as Xid: 7 with uid: 0
[ 8575.486670] fs/cifs/connect.c: Existing smb sess not found
[ 8575.486675] fs/cifs/cifssmb.c: secFlags 0x81
[ 8575.486677] fs/cifs/cifssmb.c: NTLMSSP only mechanism, enable extended security
[ 8575.486680] fs/cifs/transport.c: For smb_command 114
[ 8575.486681] fs/cifs/transport.c: Sending smb: smb_len=78
[ 8575.486684] fs/cifs/connect.c: Demultiplex PID: 2637
[ 8575.493267] fs/cifs/connect.c: RFC1002 header 0x195
[ 8575.493328] fs/cifs/transport.c: cifs_sync_mid_result: cmd=114 mid=1 state=4
[ 8575.493333] fs/cifs/cifssmb.c: Dialect: 2
[ 8575.493338] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
[ 8575.493340] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
[ 8575.493342] fs/cifs/asn1.c: cls = 0 con = 0 tag = 4 end = ffff880210fe0199 (0)
[ 8575.493344] fs/cifs/asn1.c: Exit 8 cls = 1 con = 0 tag = 14 end = ffff880210fe00d8 (253)
[ 8575.493346] fs/cifs/cifssmb.c: negprot rc -22
[ 8575.493349] fs/cifs/connect.c: CIFS VFS: leaving cifs_get_smb_ses (xid = 7) rc = -22
[ 8575.493356] fs/cifs/fscache.c: cifs_fscache_release_client_cookie: (0xffff880210f5f800/0xffff8801f4c370a0)
[ 8575.493587] fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 6) rc = -22
[ 8575.493589] CIFS VFS: cifs_mount failed w/return code = -22

Comment 5 Jeff Layton 2013-03-08 12:00:44 UTC
Ok, this almost certainly has nothing to do with cifs-utils, but is rather a kernel problem. The issue seems to be in the parsing of the SPNEGO blob sent by the server in the NEGOTIATE reply.

I'll have to see if I can reproduce this here...

Comment 6 Jeff Layton 2013-03-08 12:07:17 UTC
I suspect that the breakage is due to the change to making sec=ntlmssp the default auth method on cifs.ko. You may want to try adding sec=ntlmv2 or sec=ntlm in the mount options and see if that allows you to mount.

Comment 7 Jason Burgess 2013-03-08 13:37:07 UTC
Adding "sec=ntlm" allowed me to mount the filesystem:

/sbin/mount.cifs -v //server01/d /mnt/server01 -o username=processor,password=<removed>,sec=ntlm   
mount.cifs kernel mount options: ip=192.168.1.10,unc=\\server01\d,sec=ntlm,user=processor,pass=<removed>

# df -k
Filesystem      1K-blocks       Used  Available Use% Mounted on
//server01/d   871903776  108341096  763562680  13% /mnt/server01

Comment 8 Yan Li 2013-03-08 22:22:23 UTC
Same here, "sec=ntlm" is a viable workaround, "sec=ntlmv2" doesn't work for me.

I don't know the detail of those authentication methods, but it seems to me that we should either set ntlm as the default or fall back to it if some other (newer?) methods fail. But I only tested it against Windows 8 Pro so I don't know the best answer here. Just $0.02.

Comment 9 Jeff Layton 2013-03-10 01:48:36 UTC
(In reply to comment #8)
> Same here, "sec=ntlm" is a viable workaround, "sec=ntlmv2" doesn't work for
> me.
> 
> I don't know the detail of those authentication methods, but it seems to me
> that we should either set ntlm as the default or fall back to it if some
> other (newer?) methods fail. But I only tested it against Windows 8 Pro so I
> don't know the best answer here. Just $0.02.

The switch to use the more secure sec=ntlmssp by default (which uses NTLMv2, wrapped in a GSSAPI/SPNEGO wrapper) was made in the 3.8 kernel. sec=ntlm makes the client use NTLMv1 passwords. NTLMv1 passwords are now crackable on modern hardware in seconds.

So, I don't think it's a good idea to switch back to sec=ntlm by default, but we do need to figure out why sec=ntlmssp vs. win8 seems to be problematic in some cases.

Comment 10 Jeff Layton 2013-03-11 13:50:28 UTC
Created attachment 708387 [details]
cifs: ignore everything in SPNEGO blob after mechTypes

Ok, I was able to reproduce the problem vs. win8 and this patch seems to fix it for me. Does this work for you too?

Comment 11 Yan Li 2013-03-11 16:17:50 UTC
Thanks for the quick fix. It looks great to me. But sorry, I don't have time to compile it for now. If you have an RPM I'd be very happy to try it though.

Comment 12 Jeff Layton 2013-03-12 17:13:16 UTC
Ok, kernel building with this patch now:

    http://koji.fedoraproject.org/koji/taskinfo?taskID=5112672

...once it's done, can you test it and let me know whether it fixes the issue for you?

Comment 13 Yan Li 2013-03-12 17:34:12 UTC
Thanks for the new package! I've tested it. It works for me and fixed this bug.

Just nitpicking on the bug title, I'm testing against a Windows 8 Pro system, not a "Win8 server."

Comment 14 Jeff Layton 2013-03-12 18:17:50 UTC
Thanks for testing it. It would be helpful if you could reply to the patch that I cc'ed you on so that the CIFS maintainer knows that it has had at least one positive test.

Comment 15 Jeff Layton 2013-03-25 17:43:33 UTC
Patch has been merged into mainline and should be on its way to stable kernels as well. I'll close this once it's been merged into 3.8.z.

Comment 16 Jeff Layton 2013-04-10 20:46:48 UTC
This was merged in v3.8.5. Closing with a resolution ERRATA.


Note You need to log in before you can comment on or make changes to this bug.