Red Hat Bugzilla – Bug 919084
cifs fails to properly parse SPNEGO blob from Windows 8 CIFS server
Last modified: 2014-06-18 03:43:06 EDT
Description of problem:
Cannot mount CIFS share. Was working a week ago and nothing has changed on the target server. I run a nightly cronjob that executes "mount.cifs" with options to process files on an SMB share. One night last week, this suddenly stopped working but I am not sure what YUM update broke it.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. /sbin/mount.cifs //server01/d /mnt/server01 -o user=processor,password=<removed>
mount.cifs kernel mount options: ip=192.168.1.10,unc=\\server01\d,user=processor,pass=********
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
Mount success. Stats available of remote file system with "df".
I've tried all variations of the mount.cifs command...interactive, etc.. no dice. Issue seems to be with the "password" option. If I remove it, at least I get prompted for the password... but typing it in and hitting enter gives the same result.
If you run "dmesg" are there any helpful error messages there? Also, there was a problem a while ago where certain versions of /bin/mount would be confused by the user= syntax. If you change that option to read "username=processor" does it work?
Same here. A freshly installed Fedora 18, but x86-64, installed all latest updates. I'm trying to connect to a directory shared from Windows 8 Pro (64-bit). I can rule out network problems since I can browse the shared directory from the Files GUI.
The only related dmesg output is like:
[ 17.944445] fuse init (API version 7.20)
[ 50.505722] FS-Cache: Loaded
[ 50.508450] Key type dns_resolver registered
[ 50.522786] FS-Cache: Netfs 'cifs' registered for caching
[ 50.522915] Key type cifs.spnego registered
[ 50.522921] Key type cifs.idmap registered
[ 50.539209] CIFS VFS: cifs_mount failed w/return code = -22
[ 318.192131] hrtimer: interrupt took 3450661 ns
[ 480.670668] CIFS VFS: cifs_mount failed w/return code = -22
[ 779.624553] CIFS VFS: cifs_mount failed w/return code = -22
I've tried both user= and username=. Same command line works immediately in RHEL6.
Ok, can you turn up the debugging on the cifs kernel module using these instructions:
...and then attempt the mount, and disable debugging after it fails. Then please post the dmesg output here.
Thanks! Here's the output:
[ 8575.485703] fs/cifs/cifsfs.c: Devname: \\redacted\vb428 flags: 0
[ 8575.485720] fs/cifs/connect.c: Username: redacted
[ 8575.485724] fs/cifs/connect.c: file mode: 0x1ed dir mode: 0x1ed
[ 8575.485799] fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 6 with uid: 0
[ 8575.485801] fs/cifs/connect.c: UNC: \\redacted\vb428
[ 8575.485812] fs/cifs/connect.c: Socket created
[ 8575.485813] fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x1b58
[ 8575.486665] fs/cifs/fscache.c: cifs_fscache_get_client_cookie: (0xffff880210f5f800/0xffff8801f4c370a0)
[ 8575.486669] fs/cifs/connect.c: CIFS VFS: in cifs_get_smb_ses as Xid: 7 with uid: 0
[ 8575.486670] fs/cifs/connect.c: Existing smb sess not found
[ 8575.486675] fs/cifs/cifssmb.c: secFlags 0x81
[ 8575.486677] fs/cifs/cifssmb.c: NTLMSSP only mechanism, enable extended security
[ 8575.486680] fs/cifs/transport.c: For smb_command 114
[ 8575.486681] fs/cifs/transport.c: Sending smb: smb_len=78
[ 8575.486684] fs/cifs/connect.c: Demultiplex PID: 2637
[ 8575.493267] fs/cifs/connect.c: RFC1002 header 0x195
[ 8575.493328] fs/cifs/transport.c: cifs_sync_mid_result: cmd=114 mid=1 state=4
[ 8575.493333] fs/cifs/cifssmb.c: Dialect: 2
[ 8575.493338] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
[ 8575.493340] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
[ 8575.493342] fs/cifs/asn1.c: cls = 0 con = 0 tag = 4 end = ffff880210fe0199 (0)
[ 8575.493344] fs/cifs/asn1.c: Exit 8 cls = 1 con = 0 tag = 14 end = ffff880210fe00d8 (253)
[ 8575.493346] fs/cifs/cifssmb.c: negprot rc -22
[ 8575.493349] fs/cifs/connect.c: CIFS VFS: leaving cifs_get_smb_ses (xid = 7) rc = -22
[ 8575.493356] fs/cifs/fscache.c: cifs_fscache_release_client_cookie: (0xffff880210f5f800/0xffff8801f4c370a0)
[ 8575.493587] fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 6) rc = -22
[ 8575.493589] CIFS VFS: cifs_mount failed w/return code = -22
Ok, this almost certainly has nothing to do with cifs-utils, but is rather a kernel problem. The issue seems to be in the parsing of the SPNEGO blob sent by the server in the NEGOTIATE reply.
I'll have to see if I can reproduce this here...
I suspect that the breakage is due to the change to making sec=ntlmssp the default auth method on cifs.ko. You may want to try adding sec=ntlmv2 or sec=ntlm in the mount options and see if that allows you to mount.
Adding "sec=ntlm" allowed me to mount the filesystem:
/sbin/mount.cifs -v //server01/d /mnt/server01 -o username=processor,password=<removed>,sec=ntlm
mount.cifs kernel mount options: ip=192.168.1.10,unc=\\server01\d,sec=ntlm,user=processor,pass=<removed>
# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
//server01/d 871903776 108341096 763562680 13% /mnt/server01
Same here, "sec=ntlm" is a viable workaround, "sec=ntlmv2" doesn't work for me.
I don't know the detail of those authentication methods, but it seems to me that we should either set ntlm as the default or fall back to it if some other (newer?) methods fail. But I only tested it against Windows 8 Pro so I don't know the best answer here. Just $0.02.
(In reply to comment #8)
> Same here, "sec=ntlm" is a viable workaround, "sec=ntlmv2" doesn't work for
> I don't know the detail of those authentication methods, but it seems to me
> that we should either set ntlm as the default or fall back to it if some
> other (newer?) methods fail. But I only tested it against Windows 8 Pro so I
> don't know the best answer here. Just $0.02.
The switch to use the more secure sec=ntlmssp by default (which uses NTLMv2, wrapped in a GSSAPI/SPNEGO wrapper) was made in the 3.8 kernel. sec=ntlm makes the client use NTLMv1 passwords. NTLMv1 passwords are now crackable on modern hardware in seconds.
So, I don't think it's a good idea to switch back to sec=ntlm by default, but we do need to figure out why sec=ntlmssp vs. win8 seems to be problematic in some cases.
Created attachment 708387 [details]
cifs: ignore everything in SPNEGO blob after mechTypes
Ok, I was able to reproduce the problem vs. win8 and this patch seems to fix it for me. Does this work for you too?
Thanks for the quick fix. It looks great to me. But sorry, I don't have time to compile it for now. If you have an RPM I'd be very happy to try it though.
Ok, kernel building with this patch now:
...once it's done, can you test it and let me know whether it fixes the issue for you?
Thanks for the new package! I've tested it. It works for me and fixed this bug.
Just nitpicking on the bug title, I'm testing against a Windows 8 Pro system, not a "Win8 server."
Thanks for testing it. It would be helpful if you could reply to the patch that I cc'ed you on so that the CIFS maintainer knows that it has had at least one positive test.
Patch has been merged into mainline and should be on its way to stable kernels as well. I'll close this once it's been merged into 3.8.z.
This was merged in v3.8.5. Closing with a resolution ERRATA.