Bug 728796

It's difficult to know without more detail.

What user is running the mount.cifs command?

What error message do you get back when you try to mount the share by hand using the above mount.cifs command?

Does anything interesting show up in dmesg when you run that command?

This is all running as root, from a bash shell. Connecting with domain credentials failed:

DOM=ams2
MACH=stan3
USR=administrator

/sbin/mount.cifs //$MACH/c$ /mnt/$MACH -o "username=$USR,dom=$DOM,password=$PASS,ip=$IP"
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

[3101329.432779] Status code returned 0xc000006d NT_STATUS_LOGON_FAILURE
[3101329.432786] CIFS VFS: Send error in SessSetup = -13
[3101329.432798] CIFS VFS: cifs_mount failed w/return code = -13


Connecting with XP local administrator credentials works:

/sbin/mount.cifs //$MACH/c$ /mnt/$MACH -o "username=$USR,dom=$MACH,password=$PASS,ip=$IP"
df | grep mnt
//stan3/c$            74413076  40611700  33801376  55% /mnt/stan3

Does it work if you add 'sec=ntlmssp' on the command line? Also, what kernel are you testing this on?

/sbin/mount.cifs //$MACH/c$ /mnt/$MACH -o "username=$USR,dom=$DOM,password=$PASS,ip=$IP,sec=ntlmssp"
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

[25413.166876] Status code returned 0xc000006d NT_STATUS_LOGON_FAILURE
[25413.166882] CIFS VFS: Send error in SessSetup = -13
[25413.171047] CIFS VFS: cifs_mount failed w/return code = -13

uname -a
Linux mail3.example.com 2.6.40.3-0.fc15.x86_64 #1 SMP Tue Aug 16 04:10:59 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

Do you still have access to the rhel4 machine? If so, it would be helpful to know whether f15 is passing the password to the kernel incorrectly. One way to tell that would be to strace the mount on both machines with something like this:

# strace -f -e mount -s 4096 /sbin/mount.cifs //$MACH/c$

...and them compare whether the passwords being passed to the kernel are equivalent for both. This would help narrow down whether the problem is in the userspace mount helper, or the kernel.

No, the original centos4 box is gone, but I still have another client with the same setup (centos4 samba domain controller, xp joined to the domain, mount the xp c$ share from the domain controller using the domain credentials working).

So the two strace will be against two different xp machines in different domains. Does that still help?

Probably not, unless they both use the same password. What we need to determine is whether the password is being passed to the kernel correctly in both cases.

Temporarily changing the admin domain password on both systems to abc123

From the F15 system:

strace -f -e mount -s 4096 /sbin/mount.cifs //$MACH/c$ /mnt/$MACH -o "username=$USR,dom=$DOM,password=$PASS,ip=$IP"
Process 17728 attached
Process 17727 suspended
Process 17727 resumed
Process 17728 detached
--- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=17728, si_status=0, si_utime=0, si_stime=0} (Child exited) ---
mount("//stan3/c$", ".", "cifs", 0, "ip=192.168.2.99,unc=\\\\stan3\\c$,,ver=1,user=administrator,domain=ams2,pass=abc123") = -1 EACCES (Permission denied)
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)


From a different centos4 network:

strace -f -e mount -s 4096  /sbin/mount.cifs //$NTBIOS/c$ /mnt/$MOUNT -o "username=$USR,dom=$DOM,password=$PASS,ip=$NTDNS"
mount("//dell2400/c$", ".", "cifs", MS_MANDLOCK, "unc=//dell2400\\c$,ver=1,username=administrator,dom=510sg,ip=192.168.4.11,pass=abc123") = 0

Sorry for the delay in looking at this...

Yes, it looks like the correct pw is being passed down to the kernel in both cases. At this point, we'd probably need to look at network traces between the client and server in both cases. Ideally, you'd have them connect to the same
server to ensure that that isn't a factor. Instructions on how to do that are
here:

    https://wiki.samba.org/index.php/LinuxCIFS_troubleshooting


I will note this though:

On the f15 box, you're using "domain=ams2" and on the other one you're using "domain=510sg". If these machines use the same PDC, then why are you using a different domain name here?

Yes, those were two different servers

f15 client connecting to XP machine joined to domain ams2 with pdc on the same f15 machine as the client.

centos4 client connecting to XP machine joined to domain 510sg with pdc on the same centos4 machine as the client.

However, the last centos4 machine has now been upgraded to centos6 and the mount with domain credentials now fails. So I no longer have a test system where the mount with domain credentials works.

I can get a network trace on either the F15 system or the centos6 system. Any preferences?

Either one is fine. Let's just start with any capture that shows the failure
occurring.

Created attachment 564022 [details]
tcpdump of mount failure

That tcpdump is from the centos6 system.

I suppose I'll need to know what key you used to encrypt this?

Should be to your most recent key, RSA key 19568215, created: 2011-10-12

Ok, it's definitely falling down in the session setup code. The connection going
in the other direction seems to have worked fine though. Some questions:

Are you able to mount if you mount with sec=ntlmssp?

What kernel is the client running in this case?

Adding sec=ntlmssp to the options does not change the behavior - mount still fails.

2.6.32-220.4.2.el6.x86_64 #1 SMP Tue Feb 14 04:00:16 GMT 2012 x86_64 x86_64 x86_64 GNU/Linux

Ok, the capture shows two different machines. Each machine is apparently a
server as well as a client, and they are attempting to connect to each other's
servers. One is:

Native OS: Linux version 2.6.32-220.4.2.el6.x86_64
Native LAN Manager: CIFS VFS Client for Linux

...and the other is:

Native OS: Windows 2002 Service Pack 3 2600
Native LAN Manager: Windows 2002 5.1

The Linux client is using "normal" NTLMv1 auth, and is attempting to authenticate
with:

username = administrator
domain = 510sg

The windows machine is using NTLMSSP auth. Interestingly, the windows machine
sets the domain and username fields to NULL in the NTLMSSP auth request. It
doesn't appear to be authenticating as a user at all. It does however set the
"host name" field. I'm going to assume that this machine is doing anonymous
auth?

Unfortunately, I can't tell much from this info. I really need to see a
set of traces that allows me to compare the working and non-working cases
to the same server. Anything else leaves too many variables for me to know
where to start troubleshooting.

If you can get me that, I'll be happy to take a look.

Shirish, does this look at all familiar to you?

Jeff, looking into it.

Contents of smb.conf file could be different between centos4
and f15 (and centos6) and may offer some clue as to why the 
failure on f15 and centos6.

Without and with the patch cfbd6f84c2e26c13ded16b6bb0871edb7d75974f
applied to cifs client, does sec=ntlmv2 work? 
Perhaps Samba server does not allow ntlmv1 (current default or stated 
explicitly using mount option sec=ntlm) authentication mechanism!

This could be a case of a failing pass-through authentication.
Windows XP box just passes the (24 bytes) blob to the
Domain Controller (which is Samba server on f15 in this case)
for authentication.

In case of extended security authentication (e.g. sec=ntlmssp)
this blob needs to be LMv2 response (which is 24 bytes) but
cifs client does not generate and send along LMv2 response in
type 3 (auth) phase of the session setup (LM response is empty).
I had coded LMv2 response once, will see if I have that code
saved somewhere.  If not, will generate a patch that adds 
LMv2 response.

But meanwhile, we can see if smbclient from the f15 box succeeds
in authentication as a domain user.  smbclient can try employing
any/all of ntlm or ntlmv2 or spnego/ntlmssp authentication
mechanisms by choosing so in smb.conf file.

Still working on figuring out how to access tcpdump attached
as .pgp file.
So I am not sure whether Windows XP sends ntlm or ntlmv2 blob
within ntlmssp auth packet sent to the Samba server and whether it
also includes anything (24 bytes) in LM field of that auth packet!

Created attachment 564534 [details]
patch to add lmv2 blob to ntlmssp type 3 message

patch to add lmv2 auth blob to in ntlmssp type 3 response.
To be used with sec=ntlmssp/i mount option.

Carl, have you had a chance to test Shirish's patch? Did it help?

I no longer have any XP machines here, or at the client, so I cannot test that patch. The migration to Linux is now complete.

Ok then, at this point. I'm going to go ahead and close this with a resolution of INSUFFICIENT_DATA since we don't know enough about the problem to say whether it's fixed or not.

Please reopen if you or someone else finds a way to reproduce it and can test Shirish's patch.