Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 574750

Summary: mounting CIFS shares from AD using kerberos does not work
Product: Red Hat Enterprise Linux 5 Reporter: Harald Milz <harald.milz>
Component: samba3xAssignee: Jeff Layton <jlayton>
Status: CLOSED NOTABUG QA Contact: BaseOS QE - Apps <qe-baseos-apps>
Severity: medium Docs Contact:
Priority: low    
Version: 5.5CC: jlayton, mkranz, nalin, rob.townley, steved, tengel
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-31 10:46:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Harald Milz 2010-03-18 13:15:24 UTC
Description of problem:

when mounting shares from a AD member server using "mount.cifs -sec=krb5i", mounting fails with "error 126 Required key not available" _except_ if "uid=0" is added. If "uid=0" is added, mounting works, but the mounted share now belongs to root (not an option ;-) 


Version-Release number of selected component (if applicable):

samba3x-client-3.3.8-0.50


How reproducible:

always


Steps to Reproduce:

1. let the workstation become an AD member
2. log on as AD user
3. mount.cifs //Server/share /mnt/point -o sec=krb5i
  
Actual results:

error 126 Required key not available


Expected results:

Share should be mounted


Additional info:

Authentication is set up with pam_winbind.so krb5_auth krb5_ccache_type=FILE, and works fine as such. The user gets her or his TGT from the KDC. 

Realm names are masked for customer's privacy. 

The relevant portion of /etc/request-key.conf reads:

create       cifs.spnego    * * /usr/sbin/cifs.upcall %k
create       dns_resolver   * * /usr/sbin/cifs.upcall %k


First mount attempt with no uid= option: 

Mar 18 09:48:34 fwuserpc4 cifs.upcall: key description:
cifs.spnego;0;10513;3f000000;ver=0x2;host=FS0Z0LLQ;ip4=10.63.84.228;sec=mskrb5;uid=0x3d13;user=user04
Mar 18 09:48:34 fwuserpc4 cifs.upcall: find_krb5_cc: considering /tmp/krb5cc_0
Mar 18 09:48:34 fwuserpc4 cifs.upcall: find_krb5_cc: /tmp/krb5cc_0 is owned by
0, not 15635
Mar 18 09:48:34 fwuserpc4 cifs.upcall: find_krb5_cc: considering
/tmp/krb5cc_15635
Mar 18 09:48:34 fwuserpc4 cifs.upcall: find_krb5_cc: FILE:/tmp/krb5cc_15635 is
valid ccache
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for cifs/FS0Z0LLQ
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: failed to obtain
service ticket (-1765328160)
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for host/FS0Z0LLQ
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: failed to obtain
service ticket (-1765328160)

Second mount attempt with uid=0:

Mar 18 09:49:09 fwuserpc4 cifs.upcall: key description:
cifs.spnego;0;10513;3f000000;ver=0x2;host=FS0Z0LLQ;ip4=10.63.84.228;sec=mskrb5;uid=0x0;user=user04
Mar 18 09:49:09 fwuserpc4 cifs.upcall: find_krb5_cc: considering /tmp/krb5cc_0
Mar 18 09:49:09 fwuserpc4 cifs.upcall: find_krb5_cc: FILE:/tmp/krb5cc_0 is
valid ccache
Mar 18 09:49:09 fwuserpc4 cifs.upcall: find_krb5_cc: considering
/tmp/krb5cc_15635
Mar 18 09:49:09 fwuserpc4 cifs.upcall: find_krb5_cc: /tmp/krb5cc_15635 is owned
by 15635, not 0
Mar 18 09:49:09 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for cifs/FS0Z0LLQ
Mar 18 09:49:09 fwuserpc4 cifs.upcall: handle_krb5_mech: obtained service
ticket

(Is it okay that keyUID in the key description is "0" in both cases???) 

Mounting works, and the mount point and everything below belongs to root. After
that, *root* has a cifs/ service principal, and user04 has not: 

[root@fwuserpc4 cifs]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: superuser.CUSTOMER.DE

Valid starting     Expires            Service principal
03/18/10 09:35:28  03/18/10 21:35:34 
krbtgt/R8611.ADS.CUSTOMER.DE.CUSTOMER.DE
        renew until 03/19/10 09:35:28
03/18/10 09:40:22  03/18/10 21:35:34  cifs/FS0Z0LLQ.CUSTOMER.DE
        renew until 03/19/10 09:35:28


Kerberos 4 ticket cache: /tmp/tkt0
klist: You have no tickets cached

[user04@fwuserpc4 ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_15635
Default principal: user04.CUSTOMER.DE

Valid starting     Expires            Service principal
03/18/10 09:41:16  03/18/10 21:41:16 
krbtgt/R8611.ADS.CUSTOMER.DE.CUSTOMER.DE
        renew until 03/25/10 09:36:38


Kerberos 4 ticket cache: /tmp/tkt15635
klist: You have no tickets cached



The relevant part of cifsFYI = 7:

First mount attempt w/o uid=

 fs/cifs/cifsfs.c: Devname: //FS0Z0LLQ/Users/user04 flags: 64 
 fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 21 with uid: 0
 fs/cifs/connect.c: prefix path /user04
 fs/cifs/connect.c: Username: user04
 fs/cifs/connect.c: UNC: \\FS0Z0LLQ\Users ip: 10.63.84.228
 fs/cifs/connect.c: Socket created
 fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x1b58
 fs/cifs/connect.c: Existing smb sess not found
 fs/cifs/connect.c: Demultiplex PID: 4165
 fs/cifs/cifssmb.c: secFlags 0x1009
 fs/cifs/cifssmb.c: Kerberos only mechanism, enable extended security
 fs/cifs/transport.c: For smb_command 114
 fs/cifs/transport.c: Sending smb:  total_len 82
 fs/cifs/connect.c: rfc1002 length 0xbf
 fs/cifs/cifssmb.c: Dialect: 2
 fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2 0x348 0xbb92
 fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2 0x348 0x1bb92
 fs/cifs/asn1.c: OID len = 8 oid = 0x1 0x2 0x348 0x1bb92
 fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
 fs/cifs/asn1.c: Need to call asn1_octets_decode() function for
fs0z0llq$@R8611.ADS.CUSTOMER.DE
 fs/cifs/cifssmb.c: Must sign - secFlags 0x1009
 fs/cifs/cifssmb.c: negprot rc 0
 fs/cifs/connect.c: Security Mode: 0xf Capabilities: 0x8001f3fd TimeAdjust:
-3600
 fs/cifs/sess.c: sess setup type 7
 fs/cifs/cifs_spnego.c: key description =
ver=0x2;host=FS0Z0LLQ;ip4=10.63.84.228;sec=mskrb5;uid=0x3d13;user=user04
 fs/cifs/sess.c: ssetup freeing small buf f23ae200
 CIFS VFS: Send error in SessSetup = -126
 fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 21) rc = -126
 CIFS VFS: cifs_mount failed w/return code = -126

Second mount attempt w/ uid=0:

 fs/cifs/cifsfs.c: Devname: //FS0Z0LLQ/Users/user04 flags: 64 
 fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 22 with uid: 0
 fs/cifs/connect.c: prefix path /user04
 fs/cifs/connect.c: Username: user04
 fs/cifs/connect.c: UNC: \\FS0Z0LLQ\Users ip: 10.63.84.228
 fs/cifs/connect.c: Socket created
 fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x1b58
 fs/cifs/connect.c: Existing smb sess not found
 fs/cifs/connect.c: Demultiplex PID: 4170
 fs/cifs/cifssmb.c: secFlags 0x1009
 fs/cifs/cifssmb.c: Kerberos only mechanism, enable extended security
 fs/cifs/transport.c: For smb_command 114
 fs/cifs/transport.c: Sending smb:  total_len 82
 fs/cifs/connect.c: rfc1002 length 0xbf
 fs/cifs/cifssmb.c: Dialect: 2
 fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2 0x348 0xbb92
 fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2 0x348 0x1bb92
 fs/cifs/asn1.c: OID len = 8 oid = 0x1 0x2 0x348 0x1bb92
 fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
 fs/cifs/asn1.c: Need to call asn1_octets_decode() function for
fs0z0llq$@R8611.ADS.CUSTOMER.DE
 fs/cifs/cifssmb.c: Must sign - secFlags 0x1009
 fs/cifs/cifssmb.c: negprot rc 0
 fs/cifs/connect.c: Security Mode: 0xf Capabilities: 0x8001f3fd TimeAdjust:
-3600
 fs/cifs/sess.c: sess setup type 7
 fs/cifs/cifs_spnego.c: key description =
ver=0x2;host=FS0Z0LLQ;ip4=10.63.84.228;sec=mskrb5;uid=0x0;user=user04
 fs/cifs/transport.c: For smb_command 115
 fs/cifs/transport.c: Sending smb:  total_len 3630
 fs/cifs/connect.c: rfc1002 length 0xd5
 fs/cifs/misc.c: Null buffer passed to cifs_small_buf_release
 fs/cifs/sess.c: ssetup rc from sendrecv2 is 0
 fs/cifs/sess.c: UID = 53248 
 fs/cifs/sess.c: bleft 139
 fs/cifs/sess.c: serverOS=Windows Server 2003 R2 3790 Service Pack 2
 fs/cifs/sess.c: serverNOS=Windows Server 2003 R2 5.2
 fs/cifs/sess.c: ssetup freeing small buf f23ae740
 fs/cifs/connect.c: CIFS Session Established successfully
 fs/cifs/connect.c: file mode: 0x1b0  dir mode: 0x1f8
 fs/cifs/transport.c: For smb_command 117
 fs/cifs/transport.c: Sending smb:  total_len 88
 fs/cifs/connect.c: rfc1002 length 0x42
 fs/cifs/connect.c: disk share connection
 fs/cifs/connect.c: nativeFileSystem=NTFS
 fs/cifs/connect.c: Tcon flags: 0x1 
 fs/cifs/connect.c: CIFS Tcon rc = 0
 fs/cifs/cifssmb.c: In QFSDeviceInfo
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 72
 fs/cifs/connect.c: rfc1002 length 0x44
 fs/cifs/cifssmb.c: In QFSAttributeInfo
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 72
 fs/cifs/connect.c: rfc1002 length 0x50
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0xac
 fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 22) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_iget as Xid: 23 with uid: 0
 fs/cifs/inode.c: Getting info on \user04
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0xac
 fs/cifs/inode.c: Old time 0
 fs/cifs/inode.c: New time 765739
SELinux: initialized (dev cifs, type cifs), uses genfs_contexts
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 24 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 766760
 fs/cifs/inode.c: Getting info on \user04
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0xac
 fs/cifs/inode.c: Old time 765739
 fs/cifs/inode.c: New time 766761
 fs/cifs/inode.c: cifs_revalidate - inode unchanged
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 24) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 25 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 766763
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 25) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 26 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 766763
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 26) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 27 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 822249
 fs/cifs/inode.c: Getting info on \user04
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0xac
 fs/cifs/inode.c: Old time 766761
 fs/cifs/inode.c: New time 822250
 fs/cifs/inode.c: cifs_revalidate - inode unchanged
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 27) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 28 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 822726
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 28) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 29 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 822864
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 29) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 30 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 822903
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 30) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 31 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 1092294
 fs/cifs/inode.c: Getting info on \user04
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0xac
 fs/cifs/inode.c: Old time 822250
 fs/cifs/inode.c: New time 1092295
 fs/cifs/inode.c: cifs_revalidate - inode unchanged
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 31) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_revalidate as Xid: 32 with uid: 15635
 fs/cifs/inode.c: Revalidate: \user04 inode 0xf1363e30 count 1 dentry:
0xf10ad4d0 d_time 0 jiffies 1092299
 fs/cifs/inode.c: CIFS VFS: leaving cifs_revalidate (xid = 32) rc = 0



Apparently, cifs.upcall fails when getting a cifs/ service pricipal for users
in the AD. BTW when I attempt a mount with another local uid != 0, cifs.upcall
fails with:

Mar 18 10:04:55 fwuserpc4 cifs.upcall: key description:
cifs.spnego;0;10513;3f000000;ver=0x2;host=FS0Z0LLQ;ip4=10.63.84.228;sec=mskrb5;uid=0x1f4;user=user04
Mar 18 10:04:55 fwuserpc4 cifs.upcall: find_krb5_cc: considering /tmp/krb5cc_0
Mar 18 10:04:55 fwuserpc4 cifs.upcall: find_krb5_cc: /tmp/krb5cc_0 is owned by
0, not 500
Mar 18 10:04:55 fwuserpc4 cifs.upcall: find_krb5_cc: considering
/tmp/krb5cc_15635
Mar 18 10:04:55 fwuserpc4 cifs.upcall: find_krb5_cc: /tmp/krb5cc_15635 is owned
by 15635, not 500
Mar 18 10:04:55 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for cifs/FS0Z0LLQ
Mar 18 10:04:55 fwuserpc4 cifs.upcall: handle_krb5_mech: failed to obtain
service ticket (-1765328160)
Mar 18 10:04:55 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for host/FS0Z0LLQ
Mar 18 10:04:55 fwuserpc4 cifs.upcall: handle_krb5_mech: failed to obtain
service ticket (-1765328160)

Interestingly, in this case it tries to find a ticket cache for uid 500. Again,
the mount command was invoked as user user04. Shouldn't cifs.upcall attempt to
use the invoking user's ticket cache? 

I also think the mount.cifs (cifs.upcall?) behaviour re uid= is wrong. Methinks
the option should only be used to explicitly set the ownership of the mounted
share, and not interfere with getting a service ticket.    

I rechecked what mount.cifs sends to the kernel - the uid= option is passed
correctly (filled with the default uid=current_user if the user doesn't provide
any). cifs.upcall apparently does the right thing, looking at its source code.
It seems that keyctl_describe_alloc() returns a key description with uid=0 (if
set by the user), which makes cifs.upcall try to use krb5cc_0. 

As is happens, the root user has a domain admin TGT (just having joined the
domain with it), and the $USER has a TGT with no special rights. Could that be
the reason why the root's TGT does get a cifs/ service ticket and the $USER's
does not? Is it that trivial? Needless to say, we checked that the very same
user logged into a Vista box does get his shares, so we think it can't be the
user's rights in AD. (BTW if we give the user04 domain admin rights, mounting fails the same way.)

We also traced the network from the Windows 2003 AD server. It appears that

- when mounting with uid=0, the ticket request is received correctly by the KDC, and answered. 

- when using smbclient -k, same. 

- when using mount.cifs without uid=0, i.e. the defaults, no ticket request is received by the KDC at all, hence no answer from the KDC! 

The last one points to the routine handle_krb5_mech() in cifs.upcall (see the logs above). It makes no difference if I use the plain hostname for mounting or the FQDN (FS0Z0LLQ vs. FS0Z0LLQ.R8611.ADS.CUSTOMER.DE).

Comment 1 Jeff Layton 2010-03-22 13:39:41 UTC
Thanks for the detailed writeup. The issue is really here I think:

Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for cifs/FS0Z0LLQ
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: failed to obtain
service ticket (-1765328160)
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: getting service ticket
for host/FS0Z0LLQ
Mar 18 09:48:34 fwuserpc4 cifs.upcall: handle_krb5_mech: failed to obtain
service ticket (-1765328160)

...for some reason, acquiring the service ticket is failing. Unfortunately, we don't have a lot of info to go on here. One thing that might be helpful actually is to try pulling down the latest cifs.upcall sources from the upstream cifs-utils repo:

    http://samba.org/linux-cifs/cifs-utils/

...there were a number of changes that needed to be done when those tools were split from the samba tree, and in the process some more detailed logging of errors was added. That might give us a little more to go on.

> I also think the mount.cifs (cifs.upcall?) behaviour re uid= is wrong. 
> Methinks the option should only be used to explicitly set the ownership
> of the mounted share, and not interfere with getting a service ticket.    

Agreed. This area *really* needs to be reworked. The tricky part is doing so without breaking people who are dependent on the existing behavior. The best thing to get this changed might be to write that portion of this up and post it to the upstream linux-cifs-client list:

    https://lists.samba.org/mailman/listinfo/linux-cifs-client

...then we can discuss it within full view of the upstream community.

Comment 2 Harald Milz 2010-03-25 07:47:21 UTC
Hi, I just tested cifs-utils-4.1, and, in a nutshell, the macro behaviour (from the end user's perspective) is unchanged.

Since we need to run usermounts from the logon script, and may have a couple hundred users on one machine (running NoMachine sessions), I need to compile mount.cifs with CIFS_DISABLE_SETUID_CHECK and CIFS_LEGACY_SETUID_CHECK enabled. The latter leads to a compile error:

mount.cifs.c: In function ‘check_mountpoint’:
mount.cifs.c:135: error: ‘statbuf’ undeclared (first use in this function)
mount.cifs.c:135: error: (Each undeclared identifier is reported only once
mount.cifs.c:135: error: for each function it appears in.)
make[1]: *** [mount.cifs.o] Error 1

I had to comment the two if clauses, then it compiled.

cifs.upcall when invoking the mount without uid=0 now logs:

Mar 25 08:31:01 fwaocpc4 cifs.upcall: key description:
cifs.spnego;0;10513;3f000000;ver=0x2;host=FS0Z0LLQ;ip4=10.63.84.228;sec=mskrb5;uid=0x3d13;user=aoc04
Mar 25 08:31:01 fwaocpc4 cifs.upcall: find_krb5_cc: considering /tmp/krb5cc_0
Mar 25 08:31:01 fwaocpc4 cifs.upcall: find_krb5_cc: /tmp/krb5cc_0 is owned by 0, not 15635
Mar 25 08:31:01 fwaocpc4 cifs.upcall: find_krb5_cc: considering /tmp/krb5cc_15635_t4l12P
Mar 25 08:31:01 fwaocpc4 cifs.upcall: find_krb5_cc: FILE:/tmp/krb5cc_15635_t4l12P is valid ccache
Mar 25 08:31:01 fwaocpc4 cifs.upcall: handle_krb5_mech: getting service ticket for cifs/FS0Z0LLQ
Mar 25 08:31:01 fwaocpc4 cifs.upcall: cifs_krb5_get_req: unable to parse principal (cifs/FS0Z0LLQ).
Mar 25 08:31:01 fwaocpc4 cifs.upcall: handle_krb5_mech: failed to obtain service ticket (-1765328160)
Mar 25 08:31:01 fwaocpc4 cifs.upcall: handle_krb5_mech: getting service ticket for host/FS0Z0LLQ
Mar 25 08:31:01 fwaocpc4 cifs.upcall: cifs_krb5_get_req: unable to parse principal (host/FS0Z0LLQ).
Mar 25 08:31:01 fwaocpc4 cifs.upcall: handle_krb5_mech: failed to obtain service ticket (-1765328160)

No difference if I set the hostname to a fqdn.

Comment 3 Jeff Layton 2010-03-25 15:17:29 UTC
Could you set up syslog so that daemon.debug is logged and then redo the test? That allow some of the LOG_DEBUG calls in cifs.upcall.c to fire and give us a bit more info about what's going wrong.

Comment 4 Jeff Layton 2010-03-25 17:03:51 UTC
My mistake...the LOG_DEBUG messages are there. This is the relevant one:

Mar 25 08:31:01 fwaocpc4 cifs.upcall: cifs_krb5_get_req: unable to parse principal (cifs/FS0Z0LLQ).

...which is really strange. I'm not sure why it wouldn't be able to parse that. What version of krb5-libs do you have installed?

Comment 5 Jeff Layton 2010-03-25 17:08:13 UTC
The error here is:

#define KRB5_CONFIG_NODEFREALM                   (-1765328160L)

...could you attach the krb5.conf from this box?

Comment 6 Harald Milz 2010-03-26 09:34:45 UTC
Hi,

the default realm has been set ever since... however if I do a "kinit USER" w/o realm, kinit asks me for the realm as well. The krb5-libs package is the one from the RHEL 5.5 beta DVD ISO: krb5-libs-1.6.1-36.el5_4.1, which is -latest AFAIK. 



# krb5.conf Template für CUSTOMER
# Copyright (C) Red Hat GmbH 2009
# Autor: Harald Milz
# GPLv2
# based on works of others.

# $Id: krb5.conf.template 21 2009-12-08 14:14:01Z hmilz $
# $HeadURL: svn://192.168.88.100/aOC/aOCInstall/auth/krb5.conf.template $

[libdefaults]
	default_realm = R8708.ADS.CUSTOMER.DE
	dns_lookup_realm = false
	dns_lookup_kdc = true
	ticket_lifetime = 24h
	forwardable = yes
	clockskew = 300
	default_tgs_enctypes = RC4-HMAC DES-CBC-MD5 DES-CBC-CRC
	default_tkt_enctypes = RC4-HMAC DES-CBC-MD5 DES-CBC-CRC
	preferred_enctypes = RC4-HMAC DES-CBC-MD5 DES-CBC-CRC

[realms]
R8708.ADS.CUSTOMER.DE = {
#	auth_to_local = RULE:[1:$0\$1](^R8708\.ADS\.CUSTOMER\.DE\\.*)s/^R8708\.ADS\.CUSTOMER\.DE/R8708/
#	auth_to_local = DEFAULT
}

[logging]
	kdc = FILE:/var/log/krb5/krb5kdc.log
	admin_server = FILE:/var/log/krb5/kadmind.log
	default = SYSLOG:NOTICE:DAEMON
[domain_realm]
	.R8708.ads.CUSTOMER.de = R8708.ADS.CUSTOMER.DE
	R8708.ads.CUSTOMER.de = R8708.ADS.CUSTOMER.DE
[appdefaults]
pam = {
	ticket_lifetime = 1d
	renew_lifetime = 1d
	forwardable = true
	proxiable = false
	retain_after_close = true
	minimum_uid = 1
	try_first_pass = true
	mappings = R8708\\(.*) $1.CUSTOMER.DE
	validate = true
}
httpd = {
   mappings = R8708\\(.*) $1.CUSTOMER.DE
   reverse_mappings = (.*)@R8708\.ADS\.CUSTOMER\.DE R8708\$1
}

Comment 7 Harald Milz 2010-03-26 09:43:29 UTC
Makes sense to install the associated debuginfo package?

Comment 8 Jeff Layton 2010-03-26 12:01:06 UTC
cc'ing Nalin...

Nalin, any idea why we'd be getting KRB5_CONFIG_NODEFREALM from a krb5_parse_name() call here? The default realm seems to be properly set.

Comment 9 Jeff Layton 2010-03-26 20:52:48 UTC
(In reply to comment #2)
> Hi, I just tested cifs-utils-4.1, and, in a nutshell, the macro behaviour (from
> the end user's perspective) is unchanged.
> 
> Since we need to run usermounts from the logon script, and may have a couple
> hundred users on one machine (running NoMachine sessions), I need to compile
> mount.cifs with CIFS_DISABLE_SETUID_CHECK and CIFS_LEGACY_SETUID_CHECK enabled.
> The latter leads to a compile error:
>

I should be very clear here that the legacy setuid behavior is demonstrably unsafe. Red Hat does not ship mount.cifs with the setuid bit enabled for that reason.

Comment 10 Nalin Dahyabhai 2010-03-26 22:28:30 UTC
(In reply to comment #8)
> cc'ing Nalin...
> 
> Nalin, any idea why we'd be getting KRB5_CONFIG_NODEFREALM from a
> krb5_parse_name() call here? The default realm seems to be properly set.    

Nothing looks obviously wrong.

Comment 11 Harald Milz 2010-03-29 08:24:13 UTC
(In reply to comment #9)
> I should be very clear here that the legacy setuid behavior is demonstrably
> unsafe. Red Hat does not ship mount.cifs with the setuid bit enabled for that
> reason.    

hmmmm - if I don't run mount.cifs suid root I get an ENOPERM, else said error 126. And it is documented in the man page as well ... (at least RHEL 5.4 and 5.5 beta).

Comment 12 Harald Milz 2010-03-30 09:52:09 UTC
Hi,

I just went back to the samba-3.3.8-0.49 packages because I seemed to remeber it worked before, but no, it does not. 

If nothing looks obviously wrong in krb5.conf, what's the chance of an error in krb5-libs? 

I also tried to do the mount as root (in order to get rid of setuid). But if I do a "mount -o uid=user04", user04's TGT is used instead of root's (leading to error 126 again) . Which is where the uid= behaviour really bites us because it should only affect the uid/gid under which the share is mounted. 

If it's not a layer 8 problem and I do something utterly sick, methinks doing CIFS usermounts from a logon script using kerberos running as user is currently broken, no?

Comment 13 Jeff Layton 2010-03-30 11:00:50 UTC
(In reply to comment #12)
> Hi,
> 
> I just went back to the samba-3.3.8-0.49 packages because I seemed to remeber
> it worked before, but no, it does not. 
> 
> If nothing looks obviously wrong in krb5.conf, what's the chance of an error in
> krb5-libs? 
> 

That's my suspicion at this point. cifs.upcall doesn't really do anything different for the root user vs. other users. If root is able to determine the default realm, but other users aren't then it seems like there's something wrong in the krb5 libs or configuration.

It might be interesting to write a standalone program that just fetches the default realm via the krb5 API, and then test that as different users.

At this point, we probably need to have you open a RH support case so that someone in our support organization who can assist you in tracking this down.

Comment 14 Harald Milz 2010-03-30 11:41:10 UTC
(In reply to comment #13)

> It might be interesting to write a standalone program that just fetches the
> default realm via the krb5 API, and then test that as different users.
> 
> At this point, we probably need to have you open a RH support case so that
> someone in our support organization who can assist you in tracking this down.    

A support request has been open for a while now, and you /should/ have been copied on it. Please check #1999610

Comment 15 Harald Milz 2010-03-30 12:22:15 UTC
OMG. 

I wrote such a small program and found out that as root, the default realm was found just fine, while as user it was not. The reason turned out to be a mode 0600 /etc/krb5.conf, for reasons I don't understand right now... 

But let me check a new build of our client machine in different realms to be sure we don't miss anything.

Comment 16 Jeff Layton 2010-03-30 12:35:09 UTC
Interesting. I had wondered whether the problem might be that it couldn't open krb5.conf, but thought that would give a different error. /etc/krb5.conf definitely should be readable by users. I'm sort of surprised that you were able to get tickets at all by normal users, but maybe the SRV records in DNS took care of that.

Comment 17 Harald Milz 2010-03-31 10:10:10 UTC
(In reply to comment #16)
> I'm sort of surprised that you were
> able to get tickets at all by normal users, but maybe the SRV records in DNS
> took care of that.    

Maybe because pam_krb5 runs with root privileges?

In a nutshell: it works now. 

The reason was kinda subtle. The krb5.conf is generated from a template at post-install time where the umask is 0022, hence krb5.conf is 0644. This is why it would work on machines that were joined to the default (test) domain right after the installation. When moving to another domain, the template script is run in a normal root bash, where the umask is 0077, hence krb5.conf becomes 0600. Never worked again after moving. 

Duh.

Comment 18 Jeff Layton 2010-03-31 10:46:15 UTC
Possibly. In any case, I'll go ahead and close this as NOTABUG. Please reopen if you think we need to revisit it. We can continue the discussion about changing the behavior of the uid= option upstream.

Comment 19 tengel 2014-10-12 21:46:50 UTC
This is still a problem on 5.11 (kernel 2.6.18-398.el5xen, samba3x-client 3.6.23-6.el5) but in another way -- system level squashed @boot mounts don't work as expected (desired).

If you have a keytab generated like so from the Windows 2012 R2 server:

  ktpass.exe /princ cifsuser /ptype KRB5_NT_PRINCIPAL /out krb5.keytab +rndPass /crypto AES256-SHA1 /mapuser CIFSDOMAIN\cifsuser

...pull that keytab over, configure the upcalls and krb5.conf, then have an /etc/fstab mount like so:

  //cifsserver/cifsdata  /data  cifs  user=cifsuser,sec=krb5i,uid=nobody,gid=nobody,iocharset=utf8,file_mode=0644,dir_mode=0755,_netdev,soft  0 0

...the same -126 returns when root tries to mount it (@boot too). If 'uid=0' is set (per this bz) it works, but then all files are owned by root and hence a daemon can't write. The intent here is a share that is writable by a give UID only but mounted at boot for service  -- uid: apache for instance.

/proc/fs/cifs/cifsFYI = 9

Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifsfs.c: Devname: //cifsserver/cifsdata flags: 0 
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 2 with uid: 0
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: iocharset set to utf8
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: Username: cifsuser
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: UNC: \\cifsserver\cifsdata ip: 192.168.5.3
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: Socket created
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x6d6
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: Existing smb sess not found
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: Demultiplex PID: 1634
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifssmb.c: secFlags 0x1009
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifssmb.c: Kerberos only mechanism, enable extended security
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/transport.c: For smb_command 114
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/transport.c: Sending smb:  total_len 82
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: rfc1002 length 0xd1
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifssmb.c: Dialect: 2
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2 0x348 0xbb92
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2 0x348 0x1bb92
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/asn1.c: OID len = 8 oid = 0x1 0x2 0x348 0x1bb92
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3 0x6 0x1
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifssmb.c: Must sign - secFlags 0x1009
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifssmb.c: negprot rc 0
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: Security Mode: 0xf Capabilities: 0x8001f3fc TimeAdjust: 0
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/sess.c: sess setup type 5
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/cifs_spnego.c: key description = ver=0x2;host=cifsserver;ip4=192.168.5.3;sec=krb5;uid=0x63;user=cifsuser
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/sess.c: ssetup freeing small buf ffff88003b457300
Oct 12 21:02:54 cifs-client kernel:  CIFS VFS: Send error in SessSetup = -126
Oct 12 21:02:54 cifs-client kernel:  fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 2) rc = -126
Oct 12 21:02:54 cifs-client kernel:  CIFS VFS: cifs_mount failed w/return code = -126


The reason I consider this a bug is that RHEL5 is the only major distro I could put my hands on (RHEL/CentOS 6 and 7, Debian 7, Ubuntu 14, openSUSE 13.1, Arch, Gentoo) with this problem -- it's also the only one not using cifs-utils, so stands to reason. :) Is there an Access article already out there describing this problem and considering it Unsupported by Red Hat in RHEL5?