Bug 315311 - Add Kerberos support to CIFS mounts
Add Kerberos support to CIFS mounts
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
All Linux
high Severity high
: ---
: ---
Assigned To: Jeff Layton
Fedora Extras Quality Assurance
: FutureFeature
Depends On: 334931
Blocks: 231968
  Show dependency treegraph
Reported: 2007-10-02 08:44 EDT by Jeff Layton
Modified: 2007-12-20 07:07 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2007-12-20 07:07:01 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
patch 1 -- rudimentary patch for doing upcalls using keyctl (9.67 KB, patch)
2007-10-04 10:18 EDT, Jeff Layton
no flags Details | Diff
patch 2 -- convert blob to hex string before sending (9.59 KB, patch)
2007-10-05 15:14 EDT, Jeff Layton
no flags Details | Diff
patch 3 -- fix session setup request packet for SPNEGO auth (14.38 KB, patch)
2007-10-05 22:04 EDT, Jeff Layton
no flags Details | Diff
patch 4 -- add upcalls using keyctl for CIFS auth (14.50 KB, patch)
2007-10-05 22:09 EDT, Jeff Layton
no flags Details | Diff
patch 5 -- add upcalls using keyctl for CIFS auth (16.34 KB, patch)
2007-10-16 07:51 EDT, Jeff Layton
no flags Details | Diff
patch -- update to use new keyctl binary blob patches (12.90 KB, patch)
2007-10-19 12:25 EDT, Jeff Layton
no flags Details | Diff
patch -- add upcalls using keyctl for CIFS auth (34.73 KB, patch)
2007-10-24 15:26 EDT, Jeff Layton
no flags Details | Diff
patch -- implement SPNEGO by offloading to userspace via keyctl (34.92 KB, patch)
2007-10-25 14:18 EDT, Jeff Layton
no flags Details | Diff
patch -- get SPNEGO blob for session setup via downcall (19.59 KB, patch)
2007-11-05 11:18 EST, Jeff Layton
no flags Details | Diff

  None (edit)
Description Jeff Layton 2007-10-02 08:44:47 EDT
+++ This bug was initially created as a clone of Bug #231968 +++

Escalated to Bugzilla from IssueTracker

-- Additional comment from tao@redhat.com on 2007-03-13 07:13 EST --
1. Feature request
Add Kerberos support to CIFS mounts

2. Description
When we use Windows AD as authentication service (winbindd & Kerberos) we could
connect to Windows shares and printers using the same Kerberos authentication
too. The problem is that CIFS mounts do not support Kerberos authentication. The
old smbmount used to be able to make Kerberos-authenticated mounts (only to
Samba server).

So we need a Kerberos support for CIFS mounts too. This is a badly needed
feature and one major block on Linux's way to be really usable in large
enterprises. See IT#109085/BZ#219591 for a former discussion.

Architectures:          all
Priority:               High
Target Releases:        RHEL 5.1

3. Business Justification:
Large copororations are full with Windows shares (CIFS). Linux users need to use
them as well as Windows users. While the shares can be mounted using passwords
(every time a user does something he/she needs to enter password is not very
userfriendly nor secure). This feature is badly needed for providing common and
secure filesharing services for Linux users also.

4. Status of the code:

5. Hardware to Red Hat:

6. Partner management contact, email, phone, chat

7. Partner technical contact, email, phone, chat
This event sent from IssueTracker by ichihi  [Nokia-BI-Creation]
 issue 116054

-- Additional comment from jlayton@redhat.com on 2007-06-24 18:41 EST --
Grabbing this for now, but have no clue at this point what this entails...

-- Additional comment from jlayton@redhat.com on 2007-07-03 07:57 EST --
Adding Simo so that he's aware. I've not had a chance to look over this at all
yet, however.

-- Additional comment from ssorce@redhat.com on 2007-07-03 08:51 EST --
This is going to be tough.
I have started some work around this areas 7 months ago before coming to RedHat.

The biggest problem is deciding o a decent upcall mechanism because we don't
want to do any kerberos or ASN stuff in the kerbel. I have been talking with
Steve French on the upcall mechanism to use and we couldn't find the optimal one.

I have also been in talk with steved before you took over cifs and the idea was
that we should try to work with the nfs folks to try to share as much code as
possible and settle on just one common upcall mechanism (nfs actually use 2
different upcall mechanism right now :/) and a common pluggable daemon to do the
krb and mapping stuff.

To be done it needs real commitment and a couple of months of coding effort for
the first release.

-- Additional comment from jlayton@redhat.com on 2007-08-03 10:03 EST --
This is a big job, and I'm not familiar enough yet with the intricacies of how
krb5 mounts are supposed to work. 

Following Simo's suggestion though, the first thing we have to tackle is a way
for the kernel to upcall to a userspace daemon.

I took some time yesterday and looked over the kernel upcall mechanisms already
in use by NFS. There are two:

1) the server-side code uses the sunrpc/cache.h routines. This is what does the
/proc/net/rpc/* directories. The main file of interest in these is the "channel"
file, which acts as a conduit between userspace and the kernel cache.

2) the client-side code uses rpc_pipefs directory which is a pseudofs that
implements pipe-like files for communication between kernel and userspace. Each
nfs client creates a pipe in there and rpc.gssd handles them in turn.

We could likely use either, though rpc_pipefs might be a better choice if we
want a multithreaded cifs.gssd at some point in the future. The cache.h stuff is
probably easier to use, however, and has the benefit of caching entries for us
so we won't have to do it ourselves and won't have to upcall as much.

If we want to use rpc_pipefs, we could probably just mount it again under
/var/lib/cifs/rpc_pipefs or something and have cifs.ko create its own directory
hierarchy under there.

IMO we should first shoot for some proof-of-concept code before we try to go
making large changes to the upstream kernel. i.e. we really need to know what we
want there before we go proposing moving either or both schemes into their own
modules. For now, I'd suggest we pick one of the mechanisms and develop for it
and deal with the fact that we'll have a temporary dependency of cifs.ko on

Once we have something that works and that we feel good about it then we can
start proposing to move the relevant code to its own module upstream.


-- Additional comment from ssorce@redhat.com on 2007-08-03 10:23 EST --
I think it is a good plan.
My personal idea is that we should reach a point where we have a generic
/proc/net/krb5 dir and a generic daemon servicing it, and an internal kernel
interface that all modules that need to use kerberos/SPNEGO/GSSAPI can use to do
their job.
If we can generalize the code enough that both nfs and cifs can use the same
code I think we will have a very good service module usable by many other parts
of the kernel.

-- Additional comment from staubach@redhat.com on 2007-08-03 10:28 EST --
NFS already has a mechanism based on GSS-API.  Perhaps starting there
would be a good thing.  It is a mechanism based on the GSS-API so that
it doesn't have to know about specific things like kerberos, lipkey,
spkm, etc.  Perhaps the cifs support could/should be modified to use
that interface instead of inventing something completely new.

-- Additional comment from jlayton@redhat.com on 2007-08-03 11:04 EST --
Bear with me, I'm still getting up to speed on GSSAPI and the way NFS uses it :-)

Peter has a good point though. Perhaps we should consider trying to just use the
existing rpc.gssd daemon. That would make us heavily dependent in the short term
on NFS/RPC, but once we had something working with that we could look at ways to
make it more filesystem agnostic...

-- Additional comment from staubach@redhat.com on 2007-08-03 11:06 EST --
This might lead to an architecture which would allow the use of more than
just kerberos as well...  :-)

-- Additional comment from ssorce@redhat.com on 2007-08-03 11:53 EST --
Sure. Kerberos is just an example, SPNEGO/GSSAPI is what we aim for, but the
current kernel implementation in the NFS code is not useful for CIFS as it lacks
too many features (IIRC it support only DES keys for example), which CIFS needs
to be useful (most AD domains use ARCFOUR and now they are starting using AES).

What I don't want to do is to do GSSAPI/Kerb implementations in the kernel, as I
think that's the wrong place to do that processing.

-- Additional comment from staubach@redhat.com on 2007-08-03 13:43 EST --
I'd definitely agree with that.  All of the security flavor specific stuff
should be done at the user level in a daemon and there should be an upcall
to it so that the kernel can utilize it.

I think that that is all rpc.gssd should have been, but perhaps the
architecture wasn't so clean...

-- Additional comment from jlayton@redhat.com on 2007-08-14 16:37 EST --
I had a close look at rpc_pipefs today. I think we can do a proof-of-concept
sort of thing with that, but a lot of that code is very centered around RPC. For
instance things like rpc_mkdir() take an rpc_client arg. For CIFS we won't have
such a thing, so we'll have to fake one up or figure out a way to make it not be

Also, not all of the symbols we'll need are exported, so we'll need to change it
so that they are if we want to use rpc_pipefs for this.

I plan to compose an email this week to Steve French to see if he can direct me
to the work that's already been done for this as well...

-- Additional comment from jlayton@redhat.com on 2007-08-15 09:42 EST --
The gssd interface looks very simple. It looks like the kernel just writes a
user ID to the pipe (in network byte order, I think) and gssd just writes that
back to the pipe with a trailing binary blob containing the krb5 ticket (or SPKM
auth info, I suppose).

-- Additional comment from riek@redhat.com on 2007-08-28 14:58 EST --
Adjusting priority to "high" as the ability to authenticate CIFS mounts via
Kerberos plays a vital role in our interoperability strategy. It is on the list
of top requirements from the IPA group.

-- Additional comment from jlayton@redhat.com on 2007-09-29 09:56 EST --
Set up a Win2k3 AD server and fileserver, and a win2kpro client. Joined the
win2kpro client to the AD domain, and mounted a test export from the win2k3
server. Differences from a typical cifs mount are:

In the Negotiate Protocol request, the extended security flag is set. I believe
this makes the server send along some extra info in the negotiate protocol
response -- in particular, gssapi SPNEGO stuff. In the session setup, there is
more SPNEGO info exchanged, and I think at the end of that a key is settled upon.

After the session setup, there doesn't seem to be any more kerberos-related info
exchanged. I *think* that the authentication is done by making sure that the
signature is correct on the packets.

To Do:

- Read up on SPNEGO (looks like CIFS has some routines for this already, but I'm
guessing that they're pretty skeletal still and may suffer from bitrot)

- pull the MS royalty free interoperability docs (check the license and make
sure it doesn't "taint" me), and see if that outlines exactly how the signatures
are calculated

- determine how the signatures are actually calculated here (maybe look over
what smbclient does for kerberos mounts)

I think at this point, I'm going to set aside worrying about how the kerberos
tickets get from userspace to kernel. That's really an implementation detail
(albeit an important one), and I can do some things to fudge it for now.

At this pont, I'm more interested in understanding exactly what info the kernel
needs as far as krb5 tickets, and what it does with it once it has it.
Comment 1 Jeff Layton 2007-10-02 09:05:00 EDT
Started looking over the code that's actually in mainline at this point (I also
have sent a note to Steve French asking if he already has some out-of-tree code
written for this).

I've tinkered with the rawhide CIFS code to allow me to set SecurityFlags to
0x9009. That corresponds to:


With that, the Negotiate Protocol request has this:

.... 1... .... .... = Extended Security Negotiation: Extended security
negotiation is supported

...the server response looks correct. The mount attempt falls down after the
NegProt response is received and no session setup is sent, but that's pretty
much expected at this point.

Comment 2 Jeff Layton 2007-10-02 09:11:13 EDT
There already seems to be an expectation that CIFS is going to use netlink
sockets for upcalls (there's some skeletal code already and it seems to point in
that direction). In looking at it, I'm thinking that's a reasonable thing here.
There's a bit more overhead than with a rpc_pipefs type interface, but it
doesn't require any special setup (like actually mounting rpc_pipefs).

The largest security blobs I've seen are ~1k, which is well within a page, but
I'm not clear on what the max payload size of a netlink packet. It would be much
preferable to make sure that the request can fit in a single packet since
netlink is connectionless.
Comment 3 Simo Sorce 2007-10-02 09:41:22 EDT
in large domains the PAC can be *very* big, in the 60Kb order. A single ticket,
which contains the PAC and is passed to the server, may be that big. As you have
to pass the ticket down you need to ensure you can handle it.
Comment 4 Jeff Layton 2007-10-02 10:02:51 EDT
Ahh that's a good point. Netlink isn't considered reliable when the machine is
under memory pressure, etc...but there are similar issues with using /proc-style
interfaces. Any time you're sending large quantities of data between kernel and
userspace, things get tricky (large being anything close to or more than a
page). It may be good to see if we can strip off the PAC before sending to
userspace as well (not sure how difficult that would be).

Given that upcalls aren't likely to be a frequent occurrence (only during
negprot/session_setup), I'm thinking that this is still probably the best
mechanism to start with.

CONFIG_CIFS_UPCALL is dependent upon CONFIG_CONNECTOR. Looking at the connector
API (see Documentation/connector in the kernel sources), it allows for a seq and
ack number. If we have to break up the security blob into multiple pieces to
send to userspace then we can take advantage of that to make sure that we get
all of the pieces. If we don't then we can send a NACK of some sort to the
kernel and have it resend the entire blob.

If you're looking at the userspace piece, then I'd suggest writing it in a way
that abstracts the GSSAPI stuff from the actual send/receive mechanisms. That
way if this proves not to be workable then changing that part out shouldn't be
too difficult.
Comment 5 Simo Sorce 2007-10-02 10:19:23 EDT
(In reply to comment #4)
It may be good to see if we can strip off the PAC before sending to
> userspace as well (not sure how difficult that would be).

A) It would be impossible, it's part of the ticket, and you don't have the
encryption material to sign it :-)
B) It is the other way around, the PAC is IN the user ticket and *need* to be
passed to the server which will use it to set up your authorization credentials.
So it is userspace -> kernel, but if you keep processing in userspace, it is
actually simply unknown to the kernel, it is just that blobs that flow in and
out will be big.

> CONFIG_CIFS_UPCALL is dependent upon CONFIG_CONNECTOR. Looking at the connector
> API (see Documentation/connector in the kernel sources), it allows for a seq and
> ack number. If we have to break up the security blob into multiple pieces to
> send to userspace then we can take advantage of that to make sure that we get
> all of the pieces. If we don't then we can send a NACK of some sort to the
> kernel and have it resend the entire blob.

Yes the problem is that this interface, as you see, oblige you to implement a
lot of logic to handle just kernel-userspace communication. It is a bit like
implementing TCP again ... a pain :)

> If you're looking at the userspace piece, then I'd suggest writing it in a way
> that abstracts the GSSAPI stuff from the actual send/receive mechanisms. That
> way if this proves not to be workable then changing that part out shouldn't be
> too difficult.

GSSAPI libraries already do that as they have no clue what is the transport.

Comment 6 Jeff Layton 2007-10-02 10:37:57 EDT
> It would be impossible, it's part of the ticket, and you don't have the
> encryption material to sign it :-)

Yep, I just realized that. So we have to ship the whole thing to the kernel. If
it's that large, doing that won't be trivial, but there's not much we can do
about that regardless of the mechanism we use...
Comment 7 Jeff Layton 2007-10-04 10:18:44 EDT
Created attachment 215841 [details]
patch 1 -- rudimentary patch for doing upcalls using keyctl

This patch is a start at adding upcall capability using the keyctl facility.
It's not as "wrapperized" as we probably want, but that turns out to be harder
to do than expected since we have to do some setup work during module
initialization (setting up key types, etc).

I've experimented with it by adding a line like this to /etc/request-key.conf:

#====== ======= =============== =============== ===============================

create	cifs.spnego	*		*		|/bin/cat

With this, the security blob just gets piped into stdin of /bin/cat and the
stdout ends up being the key payload. With this, I see the same payload go in
and out:

 fs/cifs/cifsfs.c: Devname: //dhcp231-40.rdu.redhat.com/testuser flags: 64 
 fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 0 with uid: 0
 fs/cifs/connect.c: Username: testuser
 fs/cifs/connect.c: UNC: \\dhcp231-40.rdu.redhat.com\testuser ip:
 fs/cifs/connect.c: Socket created
 fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x7fffffffffffffff
 fs/cifs/connect.c: Demultiplex PID: 7122
 fs/cifs/connect.c: Existing smb sess not found
 fs/cifs/cifssmb.c: secFlags 0x8009
 fs/cifs/transport.c: For smb_command 114
 fs/cifs/transport.c: Sending smb of length 78
 fs/cifs/connect.c: rfc1002 length 0xb7
 fs/cifs/cifssmb.c: Dialect: 2
SPNEGO blob:: dump of 94 bytes of data at 0xffff81000771c150
 06065c60 0501062b 52a00205 30a05030 ` \ . . + . . . . . � R 0 P � 0
 09062e30 8248862a 020112f7 2a090602 0 . . . * . H . � . . . . . . *
 f7864886 02020112 862a0a06 12f78648 . H . � . . . . . . * . H . 
 03020201 062b0a06 82010401 0a020237 . . . . . . + . . . . . 7 . . .
 1a301ca3 161b18a0 326e6977 746a336b � . 0 . � . . . w i n 2 k 3 j t
 4540246c 504d4158 432e454c a56b4d4f l $ @ E X A M P L E . C O M

This matches the SPNEGO blob on the wire. keyctl show also shows:

760709127 --alswrv	0     0   \_ cifs.spnego: 00000000:

This currently hangs the key off of cifsSesInfo. Presumably once we do the
session setup, we can do a key_put on this and it should get freed.

This makes some assumptions so far:

1) no blobs bigger than 64k. If we need bigger than that, we might should
consider vmallocs for them (might be reasonable anyway)

2) the SPNEGO blobs don't have embedded NULLs. So far, none I've seen do, but
that might not be a safe assumption. If it's not safe, then we'll probably need
to do some sort of binary->string conversion for the blob before handing off to
userspace (base64 or something).

I'm thinking we might also want to add a second keytype for session keys so
they can be generated in an upcall as well. We could hang the mac_signing_key
off of a struct key. When upcalls aren't needed they can just generate the key
as they do now, and do an add_key with all of the data.

Note that I've also seen some lockdep warnings with keyctl. dhowells has
responded that it's a false alarm -- looking at that now...
Comment 8 Jeff Layton 2007-10-04 11:07:48 EDT
To be clear, the data that's sent from kernel to userspace can't have embedded
NULLs. That data gets passed to the request-key program as a string, and is null
terminated. The output from the userspace program *can* have embedded NULLs so
we just need a program that takes the description and the SPNEGO blob and
printf's a new SPNEGO blob suitable for the session setup to stdout. Raw data is
fine there (and would in fact be preferred).

Comment 9 Steve French 2007-10-05 11:11:49 EDT
Can't we use the connector api to pass the blob up - using the keyctl to tell
user space the read from connector the blob from kernel etc. so that there is no
need to do conversion of the embedded nuls
Comment 10 Jeff Layton 2007-10-05 11:35:04 EDT
We could, but connector has some things that make it sub-optimal. It's based on
netlink which is connectionless. If queues are full, etc, then packets can get
dropped there, so we need to implement sequencing and acks...

At the end of the day, the data gets moved via copy_to_user/copy_from_user.
Having to deal with sockets for that seems like a pain. I'd prefer to keep this
as simple as possible and have to deal with only one conduit from kernel to
userspace. keyctl is pretty darned simple...

Also, the conversion here could be something as simple as walking through each
byte of the blob and doing:

     sprintf(deststring, "%2.2x", blob[i])

...the userspace program would just have to do the reverse. That does basically
double the size of the blob though, so it would be nice if we had a more
space-efficient conversion scheme. I think though, that vmalloc is probably
suitable for this. It's not as big a deal to allocate a huge temp buffer if we
use that...
Comment 11 Jeff Layton 2007-10-05 15:14:53 EDT
Created attachment 217871 [details]
patch 2 -- convert blob to hex string before sending

I'm going to take from Steve's comment that the assumption about embedded NULLs
isn't reliable. This patch converts the blob into a hex string before sending
it off to userspace.

Tested with a simple userspace perl program that just does:

while (read(STDIN, $hexbyte, 2)) {
	printf("%c", hex($hexbyte));

The blob I get back looks correct:

SPNEGO blob:: dump of 94 bytes of data at 0xffff8100128640a8
 06065c60 0501062b 52ff0205 30ff5030 ` \ . . + . . . . . � R 0 P � 0
 09062e30 ff48ff2a 020112ff 2a090602 0 . . . * � H � � . . . . . . *
 ffff48ff 02020112 ff2a0a06 12ffff48 � H � � . . . . . . * � H � �
 03020201 062b0a06 ff010401 0a020237 . . . . . . + . . . . � 7 . . .
 1a301cff 161b18ff 326e6977 746a336b � . 0 . � . . . w i n 2 k 3 j t
 4540246c 504d4158 432e454c a56b4d4f l $ @ E X A M P L E . C O M

I guess I also need to fix up the caller of cifs_spnego_convert to set secType.
For now, I suppose we can assume that it's kerberos in this situation, but we
probably need to handle it more intelligently.

Looking at CIFS_SessSetup now to see what changes need to be made there...
Comment 12 Jeff Layton 2007-10-05 22:04:44 EDT
Created attachment 218261 [details]
patch 3 -- fix session setup request packet for SPNEGO auth

This patch supercedes the last one. With this, I get an actual sane looking
Session Setup packet getting sent over the wire (according to wireshark). Since
I don't have a userspace program that does a correct SPNEGO conversion I've
just tested it by sending back the same blob that came in the negProt reply.
The server replies to the session setup request with

I think this is all I can really do at this point until the callout program is
available. Some questions I have though:

1) I'm not sure what we want to do about the signing key. Are there any docs on
how that's calculated with Kerberos? Any thoughts on whether to calculate it in
kernel space, or whether we should do another upcall for it?

2) I'm not sure how best to handle NTLMSSP vs. Kerberos in CIFSSMBNegotiate.
How do we detect it? Or do we even care since it's just a SecurityBlob?

3) Is it possible to have a server that does kerberos but not unicode? Do I
need to break up ascii_ssetup_strings in a similar fashion how the unicode one
is done?

The patch needs to be broken up into smaller more easily digestable chunks too.
I plan to tackle that next week.

There are also some cleanup patches embedded in this one that should probably
go in before we start adding in the actual kerberos stuff.
Comment 13 Jeff Layton 2007-10-05 22:09:41 EDT
Created attachment 218271 [details]
patch 4 -- add upcalls using keyctl for CIFS auth

Last patch had a missing delta, the number of iovecs passed to SendReceive2 was
Comment 14 Steve French 2007-10-12 00:18:26 EDT
Looking at the cifsauth patch, I am wondering if we really need to save the
spnego key in the session struct - if we reconnect don't we need to regenerate it?
Comment 15 Jeff Layton 2007-10-12 06:39:58 EDT
I'd imagine we would. I just added it to that for now to give me a way to easily
pass it to the session setup. I pretty much envisioned freeing the key once the
session setup packet went out.

I'm still working on a respin based on some changes to keyctl that I discussed
with David H. and I'll have a look at that too...
Comment 16 Steve French 2007-10-13 19:08:36 EDT
I would like to check in a few of the changes to fs/cifs/sess.c within this two
week merge window (not necessarily the completed upcall - if the keyctl or other
changes are still being debated).  I may take a look at this if Shirish makes
enough progress on the cifsacl->unix_mode code patches this week.
Comment 17 Jeff Layton 2007-10-13 19:46:22 EDT
Sounds good. I have some preliminary patches that need to go in before the main
kerberos patches go in anyway. I'll try to have a set together early this week
for review.
Comment 18 Jeff Layton 2007-10-16 07:51:39 EDT
Created attachment 228611 [details]
patch 5 -- add upcalls using keyctl for CIFS auth

Revised patch that contains a new method for encoding the upcall data. This
should be much more space efficient than hex encoding. With Dave Howells' patch
to allow for larger keysizes from userspace, this scheme is only limited by
ARG_MAX. That's currently set to 128k (at least in Fedora). I think we can
reasonably deal with SPNEGO blobs of ~100k with this scheme (which should be
plenty -- Simo seemed to think that there was a 64k limit to keys anyway).

I've also sent a patchset upstream of small cleanup patches, plus a few of the
sess.c changes that Steve wanted. I'm not comfortable merging too much there
until we have something that actually work.
Comment 19 Jeff Layton 2007-10-16 14:32:04 EDT
I've opened bug 334931 to track development of the userspace piece of this.
Development is focused there at the moment...
Comment 20 Jeff Layton 2007-10-19 12:25:54 EDT
Created attachment 232741 [details]
patch -- update to use new keyctl binary blob patches

This is the latest iteration of the cifs spnego patch. Major changes are:

- takes advantage of David H's patches to allow passing of a binary blob to
request_key_with_auxdata(). This means we don't have to encode the blob when we
send it off to userspace.

- makes it so that we don't hang the key off of the cifsSesInfo struct. Instead
cifs_session_setup declares a pointer to a key and passes that around.

- some functions have changed to hopefully more descriptive names

Currently this relies on a patchset from David H that's not yet upstream. It's
probably too late to get those into 2.6.24 (some are very new), but hopefully
it'll get into -mm soon and can make 2.6.25.
Comment 22 Jeff Layton 2007-10-24 15:26:06 EDT
Created attachment 236521 [details]
patch -- add upcalls using keyctl for CIFS auth

I think this patch is now pretty close to being in a working state. With this +
a hacked up version of Igor's userspace program, I was able to get a SUCCESS
response from the server on a session setup request. The tree connect then
proceeded to fail with an NT_STATUS_ACCESS_DENIED, but I'm not certain that I
have everything set up correctly permissions-wise on the server.

I need to do some code cleanup and break up the patchset into more easily
digestible chunks. I also want to see if I can get a working mount before we
pull any of this into upstream.

I do, however, have at least one preliminary patch that should probably go in
beforehand. I'll post upstream soon too...
Comment 23 Jeff Layton 2007-10-24 15:59:50 EDT
Above patch is against Steve F's cifs-2.6 git tree. It also incorporates some of
David Howells' keyctl patches that are not yet upstream.
Comment 24 Jeff Layton 2007-10-24 16:04:33 EDT
Success! With a hacked up version of Igor Mammedov's userspace request-key
callout, I was able to sucessfully mount a win2k3 server using kerberos auth.
There's still plenty of work to be done, but the kernel piece seems to be working.
Comment 25 Jeff Layton 2007-10-25 14:18:32 EDT
Created attachment 237721 [details]
patch -- implement SPNEGO by offloading to userspace via keyctl

This patch is a cleaned up and revised version of the one posted yesterday. It
also works with a modified version of Igor's program. This also more carefully
handles the case when CONFIG_CIFS_UPCALL isn't set.

The patch also incorporates 3 patches from David Howells that aren't yet

It should apply cleanly on top of Steve F's cifs-2.6 tree with
44093ca2fef3c52dc7d186116862d74f9a676e0f as the head.

I've posted a broken-out version of this patchset upstream to the
linux-cifs-client mailing list as an RFC. Awaiting comments.
Comment 26 Jeff Layton 2007-11-05 11:18:15 EST
Created attachment 248401 [details]
patch -- get SPNEGO blob for session setup via downcall

This patchset has been partially merged into the upstream cifs-2.6 tree. As of
now, it's considered experimental and depends on CONFIG_CIFS_UPCALL which
depends on CONFIG_CIFS_EXPERIMENTAL. The userspace request-key callout is still
being worked on, but the patch has been tested with a test callout program and
works correctly.

There's still work to be done, here's the list of to-do's that I posted with
the patchset:


1) The userspace request-key program is not yet complete. Igor and Simo
Sorce have volunteered to write it. I've used Igor's userspace program
to test it, but it needs some work (primarily work on having it deal
with different UID's). I can envision this being part of the samba tree.
smbclient needs to roll up SPNEGO blobs too, so it seems like that code
could be shared. I'll defer that decision to those who are writing
the program, though...

2) Multi-stage SPNEGO conversations are not yet implemented. It
shouldn't be too tough to add them, but as of today, I'm not clear on
what they actually look like on the wire. Particularly, I don't know
what the SMB status field will look like. So for now, if multi-stage
SPNEGO is needed, the mount will probably fail. If someone is able
to trigger this and can provide captures or info about it, then we
should be able to get that added.

3) multi-user mounts are not currently implemented. I've been chatting
with Steve about the best scheme for handling them, so for now, things
are pretty much still single-user.


The main thing for our customers I think is #3. Kerberos auth is much more
useful if it allows multiple user credentials to be used. Once this part gets
settled in upstream I'll start working on that piece. The design for how to
handle multisession mounts is still being debated.
Comment 27 Jeff Layton 2007-11-05 11:21:10 EST
One other note...

With the current upstream code, the keyctl API limits the size of key payload to
32k. David Howells has a patch to increase that to 1M (using a vmalloc if
kmalloc fails), but it probably won't make it upstream until 2.6.25.

That means that for now, this patch will only work when the SPENGO blob from
userspace is < 32k. Once that patch is merged we should be able to deal with any
reasonable size blob.
Comment 28 Jeff Layton 2007-11-05 11:21:46 EST
Changing dependencies since we're more or less waiting on the userspace piece at
this point...
Comment 29 Jeff Layton 2007-11-20 14:10:18 EST
Steve has committed the last of the current patchset upstream. They should make
the 1.52 release. With that (and once the cifs.spnego.c samba patch is
committed), we should be able to support the trivial case of Kerberos mounts
where all linux users share the same session to the server (and hence, the same

To make krb5 mounts really useful though, I think we need to allow for cifs to
spawn new sessions as needed. There are a couple of different approaches (pardon
the hand-waviness, since I don't have this worked out well):

1) allow for multiple sessions per mount and have the code select the
appropriate session when a user goes to access it. This would probably mean
keeping a reference count for the session. On unmount, we'd decrement the
refcount on any session attached to the mount and drop any that hit 0. It'll
probably be a bit more complex than this, but can be done within the context of
cifs code.

2) spawn a new per-user mount when a new user accesses the share. This looks a
bit more complex:

For instance, suppose a new user with proper krb5 creds touches a mountpoint. We
spawn a new per-process namespace mount with the MNT_SHRINKABLE flag set. Where
do we attach this mount? It's possible for processes to get their own namespaces
on a clone() call, but it's pretty rare and I don't think we want CIFS to try to
do that after the fact.

If we just try to use MNT_SHRINKABLE like it's being used for existing
filesystems (AFS and NFS), then we have to decide where to put the new mount. I
suppose we could overlay the existing mount, but that seems sort of messy and
it's not clear to me what we gain from that model.

After looking at it, I think just handling a list of sessions sounds like the
easiest approach, but maybe I'm misunderstanding what Steve intends here...

Comments? Thoughts?
Comment 30 Jeff Layton 2007-12-17 11:33:17 EST
I've started looking over the multiuser piece today. When there is more than one
krb5 mount to the same server using different creds, there are currently
problems. The first krb5 mount works fine. For instance, consider these 2 mounts
in /etc/fstab:

//win2k3.example.com/export /mnt/cifs       cifs     user=root,sec=krb5i,noauto 0 0
//win2k3.example.com/testuser /mnt/testuser       cifs    
user,user=testuser,uid=50000,gid=50000,sec=krb5i,noauto 0 0

If both root and testuser have a valid TGT, both mounts should work. In practice
though, the second one always fails. I think it's because it tries to use info
from the first session incorrectly.

Investigating that now. Once we have some of these kinks worked out with the
existing code, we can start working on real multiuser mounts with kerberos.
Comment 31 Jeff Layton 2007-12-17 11:42:39 EST
Testing this at home. First mount succeeded. Second one fails with this cifsFYI

 fs/cifs/cifsfs.c: Devname: //win2k3.poochiereds.net/testuser flags: 64 
 fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 222 with uid: 0
 fs/cifs/connect.c: Username: testuser
 fs/cifs/connect.c: UNC: \\win2k3.poochiereds.net\testuser ip:
 fs/cifs/connect.c: Existing tcp session with server found
 fs/cifs/connect.c: Existing smb sess not found
 fs/cifs/connect.c: Security Mode: 0xf Capabilities: 0x8001f3fd TimeAdjust: 0
 fs/cifs/sess.c: sess setup type 6
 fs/cifs/cifs_spnego.c: key description =
 fs/cifs/transport.c: For smb_command 115
 fs/cifs/transport.c: Sending smb:  total_len 1396
 fs/cifs/connect.c: rfc1002 length 0x27
 CIFS VFS: Unexpected SMB signature
Status code returned 0xc0000022 NT_STATUS_ACCESS_DENIED
 fs/cifs/netmisc.c: Mapping smb error code 5 to POSIX err -13
 fs/cifs/misc.c: Null buffer passed to cifs_small_buf_release
 fs/cifs/sess.c: ssetup rc from sendrecv2 is -13
 fs/cifs/sess.c: ssetup freeing small buf d5e47e00
 CIFS VFS: Send error in SessSetup = -13
 fs/cifs/connect.c: No session or bad tcon
 fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 222) rc = -13
 CIFS VFS: cifs_mount failed w/return code = -13

...I'm thinking that sharing the TCP session is the issue, but I'm still
investigating why.
Comment 32 Jeff Layton 2007-12-17 13:01:21 EST
As Simo pointed out, the problem seems to be with signatures. When I disabled
mandatory signing on the server and mounted with sec=krb5 both mounts worked
correctly. Still looking at possible resolutions...
Comment 33 Jeff Layton 2007-12-17 13:08:10 EST
When I mount the same share twice (once as root and once as testuser) and turn
on /proc/fs/cifs/MultiuserMount, things seem to work correctly. When I log in as
"testuser" and touch a file on root's mountpoint, then the testuser session is
actually used.
Comment 34 Jeff Layton 2007-12-17 14:59:31 EST
I did a temporary patch to disable sharing of tcp session if the username
doesn't match and signing was enabled on the tcp session. That worked, so I
think we'll probably need something like that, though we need to handle all
possible cases and that wouldn't.

The way signing flags are handled is a bit of a mess at the moment, so some
cleanup there first might be in order. The rule should be that we only share TCP
sessions with another mount if the server is the same and:

1) signing is disabled on both mounts


2) the user is the same on both mounts

...any other situation should cause a new TCP session to be spawned.

Comment 35 Jeff Layton 2007-12-20 07:00:08 EST
The last issue turns out to have been a bug in the SPNEGO patch. New SPNEGO SMB
sessions were overwriting the key, when they should have allowed the first
session after a Negotiate Protocol exchange to have precedence. I've pushed a
patch upstream to fix that and a minor alignment issue with the string section
of the session setup.

At this point, I'm going to go ahead and close this BZ as complete. I may open
others as a point of discussion as to how we want to handle multiuser mounts.
Comment 36 Jeff Layton 2007-12-20 07:07:01 EST
Since upcalls have now been enabled in rawhide kernels, closing this with a
resolution of RAWHIDE.

Note You need to log in before you can comment on or make changes to this bug.