Bug 961969

This suggests that negotiation did not succeed.

Can you set LogLevel debug in /etc/httpd/conf.d/nss.conf and restart Apache? This will give us a clearer picture of what is going on in mod_auth_kerb.

Created attachment 747454 [details]
error log file

Slightly scrubbed logfile with debug enabled in nss.conf; connection was from Safari on OS X 10.8.3 with a valid kerberos ticket.

Can you share the equivalent time period for the krb5kdc log?

The way this is supposed to work is this the web server uses s4u2proxy to make the request on behalf of the user.

I'd expect to see a request and DELEGATION in the KDC log.

Sorry, should have thought of that.  Pasting it below since it's not too big.

May 13 15:41:07 ipa.mydomain.com krb5kdc[11249](info): AS_REQ (4 etypes {18 17 16 23}) 192.168.0.15: NEEDED_PREAUTH: host/ipa.mydomain.com for krbtgt/MYDOMAIN.COM, Additional pre-authentication required
May 13 15:41:07 ipa.mydomain.com krb5kdc[11249](info): AS_REQ (4 etypes {18 17 16 23}) 192.168.0.15: ISSUE: authtime 1368481267, etypes {rep=18 tkt=18 ses=18}, host/ipa.mydomain.com for krbtgt/MYDOMAIN.COM
May 13 15:41:08 ipa.mydomain.com krb5kdc[11249](info): TGS_REQ (4 etypes {18 17 16 23}) 192.168.0.15: ISSUE: authtime 1368481267, etypes {rep=18 tkt=18 ses=18}, host/ipa.mydomain.com for ldap/ipa.mydomain.com
May 13 15:41:28 ipa.mydomain.com krb5kdc[11249](info): TGS_REQ (7 etypes {18 17 16 23 3 2 1}) 192.168.0.50: ISSUE: authtime 1368208590, etypes {rep=18 tkt=18 ses=18}, vdanen for HTTP/ipa.mydomain.com
May 13 15:41:28 ipa.mydomain.com krb5kdc[11249](info): AS_REQ (4 etypes {18 17 16 23}) 192.168.0.15: NEEDED_PREAUTH: HTTP/ipa.mydomain.com for krbtgt/MYDOMAIN.COM, Additional pre-authentication required
May 13 15:41:28 ipa.mydomain.com krb5kdc[11249](info): AS_REQ (4 etypes {18 17 16 23}) 192.168.0.15: ISSUE: authtime 1368481288, etypes {rep=18 tkt=18 ses=18}, HTTP/ipa.mydomain.com for krbtgt/MYDOMAIN.COM

Hm, this looks suspicious:

[Mon May 13 15:41:28 2013] [debug] src/mod_auth_kerb.c(1939): [client 192.168.0.50] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos, referer: https://ipa.mydomain.com/ipa/ui/index.html
[Mon May 13 15:41:28 2013] [debug] src/mod_auth_kerb.c(1278): [client 192.168.0.50] Acquiring creds for HTTP.com, referer: https://ipa.mydomain.com/ipa/ui/index.html
[Mon May 13 15:41:28 2013] [debug] src/mod_auth_kerb.c(1372): [client 192.168.0.50] Using principal HTTP/ipa.mydomain.com for s4u2proxy, referer: https://ipa.mydomain.com/ipa/ui/index.html
[Mon May 13 15:41:28 2013] [error] [client 192.168.0.50] Credentials for HTTP/ipa.mydomain.com have expired or will soon expire - now 1368481288 endtime 1368456467, referer: https://ipa.mydomain.com/ipa/ui/index.html
[Mon May 13 15:41:28 2013] [debug] src/mod_auth_kerb.c(1372): [client 192.168.0.50] Using principal HTTP/ipa.mydomain.com for s4u2proxy, referer: https://ipa.mydomain.com/ipa/ui/index.html
[Mon May 13 15:41:28 2013] [error] [client 192.168.0.50] Credentials for HTTP/ipa.mydomain.com have expired or will soon expire - now 1368481288 endtime 1368456467, referer: https://ipa.mydomain.com/ipa/ui/index.html

Isn't the http keytab expired? You can try following:

# klist -kt /etc/httpd/conf/ipa.keytab 
Keytab name: FILE:/etc/httpd/conf/ipa.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 05/15/13 05:52:27 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   2 05/15/13 05:52:27 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   2 05/15/13 05:52:27 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   2 05/15/13 05:52:27 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM

# kinit -kt /etc/httpd/conf/ipa.keytab HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM

Valid starting     Expires            Service principal
05/15/13 08:43:02  05/16/13 08:43:02  krbtgt/IDM.LAB.BOS.REDHAT.COM.BOS.REDHAT.COM

If the kinit dies we have the root cause. But I assume this is not the case as you said it works for you on Fedora 18.

Is there any difference in how these browsers do negotiation? I see this header in my F18 Chrome:

Authorization:Negotiate YIIFiwYGK...4YMA==

Ah yes, that is definitely an issue:

# klist -kt /etc/httpd/conf/ipa.keytab
Keytab name: FILE:/etc/httpd/conf/ipa.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
# klist -kt /etc/httpd/conf/ipa.keytab
Keytab name: FILE:/etc/httpd/conf/ipa.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
   2 12/10/12 11:37:27 HTTP/ipa.mydomain.com
[root@ipa ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: HTTP/ipa.mydomain.com

Valid starting     Expires            Service principal
05/15/13 11:00:36  05/16/13 11:00:36  krbtgt/mydomain.com

So why isn't that keytab being updated?

Actually, when I look at the server that has mediawiki installed on it, it looks quite similar (these dates are from when I first installed IPA I think):

# klist -kt /etc/httpd/conf/http.keytab
Keytab name: FILE:/etc/httpd/conf/http.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   1 12/11/12 15:14:25 HTTP/wiki.mydomain.com
   1 12/11/12 15:14:25 HTTP/wiki.mydomain.com
   1 12/11/12 15:14:26 HTTP/wiki.mydomain.com
   1 12/11/12 15:14:26 HTTP/wiki.mydomain.com

Looking at response headers with the mediawiki I see:

WWW-Authenticate	Negotiate oRQw...CAg==, Negotiate oRQw...CAg==

I do not see anything similar when I go to the IPA UI with Chrome.

This is a very good question. Rob, is there any means of automatic keytab renewal in IPA? I know that SSSD has plans and test implementation for a deamon for keytab rotation but it is not in yet.

Anyway, until the automatic keytab rotation is resolved, one could rotate the keys manually:

# cd /etc/httpd/conf/
# kinit admin
Password for admin.BOS.REDHAT.COM: 
# klist -kt ipa.keytab            
Keytab name: FILE:ipa.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 05/16/13 02:34:54 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   2 05/16/13 02:34:54 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   2 05/16/13 02:34:54 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   2 05/16/13 02:34:54 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
# mv /etc/httpd/conf/ipa.keytab /etc/httpd/conf/ipa.keytab.bkp
# ipa-getkeytab -s $IPA_SERVER -p HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM -k /etc/httpd/conf/ipa.keytab
# klist -kt ipa.keytab
Keytab name: FILE:ipa.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   3 05/17/13 02:48:02 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   3 05/17/13 02:48:02 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   3 05/17/13 02:48:02 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM
   3 05/17/13 02:48:02 HTTP/vm-037.idm.lab.bos.redhat.com.BOS.REDHAT.COM



As for the inability to SSO from Mac, this may be deficiency of mod_auth_kerb. I may be wrong, but these lines in mod_auth_kerb.c looked suspicious:

static int
kerb_authenticate_user(request_rec *r)
{
...
   log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
	      "kerb_authenticate_user entered with user %s and auth_type %s",
	      (MK_USER)?MK_USER:"(NULL)",type?type:"(NULL)");

...

   /* get what the user sent us in the HTTP header */
   auth_line = (char *)MK_TABLE_GET(r->headers_in, (r->proxyreq == PROXYREQ_PROXY)
	                                    ? "Proxy-Authorization"
					    : "Authorization");
   if (!auth_line) {
      set_kerb_auth_headers(r, conf, use_krb4, use_krb5, 
	                    (use_krb5) ? "\0" : NULL);
      return HTTP_UNAUTHORIZED;
   }

If I read the code right, if Mac browser does not send Authorization/Proxy-Authorization header, the SSO may not work.

I think you're confusing the keytab with the ccache. The keytab holds the identity that of the web server. Its ccache (in various locations on different distros) holds its client tickets. The former doesn't normally change without admin intervention, the latter is managed by mod_auth_kerb.

Kerberos has to integrate into the HTTP authentication mechanism. This involves the server sending the client a WWW-Authenticate header including the type, the realm and any type-specific inforation (e.g. nonce with Digest auth). The client then responds with an Authorization header.

So you'll note that the first a web browser hits IPA mod_auth_kerb logs something like kerb_authenticate_user entered with user (NULL) and auth_type Kerberos. This is because the user sent no credentials because the browser didn't know they were required. The server responds with a WWW-Authenticate header.

The subsequent client response will include a base64-encoded GSSAPI request in the Authorization header with negotiate as the auth type (and this is why we require SSL on Kerberos HTTP connections).

So yeah, if the browser isn't sending an Authorization header then SSO won't work. The reason for this may be that the browser isn't authorized to negotiate. In Firefox this is configured using the negotiate options in about:config. In Chrome I believe you need to pass command-line options when it starts, --auth-server-whitelist="*.example.com"

Here is an oldish wiki page on it, http://freeipa.org/page/Fedora_Chrome

Note that you shouldn't (or need to) set a delegate whitelist.

(In reply to comment #9)
> So yeah, if the browser isn't sending an Authorization header then SSO won't
> work. The reason for this may be that the browser isn't authorized to
> negotiate. In Firefox this is configured using the negotiate options in
> about:config. In Chrome I believe you need to pass command-line options when
> it starts, --auth-server-whitelist="*.example.com"

Ok, but I think you missed the part where it works fine with mediawiki on rhel6, but not the IPA UI (same domain, same local network, etc.).

Why does it work on my rhel6 box with mediawiki but not the IPA ui?  All browsers work with mediawiki.

For my wiki virtualhost I have:

    <Location />
        AuthType Kerberos
        AuthName "Kerberos Login"
        KrbMethodNegotiate On
        KrbMethodK5Passwd Off
	KrbServiceName HTTP
        KrbAuthRealms MYDOMAIN.COM
        Krb5Keytab /etc/httpd/conf/http.keytab
	KrbSaveCredentials off
        #KrbServiceName HTTP
        #KrbVerifyKDC on
        #require user vdanen adanen
        require valid-user
    </Location>

This works -- I've not had to "log in" to mediawiki for two years, and I'm on that wiki pretty much every day.  This is true for Safari, Chrome, and Firefox.  They all work.

The only difference between the two that I can spot is that KrbSaveCredentials is on for the IPA host and _not_ for the mediawiki host.  I toggled that and restarted httpd on the IPA server and it didn't make a difference (for the browser; it's possible that is a red herring).

We need KrbSaveCredentials set in order to authenticate as you to LDAP. For the case where KrbConstrainedDelegation is enabled we don't actually require the TGT be forwarded.

If you want to try to set network.negotiate-auth.delegation-uris in Firefox to your domain that may fix things, we just prefer using S4U2Proxy.

Lets try Firefox since it has some client-side logging capabilities. This is what one would do on a Linux box, I'm hoping the same will work in a MacOS shell:

export NSPR_LOG_MODULES=negotiateauth:5
export NSPR_LOG_FILE=/tmp/moz.log
firefox

Any luck?

Sorry, just got around to trying this now.  A tab was restored to the wiki, so it shows up as well (first); I guess it's good to see the difference:

2024264064[10055f110]:   service = wiki.mydomain.com
2024264064[10055f110]:   using negotiate-gss
2024264064[10055f110]: entering nsAuthGSSAPI::nsAuthGSSAPI()
2024264064[10055f110]: Attempting to load gss functions
2024264064[10055f110]: entering nsAuthGSSAPI::Init()
2024264064[10055f110]: nsHttpNegotiateAuth::GenerateCredentials() [challenge=Negotiate]
2024264064[10055f110]: entering nsAuthGSSAPI::GetNextToken()
2024264064[10055f110]:   leaving nsAuthGSSAPI::GetNextToken [rv=0]
2024264064[10055f110]:   Sending a token of length 711
2024264064[10055f110]:   service = ipa.mydomain.com
2024264064[10055f110]:   using negotiate-gss
2024264064[10055f110]: entering nsAuthGSSAPI::nsAuthGSSAPI()
2024264064[10055f110]: entering nsAuthGSSAPI::Init()
2024264064[10055f110]: nsHttpNegotiateAuth::GenerateCredentials() [challenge=Negotiate]
2024264064[10055f110]: entering nsAuthGSSAPI::GetNextToken()
2024264064[10055f110]:   leaving nsAuthGSSAPI::GetNextToken [rv=0]
2024264064[10055f110]:   Sending a token of length 710

This looks normal from the client side.

As a test, can you configure Firefox on the Mac to delegate to your domain and try again?

Aha!

I only had network.negotiate-auth.trusted-uris set in about:config.  I've also set network.negotiate-auth.delegation-uris to my domain and also set network.negotiate-auth.using-native-gsslib to "true".  It seems that the "configure" (can't recall the link name now) when you go to the IPA server doesn't set these? 

Ok, based on the above I then poked at Chrome a little bit.  Seems it requires passing another commandline argument: --auth-negotiate-delegate-whitelist="*mydomain.com" (I previously was only passing --auth-server-whitelist).

So now it works.

Perhaps some fodder for documentation.  In light of this, I'd say this bug can be closed and maybe some documentation can/should be updated (I'll be updating my wiki documentation right away here).

As an aside, the ipa/config/ssbrowser.html page, under Firefox configuration, does _not_ mention delegation at all.

That is because delegation should no longer be required. I had you do that as a test.

Since you tweaked two options, delegation and native gssapi, it may be worth to try again un-setting delegation.

With delegation you send the full TGT to IPA which then uses it to perform ldap operations on your behalf. We now use S4U2Proxy where you use your TGT to authenticate, then the web server uses its credentials to perform ldap operations on your behalf.

I suppose it is possible that this is an issue with the Mac GSSAPI implementation.

Setting

network.negotiate-auth.using-native-gsslib true

is already covered in chapter 4.3.3 in documentation:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Identity_Management_Guide/index.html

I am closing this Bug as NOTABUG.