Bug 450776 - Failed to verify TGT cause of wrong keytab handling
Failed to verify TGT cause of wrong keytab handling
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: pam_krb5 (Show other bugs)
5.2
All Linux
low Severity high
: rc
: ---
Assigned To: Nalin Dahyabhai
Brian Brock
:
Depends On:
Blocks: 531471
  Show dependency treegraph
 
Reported: 2008-06-10 17:34 EDT by Klaus Ethgen
Modified: 2010-03-30 04:33 EDT (History)
4 users (show)

See Also:
Fixed In Version: pam_krb5-2.2.14-15
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 531471 (view as bug list)
Environment:
Last Closed: 2010-03-30 04:33:25 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Klaus Ethgen 2008-06-10 17:34:42 EDT
Description of problem:
The pam_krb5 do use the FIRST entry in the krb5.keytab to verify the TGT when 
using with validate. This ends in TGT verification error if the first entry is 
not the host/...@... entry matching the correct key.

How reproducible:
- Set up a complete kerberos realm with trust relationship.
- Configure a client to authenticate with this KDC.
- Add one (random) key to krb5.keytab (using ktutil or other; for example 
calling the key bla/foo@REALM).
- create a correct host/... key for the host and add it to the keytab.
- Add the option "validate" (and "debug" for this test) to your auth line of 
pam_krb5.so in system-auth.
- Be sure that you have NO ticket on any client machine (kdestroy) and no other 
keys like ssh keys and try to do a password login with ssh to the host.

Actual results:
You will find a line like
   TGT failed verification using key for 'bla/foo@REALM'
in your log

Expected results:
The line
   TGT verified using key for 'host/fqdn@REALM'
(just replace fqdn and REALM with the correct values)

Additional info:
If the first entry is the correct host key in the keytab everything works well.

Let me note that this is a RedHat specific bug as I tested the same with a 
debian system and pam_krb5 worked well.

Please note also that this has a high security impact as it is not possible to 
verify the identity of the TGT and so it can easily be faked!
Comment 2 RHEL Product and Program Management 2009-03-27 22:26:15 EDT
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 4 Klaus Ethgen 2009-05-15 05:09:59 EDT
Hello,

I am a bit confused about what steps to do to get that in the next minor release of the current version. We just have a representative for sales. For support we have some subscriptions in the RHEL support tracker.

Do you mean to open a ticket there or is there any I can set? I cannot see a rhel-x.y-flag.

For us it is very important to get this fix to at least RHEL5.
Comment 5 Klaus Ethgen 2009-05-15 06:11:03 EDT
Ups, resetting the needinfo flag
Comment 6 Nalin Dahyabhai 2009-05-15 14:38:30 EDT
(In reply to comment #4)
> I am a bit confused about what steps to do to get that in the next minor
> release of the current version. We just have a representative for sales. For
> support we have some subscriptions in the RHEL support tracker.
> 
> Do you mean to open a ticket there or is there any I can set? I cannot see a
> rhel-x.y-flag.

Apologies; that comment shouldn't have been public, as it's boilerplate generated by one of the scripts used in our internal bug processing procedures.
Comment 7 Klaus Ethgen 2009-05-15 14:58:30 EDT
So what do I have to do?

What should I do to get that bug solution to the current version (RHEL5)?
Comment 9 Nalin Dahyabhai 2009-06-10 15:10:15 EDT
(In reply to comment #0)
> Description of problem:
> The pam_krb5 do use the FIRST entry in the krb5.keytab to verify the TGT when 
> using with validate. This ends in TGT verification error if the first entry is 
> not the host/...@... entry matching the correct key.
> 
> How reproducible:
> - Set up a complete kerberos realm with trust relationship.
> - Configure a client to authenticate with this KDC.
> - Add one (random) key to krb5.keytab (using ktutil or other; for example 
> calling the key bla/foo@REALM).
> - create a correct host/... key for the host and add it to the keytab.
> - Add the option "validate" (and "debug" for this test) to your auth line of 
> pam_krb5.so in system-auth.
> - Be sure that you have NO ticket on any client machine (kdestroy) and no other 
> keys like ssh keys and try to do a password login with ssh to the host.
> 
> Actual results:
> You will find a line like
>    TGT failed verification using key for 'bla/foo@REALM'
> in your log
> 
> Expected results:
> The line
>    TGT verified using key for 'host/fqdn@REALM'
> (just replace fqdn and REALM with the correct values)

This looks normal.  The important requirement is that the key used to verify the TGT be one that's known only to the KDC and the local host, which we assume because the key is stored in a keytab that an unprivileged would-be attacker would be able to read.  The service name isn't important.

The module uses the first one it sees so that it doesn't have to worry about predicting the name of the service for which it will obtain credentials from the KDC to perform the verification.

> Please note also that this has a high security impact as it is not possible to 
> verify the identity of the TGT and so it can easily be faked!  

I'm afraid I don't follow your logic here.  So long as the key isn't public knowledge, any key will do -- the verification procedure (create an AP-REQ, and verify it using the keytab) is still followed.
Comment 10 Nalin Dahyabhai 2009-06-10 15:13:26 EDT
(In reply to comment #9)
> > Actual results:
> > You will find a line like
> >    TGT failed verification using key for 'bla/foo@REALM'
> > in your log
> > 
> > Expected results:
> > The line
> >    TGT verified using key for 'host/fqdn@REALM'
> > (just replace fqdn and REALM with the correct values)

Sorry, I missed the failure here -- why is the verification failing?  On my test system, this works.  Is the key for the service in question being shared among multiple systems?  If so, do the key version numbers and values agree when you run 'klist -k -K' across them?
Comment 11 Klaus Ethgen 2009-06-11 03:55:38 EDT
@10: Did you test the real key as _second_ key? If you have just the TGT key in the keytab or the TGT at first place everythink is fine.

@9: I think you miss how kerberos work. The keytab is a very critical point in the security and no unprivileged user must be allowed to read it at all! And yes, the requirement ist that _one_ of the known is the right but the redhat pam module just check the _first_ key!

Ah, and the service name is very important! The whole kerberos is using service names to find the right key. If there is the wrong sevice name the key is not the right one. If you "predigt" somethink to get the service name you did some really wrong. For login the only valid service name is the host/fqdn@REALM one. All other have no validity to a login and must not have at all!

To the security impact, the validate flag of the module is the only way to be sure that the connected KDC is the right one. Without this option there can be a bad person in the local network forging the KDC and get controll of the machine. The bug in pam_krb5 forbids the use of validate as it doesnt use the right key from the keytab.
Comment 12 Nalin Dahyabhai 2009-06-11 10:54:50 EDT
(In reply to comment #11)
> @10: Did you test the real key as _second_ key? If you have just the TGT key in
> the keytab or the TGT at first place everythink is fine.

Which key are you referring to here?  The key for the host service?  The test keytab I used contained keys for the "ldap" service and the "host" service, in that order.

> @9: I think you miss how kerberos work. The keytab is a very critical point in
> the security and no unprivileged user must be allowed to read it at all!

Yes, we're in agreement here.  But that's not what's going on.

>                                                                          And
> yes, the requirement ist that _one_ of the known is the right but the redhat
> pam module just check the _first_ key!

I don't understand what you're saying here.  Could you rephrase, please?
 
> Ah, and the service name is very important! The whole kerberos is using service
> names to find the right key. If there is the wrong sevice name the key is not
> the right one. If you "predigt" somethink to get the service name you did some
> really wrong. For login the only valid service name is the host/fqdn@REALM one.
> All other have no validity to a login and must not have at all!

We're not actually authenticating the user using Kerberos on the wire, though.  The main thing is that the client and server for an AP exchange agree on the server's principal name, and in this case the module acts as both.  Using the first key we find allows us to validate credentials even if the only keys present aren't remote-login keys.

> To the security impact, the validate flag of the module is the only way to be
> sure that the connected KDC is the right one. Without this option there can be
> a bad person in the local network forging the KDC and get controll of the
> machine.

Again, we agree.

>          The bug in pam_krb5 forbids the use of validate as it doesnt use the
> right key from the keytab.  

This is where you lost me.  Every key in the keytab should be a secret that clients don't know, so being able to create an AP-REQ that can be decrypted with any of the keys contained in the keytab is sufficient to verify that the TGT wasn't forged.
Comment 13 Klaus Ethgen 2009-06-11 11:45:35 EDT
Hi,

sorry that I do not quote you correctly but using a web form for discussion is
very odd.

Well, to your answer.

I think, we both talked one beside another. Sorry that my mother thong is not
English. However, I try to get ridge about it.

> Which key are you referring to here?  The key for the host service?  The test
> keytab I used contained keys for the "ldap" service and the "host" service, in
> that order.

Yes, the key for the host service. But let me ask, your ldap and host key is
different? Or is it the same key with just another name?

> Yes, we're in agreement here.  But that's not what's going on.

Well, in Note 9 you told that the keytab could be read by an enemy. And I told
you that that must be prohibit in all cases. However, maybe that was just a
mistyping of you.

> I don't understand what you're saying here.  Could you rephrase, please?

Well, I try. I said that the requirement is that one of the known keys in the
keytab must match (And not only the key, the name also!). RedHat on the other
side use only the first key to check the match (and trow the name part
completely away what is more evil).

> We're not actually authenticating the user using Kerberos on the wire, though.

Well, however that is the goal of the whole think. However, the bug I reported
forbid to do it the secure way.

> The main thing is that the client and server for an AP exchange agree on the
> server's principal name, and in this case the module acts as both.

Just to be sure, not only the server name, moreover also the client name.

> Using the first key we find allows us to validate credentials even if the only
> keys present aren't remote-login keys.

And that is the wrong and total dangerous behaviour. There is no reason for this
assumption that the first key is a valid key. Even worse the first key can be
from a other KDC which is not trusted to be used for accounting (but for other
relationship).

The correct way to check the key is to walk troughs the keytab and search for a
matching name, _then_ use the related key to check the validity of the KDC. For
the case of login into a system the name has to be "host/fqdn@REALM" and no
other service like ldap/ or else must be respected.

> This is where you lost me.

I don't think so. I think I understand you absolutely correct. But exact that
scares me!

> Every key in the keytab should be a secret that clients don't know,

Wrong. The key in the keytab has to be known by the client! By the client and
the KDC only. It is symmetric algorithms which is used in kerberos. So all
parties have to know the same secret.

> so being able to create an AP-REQ that can be decrypted with any of the keys
> contained in the keytab is sufficient to verify that the TGT wasn't forged.

No. That is a absolute dangerous assumption! There can be even evil keys in the
keytab which use is limited to just one service only.

But however, also that is not true with the redhat pam_krb5 as it do not try to
decrypt with _any_ key, it just try to decrypt with the first key only!

To make a real example. We have to use sharity to allow the user to mount his
$HOME. Sharity needs to have a trust relationship with a key in the keytab. So
that key is there. But that key is NOT the one of the KDC which is used to allow
the user to login! So that key which is known by the KDC is also in the keytab.
So the key for sharity is "nssldap/domaincontroller-fqn@REALM" and the key for
the host is "host/host-fqn@REALM". There is no way to say that the one is the
first and the other is the second. They can be in any order. The nssldap-Key is
a evil key in that case. (By the way, that is only the name of the key, we do
not use krb5 for nss.)

Regards
   Klaus
Comment 14 Nalin Dahyabhai 2009-06-11 12:22:48 EDT
(In reply to comment #13)
> > Which key are you referring to here?  The key for the host service?  The test
> > keytab I used contained keys for the "ldap" service and the "host" service, in
> > that order.
> 
> Yes, the key for the host service. But let me ask, your ldap and host key is
> different? Or is it the same key with just another name?

They are different on my test system.  I've just double-checked using 'klist -k -K' to be sure, but adding a key to a keytab with kadmin is expected to randomize the key.

> > Yes, we're in agreement here.  But that's not what's going on.
> 
> Well, in Note 9 you told that the keytab could be read by an enemy. And I told
> you that that must be prohibit in all cases. However, maybe that was just a
> mistyping of you.

Ah, I see.  Let me rephrase.  We assume that the keytab is protected so that would-be attackers can't read its contents.  This implies that any key contained in the file is known only to privileged processes on the client and the KDC. 

> > I don't understand what you're saying here.  Could you rephrase, please?
> 
> Well, I try. I said that the requirement is that one of the known keys in the
> keytab must match (And not only the key, the name also!). RedHat on the other
> side use only the first key to check the match (and trow the name part
> completely away what is more evil).

Okay.  I think this is where we're not thinking alike.  What benefit does the name actually provide?

> > We're not actually authenticating the user using Kerberos on the wire, though.
> 
> Well, however that is the goal of the whole think. However, the bug I reported
> forbid to do it the secure way.
> 
> > The main thing is that the client and server for an AP exchange agree on the
> > server's principal name, and in this case the module acts as both.
> 
> Just to be sure, not only the server name, moreover also the client name.

True, but the client's principal name is expected to be different from connection to connection, and the server is expected to be okay with that.

> > Using the first key we find allows us to validate credentials even if the only
> > keys present aren't remote-login keys.
> 
> And that is the wrong and total dangerous behaviour. There is no reason for
> this
> assumption that the first key is a valid key. Even worse the first key can be
> from a other KDC which is not trusted to be used for accounting (but for other
> relationship).

This I don't follow.  What's the purpose of having an invalid key in a keytab?  The cross-realm situation might be a problem, but cross-realm requires participation from the KDC as well.
 
> > Every key in the keytab should be a secret that clients don't know,
> 
> Wrong. The key in the keytab has to be known by the client! By the client and
> the KDC only. It is symmetric algorithms which is used in kerberos. So all
> parties have to know the same secret.

The client system has access to the key.  The user who's attempting to log in does not, and must not.

> > so being able to create an AP-REQ that can be decrypted with any of the keys
> > contained in the keytab is sufficient to verify that the TGT wasn't forged.
> 
> No. That is a absolute dangerous assumption! There can be even evil keys in the
> keytab which use is limited to just one service only.

Wait, what?  Can you elaborate on what you mean by "evil"?

> But however, also that is not true with the redhat pam_krb5 as it do not try to
> decrypt with _any_ key, it just try to decrypt with the first key only!

Trying them all would be time-consuming, and I'm not sure what we'd gain by it.
 
> To make a real example. We have to use sharity to allow the user to mount his
> $HOME. Sharity needs to have a trust relationship with a key in the keytab. So
> that key is there. But that key is NOT the one of the KDC which is used to
> allow
> the user to login! So that key which is known by the KDC is also in the keytab.
> So the key for sharity is "nssldap/domaincontroller-fqn@REALM" and the key for
> the host is "host/host-fqn@REALM". There is no way to say that the one is the
> first and the other is the second. They can be in any order. The nssldap-Key is
> a evil key in that case. (By the way, that is only the name of the key, we do
> not use krb5 for nss.)

Is the sharity service running on the local system?  Is the key only stored in the keytab and shared with the KDC?  If so, what makes it less useful than other keys?
Comment 16 Klaus Ethgen 2009-07-16 05:33:18 EDT
> > > Which key are you referring to here?  The key for the host service?  The test
> > > keytab I used contained keys for the "ldap" service and the "host" service, in
> > > that order.
> > 
> > Yes, the key for the host service. But let me ask, your ldap and host key is
> > different? Or is it the same key with just another name?
> 
> They are different on my test system.  I've just double-checked using 'klist -k
> -K' to be sure, but adding a key to a keytab with kadmin is expected to
> randomize the key.

What? Sorry, if you add a key it is put to keytab as it is. There is no
randomization there. Otherwise the whole kerberos wouldn't work at all.

> Ah, I see.  Let me rephrase.  We assume that the keytab is protected so that
> would-be attackers can't read its contents.  This implies that any key
> contained in the file is known only to privileged processes on the client and
> the KDC. 

That's true. Sorry for the misunderstanding.

> > > I don't understand what you're saying here.  Could you rephrase, please?
> > 
> > Well, I try. I said that the requirement is that one of the known keys in the
> > keytab must match (And not only the key, the name also!). RedHat on the other
> > side use only the first key to check the match (and trow the name part
> > completely away what is more evil).
> 
> Okay.  I think this is where we're not thinking alike.  What benefit does the
> name actually provide?

Ehem, sorry to say that, but I cannot tell you how the whole kerberos
system works at all. Just short: This name part specify which key should
be taken for that specific communication. The keytab can hold many
different keys from many kdcs. Not only from one.

> > > We're not actually authenticating the user using Kerberos on the wire, though.
> > 
> > Well, however that is the goal of the whole think. However, the bug I reported
> > forbid to do it the secure way.
> > 
> > > The main thing is that the client and server for an AP exchange agree on the
> > > server's principal name, and in this case the module acts as both.
> > 
> > Just to be sure, not only the server name, moreover also the client name.
> 
> True, but the client's principal name is expected to be different from
> connection to connection, and the server is expected to be okay with that.

What? Sorry, that's completely wrong. The principal name musn't change.
And a server should issue a big error message if such happens!

> > > Using the first key we find allows us to validate credentials even if the only
> > > keys present aren't remote-login keys.
> > 
> > And that is the wrong and total dangerous behaviour. There is no reason for
> > this
> > assumption that the first key is a valid key. Even worse the first key can be
> > from a other KDC which is not trusted to be used for accounting (but for other
> > relationship).
> 
> This I don't follow.  What's the purpose of having an invalid key in a keytab? 

I don't call it invalid key. It is a complete valid key but for complete
other service. Let me complete my sentence: There is no reason for this
assumption that the first key is a valid key _for that service_.

> The cross-realm situation might be a problem, but cross-realm requires
> participation from the KDC as well.

That has nothing or only little to do with cross-realm situations.

> > > Every key in the keytab should be a secret that clients don't know,
> > 
> > Wrong. The key in the keytab has to be known by the client! By the client and
> > the KDC only. It is symmetric algorithms which is used in kerberos. So all
> > parties have to know the same secret.
> 
> The client system has access to the key.  The user who's attempting to log in
> does not, and must not.

True. The user must nut but the client has to. That's what I tried to
tell.

> > > so being able to create an AP-REQ that can be decrypted with any of the keys
> > > contained in the keytab is sufficient to verify that the TGT wasn't forged.
> > 
> > No. That is a absolute dangerous assumption! There can be even evil keys in the
> > keytab which use is limited to just one service only.
> 
> Wait, what?  Can you elaborate on what you mean by "evil"?

As I tell it above, that means it is not valid for login but for other
services it has to be valid. The key valid for login has to be only the
one with name host/fqdn@REALM! No other Key should be allowed to be
checked for login facility but might be complete valid for other service
the host provides.

> > But however, also that is not true with the redhat pam_krb5 as it do not try to
> > decrypt with _any_ key, it just try to decrypt with the first key only!
> 
> Trying them all would be time-consuming, and I'm not sure what we'd gain by it.

Come on. You don't want to tell me that for real. That is how kerberos
works. And by the way how upstream (or at least other distributions)
works.

A simple strncmp() for the names in a loop is not this time consuming.
For example, the whole selinux stuff is much more expensive by the way.

> > To make a real example. We have to use sharity to allow the user to mount his
> > $HOME. Sharity needs to have a trust relationship with a key in the keytab. So
> > that key is there. But that key is NOT the one of the KDC which is used to
> > allow
> > the user to login! So that key which is known by the KDC is also in the keytab.
> > So the key for sharity is "nssldap/domaincontroller-fqn@REALM" and the key for
> > the host is "host/host-fqn@REALM". There is no way to say that the one is the
> > first and the other is the second. They can be in any order. The nssldap-Key is
> > a evil key in that case. (By the way, that is only the name of the key, we do
> > not use krb5 for nss.)
> 
> Is the sharity service running on the local system?  Is the key only stored in
> the keytab and shared with the KDC?  If so, what makes it less useful than
> other keys?  

Sharity is a "service" running on the local system. The key is NOT
shared with the KDC as the kdc do not have to know it.

It makes it much less useful, well I would say dangerous, as the key
therein is a shared secret for ALL machines running sharity. This is how
sharity has to be set up. This key is absolutely unsecure and
dangerous! Using it for login means to open the system for everybody.

Ps. I know how stupid that is. But this is how this is working.
Comment 20 errata-xmlrpc 2010-03-30 04:33:25 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0258.html

Note You need to log in before you can comment on or make changes to this bug.