RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1350012 - kinit / sssd kerberos fail over
Summary: kinit / sssd kerberos fail over
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sssd
Version: 7.4
Hardware: All
OS: All
unspecified
high
Target Milestone: rc
: ---
Assignee: Sumit Bose
QA Contact: ipa-qe
URL:
Whiteboard:
: 1380174 (view as bug list)
Depends On:
Blocks: 1420851 1644708 1647919
TreeView+ depends on / blocked
 
Reported: 2016-06-24 19:54 UTC by Eugene Keck
Modified: 2023-03-24 13:41 UTC (History)
22 users (show)

Fixed In Version: sssd-1.16.4-11.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-06 13:02:00 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github SSSD sssd issues 1983 0 None closed return multiple server addresses to the Kerberos locator plugin 2020-08-13 14:32:18 UTC
Red Hat Product Errata RHSA-2019:2177 0 None None None 2019-08-06 13:02:43 UTC

Description Eugene Keck 2016-06-24 19:54:11 UTC 
Description of problem:
kinit will only use the configured ipa_backup_server for a short period of time

Version-Release number of selected component (if applicable):
sssd-1.13.0-40.el7_2.4.x86_64

How reproducible:
Always

Steps to Reproduce:
1. setup sssd.conf with a ipa_server and ipa_backup_server
2. Take down primary ipa server
3. kinit <user>

Actual results:
kinit fails after about 20 seconds.

Expected results:
kinit <user> works

Additional info:
192.168.1.50 is my backup server (ipa_backup_server) and 192.168.1.51 is my primary server (ipa_server).

What I am seeing is kinit will fail over after I take down my primary server.

#  KRB5_TRACE=/dev/stdout kinit
[7733] 1466797071.639806: Getting initial credentials for root
[7733] 1466797071.641727: Sending request (163 bytes) to EXAMPLE.COM
[7733] 1466797071.641897: Initiating TCP connection to stream 192.168.1.50:88
[7733] 1466797071.642201: Sending TCP request to stream 192.168.1.50:88
[7733] 1466797071.643177: Received answer (169 bytes) from stream 192.168.1.50:88
[7733] 1466797071.643187: Terminating TCP connection to stream 192.168.1.50:88
[7733] 1466797071.643252: Response was from master KDC
[7733] 1466797071.643272: Received error from KDC: -1765328378/Client not found in Kerberos database
kinit: Client 'root' not found in Kerberos database while getting initial credentials

Then about 20 seconds later it moves back to the primary server 

#  KRB5_TRACE=/dev/stdout kinit                                                                        
[7741] 1466797094.952302: Getting initial credentials for root
[7741] 1466797094.954328: Sending request (163 bytes) to EXAMPLE.COM
[7741] 1466797094.954495: Initiating TCP connection to stream 192.168.1.51:88
[7741] 1466797094.954734: Terminating TCP connection to stream 192.168.1.51:88
[7741] 1466797094.954777: Sending initial UDP request to dgram 192.168.1.51:88
kinit: Cannot contact any KDC for realm 'EXAMPLE.COM' while getting initial credentials

kinit will fail until sssd needs to use the backup kerberos server then it will work for another 20 seconds until it moves back to the primary server. I was not able to get around this behavior using srv or static records in my /etc/krb5.conf
Comment 2 Sumit Bose 2016-06-27 07:56:27 UTC 
I think this is expected behavior. Please see the 'FAILOVER' section in the sssd-krb5 man page. As you can see SSSD will try to fall-back automatically when a backup server is used. If you want to stick to the failover server you should add both servers as 'ipa_server' and do not use the 'ipa_backup_server' option.
Comment 3 Joe Collette 2016-06-29 22:24:10 UTC 
I would say this is not the expected behavior and the kdc should not flip back to the primary, unless the service is available. Yes, the sssd is checking every 30 seconds or so whether the primary is back up, but ldap does not failback unless the primary server becomes available again, that is expected behavior and kinit should behave the same way.


The file /var/lib/sss/pubconf/kdcinifo.DOMAIN is being updated with the backup server IP after the primary fails, but during the first check to see if the primary IPA server is available again, the file is inadvertently updated with the primary server IP again and therefore kinit fails.

Having all servers on the ipa_server line is not an option if you want your servers to properly failback to their "local" IPA server in a particular region or datacenter. If you have all the servers on the ipa_server line, they failover to the next server in line and never revert to the first in the list. That's why there is the "ipa_backup_server" option so that you can failover to another IPA server, but when your primary comes online, it will failback.
Comment 4 Sumit Bose 2016-07-06 08:01:16 UTC 
I think the most flexible solution is described in https://fedorahosted.org/sssd/ticket/941 where multiple addresses are made available by the plugin.

I'm currently thinking about what would be the best strategy for ordering the the additional addresses. As long as all servers (primary and backup) are configured explicitly we can try the one considered active by SSSD first and then add all primary servers and finally all backup servers.

If the servers are discovered dynamically we might need to add a configurable cut-off limit because e.g. in larger AD domains there might be a huge amount of servers even if we restrict the list to servers from the local site. If the plugin will return too many servers application will be busy for quite some time to check all servers before finally giving up if e.g. there are network issues.
Comment 5 Jakub Hrozek 2016-07-07 11:53:03 UTC 
Upstream ticket:
https://fedorahosted.org/sssd/ticket/941
Comment 7 Jakub Hrozek 2016-11-28 09:12:16 UTC 
There is some work we might do in the scope of 7.4, but it's not scoped for sure. Some might be improvements to the kdcinfo plugin itself, some might be in the scope of the KCM server.
Comment 8 Jakub Hrozek 2017-02-01 13:24:39 UTC 
*** Bug 1380174 has been marked as a duplicate of this bug. ***
Comment 20 Fabiano Fidêncio 2018-06-14 18:18:49 UTC 
Fixed as part of the following series ...
 master:
  efae950
  9f68324
  c1fbc6b
  2124275
  cc79227
  d91661e
  4759a48
  f28d995
Comment 21 Jakub Hrozek 2018-06-14 19:46:11 UTC 
I'm going to move the bug back to ASSIGNED. I know the upstream tickets were really less than clear, cross-referenced each other and so did the bugzillas.

So to make it clear what was pushed to upstream:
 - the Kerberos locator plugin is now able to consume multiple addresses from the kdcinfo files. This will most probably be in 7.6 (modulo the usual disclaimer about future releases..)
 - BUT there is no mechanism yet in SSSD which would write multiple addresses into the kdcinfo files

 - the patches add the ability to generate the kdcinfo files for trusted domains of directly joined AD clients and IPA masters in a trust relationship with an AD domain
  - BUT there is no mechanism yet for writing the kdcinfo files on IPA clients in an IPA-AD trust setup. I'm working on a subsequent patch that would implement this and I hope that the patch makes 7.6 (modulo the usual disclaimer about future releases..)

We're now working on another, subsequent patchset that would write multiple addresses into kdcinfo files, but /only in the IPA-AD client case/.

I hope it is clearer what was implemented and what is still in the works.
Comment 23 Jakub Hrozek 2019-03-19 22:44:56 UTC 
First ticket fixed:
    master: 63ccbfe
    sssd-1-16: 96e4d71
Comment 24 Jakub Hrozek 2019-03-23 19:38:19 UTC 
Another master PR:
* master: 208a79a
* sssd-1-16: c225ed7
Comment 25 Jakub Hrozek 2019-04-02 20:41:11 UTC 
Final PR:
 * master: e8d806d9bbb1ba288ed6a83158113f4d8f8a8929
 * sssd-1-16: ec0c31a71ef8ec61e11b1c6d21edcf9b923ce4ca
Comment 33 errata-xmlrpc 2019-08-06 13:02:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2177
Note You need to log in before you can comment on or make changes to this bug.