In current Fedora Rawhide / 29 openQA tests, all attempts to log in to a console on a system enrolled in the FreeIPA domain fail with a 'Login incorrect' error. kinit and logging into the web UI appear to work OK. In the simplest test scenario, the server test instance deploys itself as a server (with ipa-server-install), adds a user account called 'test1', adds an HBAC rule allowing test1 to log in to any system, and sets a permanent password for test1 by kiniting as that user and entering a new password when prompted. The client test instance then enrols into the domain using 'realm join' and the domain admin credentials, tests a few other things, then tries to log into a VT as test1, with the correct password; this should be allowed, but is denied. I will attach logs from both the server and the client. There are several errors logged, but I'm not sure what the fatal problem is yet, so just reporting against freeipa for now. This seems a clear Beta blocker, per Basic criteria: https://fedoraproject.org/wiki/Basic_Release_Criteria#Remote_authentication - "...the system must respect the identity, authentication and access control configuration provided by the domain."
Created attachment 1479106 [details] tarball of /var/log from the server
Created attachment 1479107 [details] tarball of /var/log from the client
per the openQA logs, the failed attempt at a console login occurs at 2018-08-27T18:04:20.0843 UTC. That's 14:04:20 "local time" for the logs that log in local time. Also note these tests have failed in the 20180827.n.0 Rawhide compose, which already has 389-ds-base 1.4.0.16, so this probably isn't the same cause as https://bugzilla.redhat.com/show_bug.cgi?id=1620315 .
Comment on attachment 1479107 [details] tarball of /var/log from the client This looks like an SSSD issue within KCM. in krb5_child.log we see (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [main] (0x0400): krb5_child started. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [unpack_buffer] (0x0100): cmd [241] uid [1721800001] gid [1721800001] validate [true] enterprise principal [false] offline [false] UPN [test1] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [unpack_buffer] (0x0100): ccname: [KCM:] old_ccname: [KCM:] keytab: [/etc/krb5.keytab] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [switch_creds] (0x0200): Switch user to [1721800001][1721800001]. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [switch_creds] (0x0200): Switch user to [0][0]. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [k5c_setup_fast] (0x0100): Fast principal is set to [host/client003.domain.local] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [check_fast_ccache] (0x0200): FAST TGT is still valid. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [become_user] (0x0200): Trying to become user [1721800001][1721800001]. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [set_lifetime_options] (0x0100): No specific renewable lifetime requested. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [set_lifetime_options] (0x0100): No specific lifetime requested. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [set_canonicalize_option] (0x0100): Canonicalization is set to [true] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [main] (0x0400): Will perform online auth (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [get_and_save_tgt] (0x0400): Attempting kinit for realm [DOMAIN.LOCAL] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [validate_tgt] (0x0400): TGT verified using key for [host/client003.domain.local]. (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [create_ccache] (0x0020): 1007: [-1765328188][Internal credentials cache error] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [map_krb5_error] (0x0020): 1808: [-1765328188][Internal credentials cache error] (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [k5c_send_data] (0x0200): Received error code 1432158209 (Mon Aug 27 14:04:20 2018) [[sssd[krb5_child[24055]]]] [main] (0x0400): krb5_child completed successfully So create_ccache reports 'Internal credentials cache error' -- this is KCM: ccache collection on Rawhide which is provided by the SSSD itself. Unfortunately, sssd_kcm.log is empty.
Jakub, could you please have a look?
The test currently does 'sss_debuglevel 6' to make sssd log more verbosely; let me know if there's anything else I can do to get more logs that would help.
Unfortunately sss_debuglevel currently doesn't work with sssd-kcm because sss_debuglevel relies on sssd starting all the services, but sssd-kcm is socket-activated. Is it possible to re-run the test with debug_level=10 in the [kcm] section of sssd.conf?
(In reply to Jakub Hrozek from comment #7) > Unfortunately sss_debuglevel currently doesn't work with sssd-kcm because > sss_debuglevel relies on sssd starting all the services, but sssd-kcm is > socket-activated. > > Is it possible to re-run the test with debug_level=10 in the [kcm] section > of sssd.conf? Nevermind, I reproduced the bug. Can you test a scratch build for me, please?
Here is a COPR repo: https://copr.fedorainfracloud.org/coprs/jhrozek/sssd-kcmfail/
Roger, will do. Thanks!
That looks like it works.
sssd-2.0.0-3.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-294230d586
+1 blocker
sssd-2.0.0-3.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-294230d586
+1 Blocker
Voted as a blocker in-bug: The decision to classify this bug as an "AcceptedBlocker" (Beta) was made as it violates the following criteria: "...the system must respect the identity, authentication and access control configuration provided by the domain."
sssd-2.0.0-3.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.
Hey all, I think it's still/again broken for sssd-2.0.0-5.fc29.x86_64 sssd-kcm-2.0.0-5.fc29.x86_64 on Fedora release 29. Further, I'd like to not that I have the same problem for GDM logins of FreeIPA users as well. If I login to the root account and then use su <freeipa-user> followed by kinit to fill the ticket cache it is afterwards possible to login with the freeipa-user. When I again destroy the ticket cache with kdestroy it isn't possible anymore. Greetings, Mike
Hum, I doubt it's *this* bug, but it does indeed appear that FreeIPA is busted in F29 at present; if you look at the openQA update test history: https://openqa.fedoraproject.org/group_overview/2?limit_builds=100 it seems to have been failing since FEDORA-2018-7db7ccda4d , which was a krb5 update and so a possible suspect as the cause...but it seems to have failed on other updates before that update was pushed stable, so the cause may in fact be something else. I wish someone hadn't broken this over the shutdown :( I and the whole FreeIPA team (AFAIK) are supposed to be off work until Jan 2, but I guess I'll try and round up some folks to try and figure this out.
OK, so, looking into it more closely, it does seem like that update (and its partner for F28) are the problem. In that build, -22, as well as adding a patch for the security bug that's mentioned in the changelog, rharwood re-enabled a patch that I had *disabled* in -21 because it was causing FreeIPA to break completely. It seems that patch *still* causes FreeIPA to break completely. So I'm sending out a -23 with the patch disabled again. Please check it out and see if it helps. https://bodhi.fedoraproject.org/updates/FEDORA-2018-dc944aaa79
Updating to the krb5 -23 packages that you've pushed to testing resolved my issue. Thanks! dnf --enablerepo=updates-testing update krb5-workstation Packages Altered: Upgrade krb5-devel-1.16.1-23.fc29.x86_64 @updates-testing Upgraded krb5-devel-1.16.1-22.fc29.x86_64 @@System Upgrade krb5-libs-1.16.1-23.fc29.i686 @updates-testing Upgraded krb5-libs-1.16.1-22.fc29.i686 @@System Upgrade krb5-libs-1.16.1-23.fc29.x86_64 @updates-testing Upgraded krb5-libs-1.16.1-22.fc29.x86_64 @@System Upgrade krb5-workstation-1.16.1-23.fc29.x86_64 @updates-testing Upgraded krb5-workstation-1.16.1-22.fc29.x86_64 @@System Upgrade libkadm5-1.16.1-23.fc29.x86_64 @updates-testing Upgraded libkadm5-1.16.1-22.fc29.x86_64 @@System
Great, thanks for the confirmation.
*** Bug 1662175 has been marked as a duplicate of this bug. ***
*** Bug 1663112 has been marked as a duplicate of this bug. ***