Bug 1582111 - Regression: authselect 0.4-3 breaks FreeIPA sudo rules
Summary: Regression: authselect 0.4-3 breaks FreeIPA sudo rules
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: authselect
Version: 28
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Pavel Březina
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-24 09:07 UTC by Martin Pitt
Modified: 2018-09-07 09:23 UTC (History)
12 users (show)

Fixed In Version: authselect-0.4-4.fc28
Clone Of:
Environment:
Last Closed: 2018-09-07 09:19:56 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
log with authselect 0.4-3 and freeipa pre1 (15.97 KB, application/x-gzip)
2018-05-24 12:35 UTC, Martin Pitt
no flags Details
logs with authselect 0.4-3 and freeipa pre2 (15.71 KB, application/x-gzip)
2018-05-24 12:46 UTC, Martin Pitt
no flags Details
logs with authselect 0.4-3 and freeipa pre2 and enable-feature with-sudo (15.71 KB, application/x-gzip)
2018-05-24 12:50 UTC, Martin Pitt
no flags Details
logs from upgrading authselect and ipa (16.12 KB, application/x-gzip)
2018-05-24 13:09 UTC, Martin Pitt
no flags Details
nsswitch.conf with authselect 1.0-1 and freeipa 4.7.0 (2.51 KB, text/plain)
2018-08-27 20:33 UTC, Martin Pitt
no flags Details
ipaclient-install.log with authselect 1.0-1 and freeipa 4.7.0 (114.05 KB, text/plain)
2018-08-27 20:34 UTC, Martin Pitt
no flags Details
/etc/authselect/user-nsswitch.conf with authselect 1.0-1 and freeipa 4.7.0 (1.69 KB, text/plain)
2018-08-28 08:22 UTC, Martin Pitt
no flags Details

Description Martin Pitt 2018-05-24 09:07:32 UTC
Description of problem: I was testing a FreeIPA update (https://bodhi.fedoraproject.org/updates/FEDORA-2018-93dfeefc68) which pulls in a newer authselect (0.4-3). This breaks FreeIPA centrally managed sudo rules. Turns out this isn't the fault of freeipa, but just upgrading authselect from the previous version 0.4-1 to 0.4-3 already breaks it


Version-Release number of selected component (if applicable):

0.4-3.fc28

How reproducible: Always


Steps to Reproduce:
1. Enroll into a FreeIPA domain
2. Add a sudo rule for the "admin" user:
   ipa sudorule-add --hostcat=all --cmdcat=all All && ipa sudorule-add-user --groups=admins All

3. Log in as that user and try to run `sudo whoami`

Actual results:

$ sudo whoami
[sudo] password for admin: 
admin is not in the sudoers file.  This incident will be reported.

Expected results:

sudo succeeds (like it does with the previous authselect)

Additional info:

 * This is broken for both qualified (admin) or unqualified (admin) user names.
 * This was detected by Cockpit's integration tests when trying to refresh the Fedora 28 image (https://github.com/cockpit-project/cockpit/pull/9224)

Comment 1 Alexander Bokovoy 2018-05-24 10:16:43 UTC
This is due to https://github.com/pbrezina/authselect/commit/4b1981a67216f56e67cff3887fe38ee8063ee0b2. Unfortunately, FreeIPA wasn't consulted and updated to add 'with-sudo' option, thus it broke.

I'm moving this to freeipa for handling.

Comment 2 Martin Pitt 2018-05-24 10:35:25 UTC
ab | pitti: until authselect use is fixed in freeipa, you can get sudo rules back with 'authselect enable-feature with-sudo'

Comment 3 Alexander Bokovoy 2018-05-24 11:32:55 UTC
Upstream ticket:
https://pagure.io/freeipa/issue/7562

Comment 4 Martin Pitt 2018-05-24 12:35:05 UTC
Created attachment 1441043 [details]
log with authselect 0.4-3 and freeipa pre1

As requested by Alexander, I'm collecting config/log files from various scenarios.

This is authselect 0.4-3 (the "broken" one) with the current freeipa-client-4.6.90.pre1-6.1.fc28 ; the minimal change to the previously working scenario with authselect 0.4-1 and the same freeipa

Comment 5 Alexander Bokovoy 2018-05-24 12:41:36 UTC
great, thanks.

With freeipa pre1/authselect 0.4-3:

------------------------
2018-05-24T12:27:49Z DEBUG Starting external process
2018-05-24T12:27:49Z DEBUG args=['/usr/sbin/authconfig', '--enablesssd', '--enablesssdauth', '--enablemkhomedir', '--update']
2018-05-24T12:27:49Z DEBUG Process finished, return code=0
2018-05-24T12:27:49Z DEBUG stdout=Running authconfig compatibility tool.

IMPORTANT: authconfig is replaced by authselect, please update your scripts.
See Fedora 28 Change Page: https://fedoraproject.org/wiki/Changes/AuthselectAsDefault
See man authselect-migration(7) to help you with migration to authselect

Executing: /usr/bin/authselect select sssd --force with-mkhomedir
--------------------------

So my theory is right: authselect is called via authconfig compatibility layer and sets the default configuration without sudo support.

Comment 6 Martin Pitt 2018-05-24 12:46:22 UTC
Created attachment 1441044 [details]
logs with authselect 0.4-3 and freeipa pre2

This is now an image with the newer authselect (0.4-3) and freeipa 4.6.90.pre2-3.fc28 (from https://bodhi.fedoraproject.org/updates/FEDORA-2018-93dfeefc68). Again, this is "clean install, then upgrade packages, *then* enroll into the domain". Same sudo issue.

Comment 7 Martin Pitt 2018-05-24 12:50:51 UTC
Created attachment 1441045 [details]
logs with authselect 0.4-3 and freeipa pre2 and enable-feature with-sudo

Same as before, but now I ran "authselect enable-feature with-sudo", which added the appropriate authselect files. Et voilà:

[root@x0 ~]# authselect enable-feature with-sudo
[root@x0 ~]# su - admin
Last login: Do Mai 24 08:48:51 EDT 2018
[admin@x0 ~]$ sudo whoami
[sudo] password for admin: 
root

Attaching the logs again for completeness.

Comment 8 Alexander Bokovoy 2018-05-24 12:52:25 UTC
And with pre2/authselect 0.4-3 we seem to run authselect directly:

-----------------------
2018-05-24T12:40:21Z DEBUG Starting external process
2018-05-24T12:40:22Z DEBUG args=['/usr/bin/authselect', 'current', '--raw']
2018-05-24T12:40:22Z DEBUG Process finished, return code=2
2018-05-24T12:40:22Z DEBUG stdout=No existing configuration detected.

2018-05-24T12:40:22Z DEBUG stderr=
2018-05-24T12:40:22Z DEBUG Current configuration not managed by authselect
2018-05-24T12:40:22Z WARNING WARNING: The configuration pre-client installation is not managed by authselect and cannot be backed up. Uninstallation may not be able to revert to the original state.
2018-05-24T12:40:22Z DEBUG Loading StateFile from '/var/lib/ipa-client/sysrestore/sysrestore.state'
2018-05-24T12:40:22Z DEBUG Saving StateFile to '/var/lib/ipa-client/sysrestore/sysrestore.state'
2018-05-24T12:40:22Z DEBUG Starting external process
2018-05-24T12:40:22Z DEBUG args=['/usr/bin/authselect', 'select', 'sssd', 'with-mkhomedir', '--force']
2018-05-24T12:40:22Z DEBUG Process finished, return code=0
2018-05-24T12:40:22Z DEBUG stdout=
2018-05-24T12:40:22Z DEBUG stderr=
2018-05-24T12:40:22Z INFO SSSD enabled
-----------------------

So we definitely need to add with-sudo here.

Comment 9 Martin Pitt 2018-05-24 13:09:39 UTC
Created attachment 1441054 [details]
logs from upgrading authselect and ipa

Upgrade scenario: Starting from IPA pre1 and authselect 0.4-1, then enrolling into a domain (sudo works), and *then* upgrading packages:

Upgrading:
 authselect                                   x86_64                        0.4-3.fc28                                   updates                              24 k
 authselect-compat                            x86_64                        0.4-3.fc28                                   updates                              30 k
 authselect-libs                              x86_64                        0.4-3.fc28                                   updates                              49 k
 freeipa-client                               x86_64                        4.6.90.pre2-3.fc28                           @commandline                        157 k
 freeipa-client-common                        noarch                        4.6.90.pre2-3.fc28                           @commandline                         65 k
 freeipa-common                               noarch                        4.6.90.pre2-3.fc28                           @commandline                        604 k
 nss                                          x86_64                        3.36.1-1.1.fc28                              updates                             646 k
 nss-softokn                                  x86_64                        3.36.1-1.1.fc28                              updates                             383 k
 nss-softokn-freebl                           x86_64                        3.36.1-1.1.fc28                              updates                             227 k
 nss-sysinit                                  x86_64                        3.36.1-1.1.fc28                              updates                              64 k
 nss-tools                                    x86_64                        3.36.1-1.1.fc28                              updates                             480 k
 nss-util                                     x86_64                        3.36.1-1.0.fc28                              updates                              90 k
 python3-ipaclient                            noarch                        4.6.90.pre2-3.fc28                           @commandline                        556 k
 python3-ipalib                               noarch                        4.6.90.pre2-3.fc28                           @commandline                        558 k
Installing dependencies:
 libsss_simpleifp                             x86_64                        1.16.1-2.fc28                                fedora                               76 k
 python3-sss                                  x86_64                        1.16.1-2.fc28                                fedora                               92 k
 sssd-dbus                                    x86_64                        1.16.1-2.fc28                                fedora                              187 k
 sssd-tools                                   x86_64                        1.16.1-2.fc28                                fedora                              349 k

This created a 0-byte (!) file /var/log/ipaupgrade.log. Then I rebooted. This now also broke sudo:

# grep -r sudo /etc/authselect/
/etc/authselect/nsswitch.conf:sudoers:    files

after reboot, /var/log/ipaupgrade.log is still empty, so that doesn't help much. However, attaching all the files as before.

Comment 10 Alexander Bokovoy 2018-05-24 13:15:57 UTC
Ok, so upgrade case is equal to the first one: authconfig-compat creates default profile 'sssd', then authselect upgrade removes sudo because it is not part of the default profile anymore.

Comment 11 Florence Blanc-Renaud 2018-05-30 08:56:03 UTC
Moving the BZ to authselect component as authselect should handle upgrade from authconfig to authselect when sudo support is enabled.

Comment 13 Pavel Březina 2018-06-12 08:39:26 UTC
I provided a scratch build to the other bug [1] and I'm waiting for some feedback before I push the build to Fedora.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1577615#c7

Comment 14 Martin Pitt 2018-06-15 08:13:12 UTC
I installed the authselect{,-libs}-0.4-3.1.fc28.x86_64 scratch build, joined a domain, and nsswitch.conf still only says "sudoers:    files", and sudo is not working. The fix above only seems to apply to upgrades, not to fresh installs and realm joins?

Comment 15 Florence Blanc-Renaud 2018-06-15 08:18:08 UTC
Hi Martin,
you are right, the fix only applies to upgrades. We also need to make a fix on ipa side for fresh installs (in the installer, configure authselect select sssd *with-sudo*). This is why we kept 2 different BZs (this one against authselect, and 1577615 against freeipa).

Comment 16 Fedora Update System 2018-06-18 11:53:47 UTC
authselect-0.4-4.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-9622a5cc95

Comment 17 Fedora Update System 2018-06-18 17:08:49 UTC
authselect-0.4-4.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-9622a5cc95

Comment 18 Christian Heimes 2018-06-19 06:54:44 UTC
I pushed Florence's fix to FreeIPA upstream, https://pagure.io/freeipa/c/eda831dba1e09e7f4660c64756343538042b48e0

Comment 19 Pavel Březina 2018-06-19 07:30:45 UTC
Thank you.

Comment 20 Florence Blanc-Renaud 2018-06-19 13:38:50 UTC
Upgrade to authselect-0.4.4.fc28 manually tested, correctly adds the with-sudo feature.

Comment 21 Fedora Update System 2018-06-20 14:48:33 UTC
authselect-0.4-4.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 22 Martin Pitt 2018-08-27 14:02:57 UTC
Maybe upgrades are fixed, but fresh installs are not. A fresh install of Fedora 28 with updates, and authselect-0.4-4.fc28.x86_64 still has

    sudoers:    files


in nsswitch.conf by default, so that sudo does not work out of the box with FreeIPA.

Comment 23 Martin Pitt 2018-08-27 14:03:37 UTC
This is with freeipa-client-4.6.90.pre2-3.fc28.x86_64

Comment 24 Christian Heimes 2018-08-27 14:06:46 UTC
Could you try again with latest FreeIPA? 4.7.0 is GA.

Comment 25 Martin Pitt 2018-08-27 20:33:09 UTC
Whoops, that was an older image indeed, sorry about that (refreshes are stuck on getting a new Fedora Atomic out, but that's a different story).

Re-tested with authselect 1.0-1.fc28 and freeipa-client 4.7.0-1.fc28, and it still fails. In fact now "sudo" doesn't appear in /etc/nsswitch.conf at all any more. I attach that and the current ipa client install log.

Comment 26 Martin Pitt 2018-08-27 20:33:49 UTC
Created attachment 1479057 [details]
nsswitch.conf with authselect 1.0-1 and freeipa 4.7.0

Comment 27 Martin Pitt 2018-08-27 20:34:25 UTC
Created attachment 1479058 [details]
ipaclient-install.log with authselect 1.0-1 and freeipa 4.7.0

Comment 28 Martin Pitt 2018-08-27 20:36:45 UTC
From ipaclient-install.log it seems that authselect indeed initially enables sudo:

sudoers: files sss
2018-08-27T20:00:27Z INFO Configured sudoers in /etc/nsswitch.conf
[...]
2018-08-27T20:00:36Z DEBUG args=['/usr/bin/authselect', 'select', 'sssd', 'with-mkhomedir', 'with-sudo', '--force']
2018-08-27T20:00:37Z DEBUG Process finished, return code=0

but later on, sssd seems to revert it?

2018-08-27T20:00:36Z DEBUG args=['/usr/bin/authselect', 'select', 'sssd', 'with-mkhomedir', 'with-sudo', '--force']
2018-08-27T20:00:37Z DEBUG Process finished, return code=0
2018-08-27T20:00:37Z DEBUG stdout=Backup stored at /var/lib/authselect/backups/2018-08-27-20-00-37.trqwy2
Profile "sssd" was selected.
The following nsswitch maps are overwritten by the profile:
- passwd
- group
- netgroup
- automount
- services
- sudoers

Comment 29 Pavel Březina 2018-08-28 08:07:24 UTC
Can you please also pass the content of /etc/authselect/user-nsswitch.conf please?

Comment 30 Pavel Březina 2018-08-28 08:19:17 UTC
Also is /etc/nsswitch.conf symbolik link to /etc/authselect/nsswitch.conf? And what is the output of `authselect check`?

Comment 31 Martin Pitt 2018-08-28 08:22:16 UTC
Created attachment 1479170 [details]
/etc/authselect/user-nsswitch.conf with authselect 1.0-1 and freeipa 4.7.0

Attached /etc/authselect/user-nsswitch.conf . There is also no sudo rule there.

Comment 32 Martin Pitt 2018-08-28 08:28:06 UTC
It is a symbolic link:

/etc/nsswitch.conf -> /etc/authselect/nsswitch.conf

# authselect check
Current configuration is valid.

Comment 33 Pavel Březina 2018-08-28 09:03:12 UTC
OK. Thank you. And what is the output of 'authselect current'?

Comment 35 Martin Pitt 2018-08-28 09:11:50 UTC
# authselect current
Profile ID: sssd
Enabled features:
- with-mkhomedir

Comment 37 Pavel Březina 2018-08-28 10:02:14 UTC
Apparently, the current authselect configuration is really valid.

What I can see from given information is that the nsswitch.conf was generated at 4PM but ipa-client-install runs authselect at 8PM the same day. Can you check if other files under /etc/authselect were generated at the same time?

I know IPA creates backup of pre-installation state. Is it possible that the system was restored to this state?

Comment 38 Martin Pitt 2018-08-28 10:22:30 UTC
Timestamps:

[root@x0 ~]# ls -l /etc/nsswitch.conf /etc/authselect/ /var/log/ipaclient-install.log
lrwxrwxrwx. 1 root root  29 28. Aug 04:20 /etc/nsswitch.conf -> /etc/authselect/nsswitch.conf
-rw-------. 1 root root 116745 28. Aug 04:20 /var/log/ipaclient-install.log

/etc/authselect/:
total 40
-rw-r--r--. 1 root root   20 28. Aug 04:20 authselect.conf
drwxr-xr-x. 2 root root    6 14. Aug 06:32 custom
-rw-r--r--. 1 root root  195 28. Aug 04:20 dconf-db
-rw-r--r--. 1 root root  205 28. Aug 04:20 dconf-locks
-rw-r--r--. 1 root root   91 28. Aug 04:20 fingerprint-auth
-rw-r--r--. 1 root root 2574 28. Aug 04:20 nsswitch.conf
-rw-r--r--. 1 root root 2062 28. Aug 04:20 password-auth
-rw-r--r--. 1 root root  399 28. Aug 04:20 postlogin
-rw-r--r--. 1 root root   91 28. Aug 04:20 smartcard-auth
-rw-r--r--. 1 root root 2062 28. Aug 04:20 system-auth
-rw-r--r--. 1 root root 1731 27. Aug 15:56 user-nsswitch.conf

> Is it possible that the system was restored to this state?

What would do this? I didn't manually un-enroll this system or anything. E. g. I can log into it as a FreeIPA user:

# ssh -l admin localhost
[...]

[admin@x0 ~]$ id
uid=573400000(admin) gid=573400000(admins) groups=573400000(admins) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

It's just sudo that is broken:

$ sudo whoami
[sudo] password for admin: 
admin is not in the sudoers file.  This incident will be reported.

Is that not reproducible on a standard Fedora 28 VM on your side? There's nothing magic in my VM that I'm aware of..

Comment 40 Pavel Březina 2018-08-29 13:34:14 UTC
Timestamps do not match.

- Attached nsswitch.conf was generated at Mon Aug 27 16:00:48 2018 as we can see in the file header.
- ipa-client-install was called at 2018-08-27T20:00:03Z as we can see from the log.
- And output of your ls commands says that files were generated a day later at 28. Aug 04:20 

IPA server and client installation works for me well with following packages:

[root@client vagrant]# rpm -qa authselect
authselect-1.0-1.fc28.x86_64

[root@client vagrant]# rpm -qa | grep freeipa
freeipa-common-4.7.0-1.fc28.noarch
freeipa-client-common-4.7.0-1.fc28.noarch
freeipa-client-4.7.0-1.fc28.x86_64

Comment 41 Martin Pitt 2018-08-29 14:00:14 UTC
The day before (on Aug 27) I updated the VM to the latest freeipa + authselect packages, then on Aug 28 I ran the test again and collected the logs and packages. So it seems like user-nsswitch.conf actually gets created at package install/upgrade time (Aug 27), not on realm enrollment time (Aug 28)? I. e. maybe this only affects upgrades, not fresh installs?

Comment 42 Pavel Březina 2018-08-31 11:06:48 UTC
Upgrade case works for me as well.

1. install authselect-0.4-3, freeipa-client 4.6.90.pre1-6.1.fc28
2. ipa-client-install
3. dnf update authselect freeipa-client
4. authselect current
Profile ID: sssd
Enabled features:
- with-sudo

Frankly, I am now lost in all those comments. Can you please describe exactly your steps that you have taken?

Comment 43 Martin Pitt 2018-09-07 09:19:56 UTC
I now tried this again with a very recent Fedora 28, without any package upgrades (this is a clean virt-install), with

authselect-1.0-1.fc28.x86_64
freeipa-client-4.7.0-1.fc28.x86_64

The reproduction steps from the description are still valid. But I repeat them here, with more detail, for absolute clarity:

 1. Have a FreeIPA server nearby. I use a VM which serves the "COCKPIT.LAN" domain. There's nothing particularly magic or non-standard about our's, you can see how it it set up here: https://github.com/cockpit-project/cockpit/blob/master/bots/images/scripts/ipa.setup . This is called "f0.cockpit.lan".

 2. Configure the IPA server to make sudo work out of the box (see https://pagure.io/freeipa/issue/7538)

  # kinit -f admin
  # ipa sudorule-add --hostcat=all --cmdcat=all All && ipa sudorule-add-user --groups=admins All

 (This can also be done on the enrolled client, but let's do as little as possible there)

 3. Boot a current Fedora 28 machine. No custom configuration, no local "admin" user, just the stock freeipa-client and authselect packages.

  # id admin
  id: ‘admin’: no such user


 4. Make sure it's in the same DNS domain:

  # hostnamectl set-hostname x0.cockpit.lan

 5. Enroll into domain:

  # printf '[cockpit.lan]\nfully-qualified-names = no\n'  >> /etc/realmd.conf
  (Workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1575538)

  # realm join -vU admin cockpit.lan
  (Type admin password, everything else should be automatic)

 6. Wait until everything actually works (after the above command it won't, there's still stuff happening in the background). These should succeed:

  # while ! id admin; do sleep 5; done  # this might take a minute or two
  # ssh -l admin localhost  # this might also not succeed at the first time

  Now you are logged in as "admin" FreeIPA user.

 7. Check nsswitch:

 [admin@x0 /]$ grep sudo /etc/nsswitch.conf
 (No hits)

 8. Try to sudo:

 [admin@x0 /]$ sudo whoami
 [sudo] password for admin: 
 admin is not in the sudoers file.  This incident will be reported.


You can do the same without the "fully-qualified-names = no" change from above, and then use "admin@localhost" as user name. Same result. This can be fixed with running "authselect enable-feature with-sudo" after joining, which will set up nsswitch correctly.
  

I now modified step 5 to not use "realm join", but

  # ipa-client-install
  (Confirm defaults, user "admin", type your password, everything else should be automatic)

And indeed it works now. So it seems this is a bug in conjunction with realmd. I'll file a new bug there instead.

Comment 44 Martin Pitt 2018-09-07 09:23:28 UTC
See bug 1620097. Thanks!


Note You need to log in before you can comment on or make changes to this bug.