Bug 1065009

/var/log/secure:

Feb 13 11:55:12 localhost gdm-password]: pam_unix(gdm-password:auth): authentication failure; logname=(unknown) uid=0 euid=0 tty=:0 ruser= rhost=  user=aduser1
Feb 13 11:55:13 localhost gdm-password]: pam_sss(gdm-password:auth): authentication success; logname=(unknown) uid=0 euid=0 tty=:0 ruser= rhost= user=aduser1
Feb 13 11:55:13 localhost gdm-password]: pam_unix(gdm-password:session): session opened for user aduser1 by (unknown)(uid=0)
Feb 13 11:55:15 localhost polkitd[722]: Unregistered Authentication Agent for unix-session:c7 (system bus name :1.521, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Feb 13 11:55:15 localhost gdm-launch-environment]: pam_unix(gdm-launch-environment:session): session closed for user gdm
Feb 13 11:55:16 localhost gdm-password]: pam_unix(gdm-password:session): session closed for user aduser1
Feb 13 11:55:17 localhost gdm-launch-environment]: pam_unix(gdm-launch-environment:session): session opened for user gdm by aduser1(uid=0)
Feb 13 11:55:18 localhost polkitd[722]: Registered Authentication Agent for unix-session:c8 (system bus name :1.551 [gnome-shell --mode=gdm], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)


/var/log/messages:

Feb 13 11:55:12 localhost dbus-daemon: dbus[666]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper)
Feb 13 11:55:12 localhost dbus[666]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper)
Feb 13 11:55:13 localhost dbus-daemon: dbus[666]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd'
Feb 13 11:55:13 localhost dbus[666]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd'
Feb 13 11:55:13 localhost setroubleshoot: Plugin Exception restorecon
Feb 13 11:55:13 localhost setroubleshoot: SELinux is preventing /usr/libexec/sssd/krb5_child from 'read, open' accesses on the file . For complete SELinux messages. run sealert -l e6790be0-cb60-405a-b73a-fd00996543dd
Feb 13 11:55:13 localhost setroubleshoot: SELinux is preventing /usr/libexec/sssd/krb5_child from write access on the file . For complete SELinux messages. run sealert -l d78722f2-1652-4523-930a-3a5a524a873a
Feb 13 11:55:13 localhost gdm: could not save session and language settings
Feb 13 11:55:13 localhost systemd: Starting user-188801197.slice.
Feb 13 11:55:13 localhost systemd: Created slice user-188801197.slice.
Feb 13 11:55:13 localhost systemd: Starting Session 92 of user aduser1.
Feb 13 11:55:13 localhost systemd: Started Session 92 of user aduser1.
Feb 13 11:55:13 localhost systemd-logind: New session 92 of user aduser1.
Feb 13 11:55:13 localhost systemd-logind: Linked /tmp/.X11-unix/X0 to /run/user/188801197/X11-display.
Feb 13 11:55:13 localhost gdm: AccountsService: ActUserManager: user (null) has no username (uid: 0)
Feb 13 11:55:14 localhost gdm: AccountsService: ActUserManager: user (null) has no username (uid: 0)
Feb 13 11:55:14 localhost gdm: AccountsService: ActUserManager: user (null) has no username (uid: 0)
Feb 13 11:55:15 localhost colord: device removed: xrandr-VGA-0
Feb 13 11:55:15 localhost gdm: Child process -22449 was already dead.
Feb 13 11:55:15 localhost systemd-logind: Removed session c7.
Feb 13 11:55:16 localhost gnome-session[23255]: ERROR: Failed to connect to system bus: Exhausted all available authentication mechanisms (tried: EXTERNAL, DBUS_COOKIE_SHA1, ANONYMOUS) (available: EXTERNAL, DBUS_COOKIE_SHA1, ANONYMOUS)
aborting...
Feb 13 11:55:16 localhost kernel: [39259.648100] traps: gnome-session[23255] trap int3 ip:7f7e1ac03ccd sp:7fff278df560 error:0
Feb 13 11:55:16 localhost abrt-hook-ccpp: Saved core dump of pid 23255 (/usr/bin/gnome-session) to /var/tmp/abrt/ccpp-2014-02-13-22:25:16-23255 (2162688 bytes)
Feb 13 11:55:16 localhost gdm: GLib-GObject: g_object_ref: assertion `object->ref_count > 0' failed
Feb 13 11:55:16 localhost gdm: GLib-GObject: g_object_unref: assertion `object->ref_count > 0' failed
Feb 13 11:55:16 localhost dbus-daemon: dbus[666]: [system] Rejected send message, 1 matched rules; type="method_call", sender=":1.25" (uid=0 pid=11920 comm="/usr/sbin/gdm ") interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply="0" destination=":1.543" (uid=0 pid=23473 comm="/usr/libexec/gdm-simple-slave --display-id /org/gn")
Feb 13 11:55:16 localhost dbus[666]: [system] Rejected send message, 1 matched rules; type="method_call", sender=":1.25" (uid=0 pid=11920 comm="/usr/sbin/gdm ") interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply="0" destination=":1.543" (uid=0 pid=23473 comm="/usr/libexec/gdm-simple-slave --display-id /org/gn")
Feb 13 11:55:17 localhost gdm: AccountsService: SetLanguage call failed: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface `org.freedesktop.Accounts.User' on object at path /org/freedesktop/Accounts/User42
Feb 13 11:55:17 localhost systemd: Starting Session c8 of user gdm.
Feb 13 11:55:17 localhost systemd-logind: New session c8 of user gdm.
Feb 13 11:55:17 localhost systemd: Started Session c8 of user gdm.
Feb 13 11:55:17 localhost dbus-daemon: dbus[666]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service'
Feb 13 11:55:17 localhost dbus[666]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service'
Feb 13 11:55:17 localhost systemd: Starting Locale Service...
Feb 13 11:55:17 localhost dbus-daemon: dbus[666]: [system] Successfully activated service 'org.freedesktop.locale1'
Feb 13 11:55:17 localhost dbus[666]: [system] Successfully activated service 'org.freedesktop.locale1'
Feb 13 11:55:17 localhost systemd: Started Locale Service.
Feb 13 11:55:17 localhost colord: Device added: xrandr-VGA-0
Feb 13 11:55:23 localhost dbus-daemon: string index out of range
Feb 13 11:55:25 localhost systemd-logind: Removed session 92.
Feb 13 11:55:25 localhost systemd: Stopping user-188801197.slice.
Feb 13 11:55:25 localhost systemd: Removed slice user-188801197.slice.
Feb 13 11:55:30 localhost abrt-server: Generating core_backtrace
Feb 13 11:55:30 localhost dbus-daemon: dbus[666]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)
Feb 13 11:55:30 localhost dbus[666]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)
Feb 13 11:55:30 localhost dbus-daemon: dbus[666]: [system] Successfully activated service 'org.freedesktop.problems'
Feb 13 11:55:30 localhost dbus[666]: [system] Successfully activated service 'org.freedesktop.problems'
Feb 13 11:55:30 localhost abrt-server: Sending an email...
Feb 13 11:55:30 localhost abrt-server: Email was sent to: seceng-idm-qe-list

Created attachment 862903 [details]
abrt crash report

so this is the key entry in the log:
ERROR: Failed to connect to system bus: Exhausted all available authentication mechanisms (tried: EXTERNAL, DBUS_COOKIE_SHA1, ANONYMOUS) (available: EXTERNAL, DBUS_COOKIE_SHA1, ANONYMOUS)
aborting...

presumably either a dbus or sssd issue.

Likely an sssd issue - the system bus eventually calls into getpwuid_r() on the passed username, and if it fails to look that up, it will reject EXTERNAL.

sssd seems to be working.  IPA and AD users are able to ssh into the host itself.  I can also getent the users:

[root@rhel7client abrt]# getent passwd scott
scott:*:1875600050:1875600050:f l:/home/scott:/bin/sh

[root@rhel7client abrt]# getent passwd aduser1.com
aduser1.com:*:1389001104:1389001104::/home/subdom.ipaqe.com/aduser1:


Per halfline's suggestion, I restarted messagebus:

systemctl restart messagebus

This killed my login session as expected but, I had to also restart GDM to get the login screen back:

systemctl restart gdm

At this point, I'm able to login as AD user.

so i talked with spoore a bit on irc. a few more notes:

1) local users work
2) IPA and AD users don't
3) The system hasn't been rebooted since SSSD client configuration was performed.

This almost sounds like the libc nsswitch.conf caching bug but as I understand things, sss is always in nsswitch.conf now to address that issue.

So I assume it's some sort of bug in sssd.  reassigning again.

(i posted comment 8 at the same time comment 6 and comment 7 were written. there was a midair collision)

the libc caching bug i alluded to in comment 8 is https://sourceware.org/bugzilla/show_bug.cgi?id=12459

so walters noticed this in the nsswitch code:

https://sourceware.org/git/?p=glibc.git;a=blob;f=nss/nsswitch.c;h=07ca2252321c57e98023b87f129d9a4acddf9055;hb=HEAD#l358

in other words having sss in /etc/nsswitch.conf isn't sufficient. dlopen() failures are cached.  So there's really nothing we can do without libc changes (short of fork()/exec() a helper to call getpwuid)

moving to libc.

(In reply to Ray Strode [halfline] from comment #11)
> so walters noticed this in the nsswitch code:
> 
> https://sourceware.org/git/?p=glibc.git;a=blob;f=nss/nsswitch.c;
> h=07ca2252321c57e98023b87f129d9a4acddf9055;hb=HEAD#l358
> 
> in other words having sss in /etc/nsswitch.conf isn't sufficient. dlopen()
> failures are cached.  So there's really nothing we can do without libc
> changes (short of fork()/exec() a helper to call getpwuid)
> 
> moving to libc.

The only workaround I can suggest is "Have sss installed first."

This is an RFE and won't be easily fixable since this has to go upstream first.

We need a new syntax to express that the particular service may not be installed and should be retried on *every* request to that service (expensive). We might make this less expensive by using inotify to detect that the service provider plugin (shared library) has appeared.

With the existing behaviour being "try this service once at startup and never again if not installed."

Would something like that suffice for everyone?

In general I think that testing for installed services is important. One should not have to restart all dependent processes to get access to a new service. One should also not force the application to issue a new API call to test for the service (though that might be useful given the old syntax and a configuration that is setup for optimal performance).

I also think we should be using inotify to monitor /etc/nsswitch.conf and reloading, but that's another bug.

I'm putting this into the glibc backlog.

Works for me.  For our particular troubles with IPA on desktop, we've got the workarounds to keep us going.

Thanks,
Scott

jhrozek, what would you think about splitting libnss_sss out into a subpackage and making it installed by default even when sssd isn't installed?

(In reply to Ray Strode [halfline] from comment #15)
> jhrozek, what would you think about splitting libnss_sss out into a
> subpackage and making it installed by default even when sssd isn't installed?

It's already outside the SSSD daemon package:

$ rpm -qf /lib64/libnss_sss.so.2
sssd-client-1.11.90-0.fc20.x86_64

The sssd-client package is not really huge at this point, but if you'd prefer to make it even smaller, then why not. We actually received a request to release the client libraries as a separate tarball not long ago..

Currently the sssd-client package contains:
$ rpm -ql sssd-client-1.11.90-0.fc20.x86_64
/etc/cifs-utils/idmap-plugin
/lib64/libnss_sss.so.2
/lib64/security/pam_sss.so
/usr/lib64/cifs-utils/cifs_idmap_sss.so
/usr/lib64/krb5/plugins/authdata/sssd_pac_plugin.so
/usr/lib64/krb5/plugins/libkrb5/sssd_krb5_locator_plugin.so
/usr/share/doc/sssd-client
/usr/share/doc/sssd-client/COPYING
/usr/share/doc/sssd-client/COPYING.LESSER
/usr/share/man/man8/pam_sss.8.gz
/usr/share/man/man8/sssd_krb5_locator_plugin.8.gz

Any movement on this.  I want to get sssd-client as a default package in Docker Container Images.

(In reply to Daniel Walsh from comment #19)
> Any movement on this.  I want to get sssd-client as a default package in
> Docker Container Images.
Yes:

    Bug 1065374 Comment 3 Michal Kovarik 2014-03-20 05:12:33 EDT

    Verified on RHEL-7.0-20140317.0. Package sssd-client is part of base group.

Though, maybe docker container images use core and not base?

Red Hat Enterprise Linux 7 is going to enter Maintenance Support Phase 1 during 2019, and this change means that our customers are looking forward to a stable release without the kinds of changes that this auto-reloading might imply. The lead time to make this change is going to be quite long, we will have to enable this upstream, test it thoroughly for a time with various workloads (enabled by default), and then after enough experience decide if it's ready to backport to a stable RHEL release. All of this work rules out this change for RHEL 7, but we will continue this work as part of the RHEL 8 request in bug 132608. Therefore I'm closing this RFE in favour of bug 132608. Lastly, we understand that this kind of feature is important and will continue investigating how to best implement it upstream.

*** This bug has been marked as a duplicate of bug 132608 ***