Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2117169

Summary: ChallengeResponseAuthentication / KbdInteractiveAuthentication is set to yes, pam_env outcomes not being applied to AD users via pam_sss
Product: Red Hat Enterprise Linux 8 Reporter: Quynh Anh Pham <qpham>
Component: opensshAssignee: Dmitry Belyavskiy <dbelyavs>
Status: CLOSED MIGRATED QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: high Docs Contact:
Priority: medium    
Version: 8.6CC: gregor.scott, jjelen
Target Milestone: rcKeywords: MigratedToJIRA, Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-19 20:53:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Quynh Anh Pham 2022-08-10 08:34:01 UTC
Description of problem:
OpenSSH 8.0p1-13 installed

with ChallengeResponseAuthentication set to yes in /etc/ssh/sshd_config

with sss configured for AD authentication

with pam_env entries in the pam setup


that the environment variables assigned via pam_env are not assigned the authorised user shell.

Version-Release number of selected component (if applicable):
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021

How reproducible:
Always

Steps to Reproduce:

Customer goal: What can I do to ensure that the environment command is only run for users that authenticate via sss, regardless of which protocol is used?

1. With ChallengeResponseAuthentication in /etc/ssh/sshd_config set to yes
1.1. Test 1 - confirm sshd sssd authentication updates environment variables
I setup the auth section of password-auth to look like this:


auth        required                                     pam_faildelay.so delay=2000000
# for users that authenticate via sssd
auth        required                                     pam_env.so conffile=/etc/security/pam_env_sssd.conf debug
auth        sufficient                                   pam_sss.so forward_pass domains=example.com
auth        required                                     pam_deny.so debug
Result from Test 1

the environment variables are set

as expected, only sssd-authentication works


1.2. Test 2 - make the sssd the first auth attempt
The auth section of password-auth was setup like this:


auth        required                                     pam_faildelay.so delay=2000000
auth        [success=2 default=ignore]                   pam_succeed_if.so debug user notingroup examplegroup
# for users that authenticate via sssd
auth        required                                     pam_env.so conffile=/etc/security/pam_env_sssd.conf debug
auth        sufficient                                   pam_sss.so forward_pass domains=example.com
# for users that authenticate locally
auth        [success=1 default=ignore]                   pam_usertype.so issystem
auth        [success=ok ignore=ignore default=1]         pam_localuser.so
auth        sufficient                                   pam_unix.so nullok

auth        required                                     pam_deny.so debug
Result from Test 2

All ssh authentications (both sssd and local) get the env variables set - not what was expected

Only telnet sssd authentications get the env variables set - exactly as expected


1.3. Test 3 - try a fall-through setup with pam_unix tried first
The auth section of password-auth was setup like this:


auth        required                                     pam_faildelay.so delay=2000000
# for users that authenticate locally
auth        sufficient                                   pam_unix.so nullok
# for users that authenticate via sssd
auth        required                                     pam_env.so conffile=/etc/security/pam_env_sssd.conf debug
auth        sufficient                                   pam_sss.so forward_pass domains=example.com
auth        required                                     pam_deny.so debug
Result from Test 3

No local auth (via ssh or telnet) have the environment variables set - as expected

The sssd ssh authentication does not set the environment variables - not what was expected

The sssd telnet authentication does set the environment variables - as expected

2. I made the requested changes to the /etc/ssh/sshd_config file


# grep -E "^PasswordAuthentication|^KbdInteractiveAuthentication" /etc/ssh/sshd_config
PasswordAuthentication yes
KbdInteractiveAuthentication no

I had to _add_ the setting KbdInteractiveAuthentication to the /etc/ssh/sshd_config file as it was not there.


2.2. After restarting sshd the setting for KbdInteractiveAuthentication was still yes:


# sshd -T|grep authen
hostbasedauthentication no
pubkeyauthentication yes
kerberosauthentication no
gssapiauthentication yes
passwordauthentication yes
kbdinteractiveauthentication yes
challengeresponseauthentication yes
authenticationmethods any

So despite adding it to the conf file the running sshd daemon does not reflect the setting, which tells me that it cannot be directly set.


2.3. The man entry for sshd_config indicates that KbdInteractiveAuthentication defaults to ChallengeResponseAuthentication.
So I removed the KbdInteractiveAuthentication  entry that I added and changed ChallengeResponseAuthentication to be no.


# grep -E "^PasswordAuthentication|^KbdInteractiveAuthentication|^ChallengeResponseAuthentication" /etc/ssh/sshd_config
PasswordAuthentication yes
ChallengeResponseAuthentication no


2.4. After re-starting sshd the running daemon now reports these authentication settings:


# sshd -T|grep authen
hostbasedauthentication no
pubkeyauthentication yes
kerberosauthentication no
gssapiauthentication yes
passwordauthentication yes
kbdinteractiveauthentication no
challengeresponseauthentication no
authenticationmethods any

So the KbdInteractiveAuthentication setting can only be adjusted by changing ChallengeResponseAuthentication in the conf file.


2.5. Logging in as user exampleuser using AD credentials results in the pam_env activity showing up in the /var/log/secure log file (as expected):


Aug  2 11:19:02 hostname sshd[7245]: Connection from <IP address> port 30302 on <IP address> port 22
Aug  2 11:19:07 hostname sshd[7245]: Failed publickey for exampleuser from <IP address> port 30302 ssh2: RSA SHA256:UC8pVsdsEtrR5YjFeXNof
Aug  2 11:19:11 hostname unix_chkpwd[7249]: password check failed for user (exampleuser)
Aug  2 11:19:11 hostname sshd[7245]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=<IP address>  user=exampleuser
Aug  2 11:19:11 hostname sshd[7245]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=<IP address> user=exampleuser
Aug  2 11:19:11 hostname sshd[7245]: Accepted password for exampleuser from <IP address> port 30302 ssh2
Aug  2 11:19:11 hostname sshd[7245]: pam_env(sshd:setcred): pam_putenv("AUTH_USER=exampleuser")
Aug  2 11:19:11 hostname sshd[7245]: pam_env(sshd:setcred): pam_putenv("AUTH_ENV=sssd")
Aug  2 11:19:11 hostname systemd[7258]: pam_unix(systemd-user:session): session opened for user exampleuser by (uid=0)
Aug  2 11:19:11 hostname sshd[7245]: pam_unix(sshd:session): session opened for user exampleuser by (uid=0)
Aug  2 11:19:11 hostname sshd[7245]: User child is on pid 7267
Aug  2 11:19:11 hostname sshd[7267]: pam_env(sshd:setcred): pam_putenv("AUTH_USER=exampleuser")
Aug  2 11:19:11 hostname sshd[7267]: pam_env(sshd:setcred): pam_putenv("AUTH_ENV=sssd")
Aug  2 11:19:11 hostname sshd[7267]: Starting session: shell on pts/2 for exampleuser from <IP address> port 30302 id 0
So disabling the ChallengeResponseAuthentication setting allows the pam_env updates to proceed.


This is good, and bad.


Without the ChallengeResponseAuthentication setting our attempts to use Google Authenticator for 2FA are broken.


3. I have been able to get both the pam_env environment variables being set for appropriate users as well as enabling the google authenticator by configuring the sshd_config file with these settings:


# main settings section
PasswordAuthentication yes
ChallengeResponseAuthentication yes

# Match section
Match Group era User *,!rr
    KbdInteractiveAuthentication no
The members of the unix user group  all need to have the environment variables defined IF they trigger the pam_sss.so module, and for that to happen the KbdInteractiveAuthentication setting (which appears to be the equivalent of the ChallengeResponseAuthentication entry in the main settings area).


With this in place, the appropriate users who need google authenticator 2FA get it, and those that need environment variables defined for the pam_sss module get them set as well.


I find the relationship between KbdInteractiveAuthentication and ChallengeResponseAuthentication very confusing.

Actual results:
there is a real problem within sshd when ChallengeResponseAuthentication / KbdInteractiveAuthentication is set to yes and the pam stack includes pam_env and pam_sss, resulting in the pam_env outcomes not being applied to the resulting user shell.

Expected results:
ChallengeResponseAuthentication / KbdInteractiveAuthentication option in sshd_config should not affect pam_env outcomes for AD users authenticated via pam_sss

Additional info:

Comment 1 Dmitry Belyavskiy 2022-09-08 12:21:12 UTC
May I ask you to narrow the scope of the issue? It's very difficult to reproduce the bug according to the current description

Comment 2 Quynh Anh Pham 2022-09-13 01:46:35 UTC
Dear Dmitry,

This is the main issue here:

1.3. Test 3 - try a fall-through setup with pam_unix tried first
The auth section of password-auth was setup like this:


auth        required                                     pam_faildelay.so delay=2000000
# for users that authenticate locally
auth        sufficient                                   pam_unix.so nullok
# for users that authenticate via sssd
auth        required                                     pam_env.so conffile=/etc/security/pam_env_sssd.conf debug
auth        sufficient                                   pam_sss.so forward_pass domains=example.com
auth        required                                     pam_deny.so debug
Result from Test 3

No local auth (via ssh or telnet) have the environment variables set - as expected

The sssd ssh authentication does not set the environment variables - not what was expected  ==> the main issue

The sssd telnet authentication does set the environment variables - as expected


==> work around that we found:

3. I have been able to get both the pam_env environment variables being set for appropriate users as well as enabling the google authenticator by configuring the sshd_config file with these settings:


# main settings section
PasswordAuthentication yes
ChallengeResponseAuthentication yes

# Match section
Match Group era User *,!rr
    KbdInteractiveAuthentication no
The members of the unix user group  all need to have the environment variables defined IF they trigger the pam_sss.so module, and for that to happen the KbdInteractiveAuthentication setting (which appears to be the equivalent of the ChallengeResponseAuthentication entry in the main settings area).

Comment 3 RHEL Program Management 2023-09-19 20:48:12 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 4 RHEL Program Management 2023-09-19 20:53:13 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.