Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1663445

Summary:	nscd when running results in pam module getting garbage value for password - #010#012#015#177INCORRECT
Product:	Red Hat Enterprise Linux 6	Reporter:	Hareesh Joshi <hareesh.joshi>
Component:	glibc	Assignee:	glibc team <glibc-bugzilla>
Status:	CLOSED NOTABUG	QA Contact:	qe-baseos-tools-bugs
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.7	CC:	ashankar, codonell, dj, fweimer, hareesh.joshi, jkachuck, mnewsome, pfrankli
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Other
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-01-15 17:35:23 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Hareesh Joshi 2019-01-04 11:21:12 UTC

Description of problem:

I've implemented a PAM module to authenticate non OS users via SSH. 
When a non OS user tries to login, the PAM module kicks in and authenticates the user based on the logic that I've written. 
The username is read from the SSH prompt and password is also read from the same prompt. 
This works fine until I install/start nscd service. 

When the nscd service is running, the password that the PAM module tries to fetch from the ssh prompt turns out to be #010#012#015#177INCORRECT always.
Hence the login fails as the user entered password in ignored. 

But the OS user logins like root etc, work fine. 

Version-Release number of selected component (if applicable):


How reproducible:

Steps to Reproduce:

1. Implement a PAM module to authenticate non-os-user like "ApplicationAdmin" with username and password
2. In the PAM module log the username and password received using pam_syslog
3. Install start nscd 
4. Try to login as "ApplicationAdmin"/<urpassword>
5. With the help of log in step 2, you can see that <urpassword> is not what you expect it to be. It will be #010#012#015#177INCORRECT

Actual results:
User should be able to login if the "ApplicationAdmin"/<urpassword> are correct.

Expected results:
User fails to login even if the credentials are right.

Additional info:
UsePAM in sshd_conf is Yes
/etc/nsswitch.conf has is as expected.

Comment 2 Florian Weimer 2019-01-04 11:37:14 UTC

Can you reproduce this issue just with Red Hat Enterprise Linux system components?  Thanks.

Comment 3 Hareesh Joshi 2019-01-04 11:57:54 UTC

None are non RHEL components. 
Should I disable the PAM implementation?

Comment 4 Joseph Kachuck 2019-01-04 14:39:05 UTC

Hello Hareesh,
Are you able to do this on RHEL 7 or RHEL 8?
Which PAM module are you implementing?

Have you been able to do this previously?

If you are looking for steps on how to complete this task. Please open a support case on access.redhat.com.
If you are stating this is a bug. Please confirm all steps/commands entered to recreate this issue.

Thank You
Joe Kachuck

Comment 5 Hareesh Joshi 2019-01-04 16:12:23 UTC

Hi Joe,

Thanks for looking into this.
I'm using CentOS6.7 which is based on RHEL. I've implemented sshd pam module.

I didn't have any problem with this module until I installed nscd service.

I'm stating that this is a defect with nscd service which is interfering with the password read operation of pam module.
I've listed the steps above on how to reproduce the issue. The following more detailed steps might help you better -

1. Implement a pam module say, pam_myapp.c which provides implementation for pam_sm_authenticate, pam_sm_acct_mgmt.
2. Deploy the module in /lib64/security as a shared object say - pam_myapp.so
3. Update the /etc/pam.d/sshd config file to include pam_myapp.so under auth, account, password and session module interfaces
4. Try to login with say, "ApplicationAdmin" as the username and password say "LetMeIn". Not that, ApplicationAdmin is not an OS user, i.e., password/shadow files have
no idea about this user. At this point, pam_myapp kicks in and authenticates the user based on the username/password provided.
5. This works fine as long as the username is "ApplicationAdmin" and password in "LetMeIn".
6. If the user tries to login as say, root, pam_myapp authentication fails and then password-auth kicks in and regular sshd login continues.
7. Now install and start nscd service
8. Attempt to login as ApplicationAdmin/LetMeIn via ssh. The login fails.
9. Enabled logs in the pam module to see if username and password are passed appropriately to the pam module. I see that password passed is #010#012#015#177INCORRECT instead of LetMeIn.

Hence pam_myapp fails to authenticate the user even though the user provided proper creds.

If I stop nscd service or disable password caching in nscd (using /etc/nscd.conf) the password is passed appropriately to the pam module.

Comment 6 Joseph Kachuck 2019-01-04 17:28:49 UTC

Hello Hareesh,
Are you able to recreate this issue on RHEL 7 or 8?
CentOS while based on RHEL is not a Red Hat supported release.

Thank You
Joe Kachuckhareesh.joshi

Comment 7 Hareesh Joshi 2019-01-04 17:37:40 UTC

Hi Joe,

Yeah, I understand you on CentOS aspect. 
While I haven't tried on RHEL 7 or 8, I'm hoping that glibc team would look into this issue as to why nscd is interfering with pam.

Comment 8 Carlos O'Donell 2019-01-04 18:38:32 UTC

(In reply to Hareesh Joshi from comment #7)
> I'm hoping that glibc team would look
> into this issue as to why nscd is interfering with pam.

I do not think this is nscd.

This OpenSSH developer list post suggests you are running into an OpenSSH issue:
"Bad Password - #010#012#015#177INCORRECT : ssh -> pam -> libpam_sqlite -> sqlite3"
https://lists.mindrot.org/pipermail/openssh-unix-dev/2014-March/032283.html

You need an NSS service provider to provide the "Identity" information for the user.

Please tell me if that helps.

Comment 9 Hareesh Joshi 2019-01-04 19:22:19 UTC

Yes Carlos, I did come across the link you quoted. 
And as I mentioned in the defect description earlier, I do have nsswitch.conf modified appropriately.

Here's the excerpt - 

  passwd:     files myapp
  shadow:     files 
  group:      files


>>I do not think this is nscd. 
  Curious to know - why does it work perfectly fine on disabling nscd service. 

>>You need an NSS service provider to provide the "Identity" information for the user.
  I think passwd deals with Identity. Should I have shadow field modified as well?


Thank you for looking into this.

Comment 10 Carlos O'Donell 2019-01-04 19:29:14 UTC

(In reply to Hareesh Joshi from comment #9)
> Yes Carlos, I did come across the link you quoted. 
> And as I mentioned in the defect description earlier, I do have
> nsswitch.conf modified appropriately.
> 
> Here's the excerpt - 
> 
>   passwd:     files myapp
>   shadow:     files 
>   group:      files

That doesn't include gid, and so fails to provide a complete identity.

> >>I do not think this is nscd. 
>   Curious to know - why does it work perfectly fine on disabling nscd
> service. 

I don't know. You would need to do a root cause analysis.

When you don't use nscd the NSS call is made by the process itself and so the credentials of the process might be used.

In the case of nscd the process is isolated and hardened and so is distinct from the process requesting the information.

This difference might matter.

> >>You need an NSS service provider to provide the "Identity" information for the user.
>   I think passwd deals with Identity. Should I have shadow field modified as
> well?

I would expect passwd, shadow, and group all need to be handled or you will have odd behaviour from applications that are using the normative APIs to lookup user information.

Comment 11 Carlos O'Donell 2019-01-15 17:35:23 UTC

Given that this is not a glibc issue I'm closing this as CLOSED/NOTABUG. Thank you for submitting the bug. Please reopen if you complete a root cause analysis and find a defect in the glibc component.