Bug 1261023

Summary: login via ssh as remote user often fails with: "Write failed: Broken pipe"
Product: Red Hat Enterprise Linux 7 Reporter: Patrik Kis <pkis>
Component: sssdAssignee: SSSD Maintainers <sssd-maint>
Status: CLOSED DUPLICATE QA Contact: Kaushik Banerjee <kbanerje>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.2CC: codonell, fweimer, grajaiya, jgalipea, jhrozek, lslebodn, mkosek, mzidek, pbrezina, preichl, sbose, sgoveas
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-15 13:56:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
strace of sshd child none

Description Patrik Kis 2015-09-08 12:46:56 UTC 
Description of problem:
Login attempts via ssh as remote user (AD or IPA) often fails with:

# ssh <user>@<domain>@localhost
<user>@<domain>@localhost's password: 
Write failed: Broken pipe

The problem was observed on the latest RHEL-7.2 test composes; not sure the issue is caused directly by sssd, but I'm clueless (sssd debugs will be attached). The problems were newer seen on RHEL-7.1 machines. The issue appears only time to time, usually after sssd restart. When join to AD is via winbind the problem does not appear.

Version-Release number of selected component (if applicable):
sssd-1.13.0-26.el7
openssh-6.6.1p1-18.el7
realmd-0.16.1-3.el7
pam-1.1.8-12.el7_1.1

How reproducible:
~50%

Steps to Reproduce:
1. join to AD with realmd
2. try to login via ssh
3. if login succeed try to restart sssd and/or join again

Actual results:
ssh often fails

Expected results:
ssh is reliable

Additional info:
Comment 2 Sumit Bose 2015-09-08 14:03:03 UTC 
Both authentication and access control part look good in the SSSD logs. Can you add pam and nss responder logs and sshd logs with a high debug level as welll?
Comment 3 Lukas Slebodnik 2015-09-08 14:07:30 UTC 
It would be also good to attach /var/log/secure
Comment 9 Sumit Bose 2015-09-10 12:50:41 UTC 
Created attachment 1072168 [details]
strace of sshd child

Thank you for providing the test environment, it was very useful. I used strace to see what sshd is doing and found a "corrupted double-linked list" error in one of the sshd child processes.

Currently I cannot make sense of the output so I attached the file for others to check.
Comment 10 Jakub Hrozek 2015-09-11 12:45:02 UTC 
(In reply to Sumit Bose from comment #9)
> Created attachment 1072168 [details]
> strace of sshd child
> 
> Thank you for providing the test environment, it was very useful. I used
> strace to see what sshd is doing and found a "corrupted double-linked list"
> error in one of the sshd child processes.
> 
> Currently I cannot make sense of the output so I attached the file for
> others to check.

Thanks a lot for looking into this. 

Do you think it's time to reassign this bugzilla to sshd, then?
Comment 13 Florian Weimer 2015-09-14 15:45:53 UTC 
We have tracked this down to a security bug, bug 1262914 in glibc.
Comment 19 Florian Weimer 2015-09-15 13:56:01 UTC 
We have confirmed that this issue is indeed caused by glibc bug 1262914, based on further analysis of the issue.  After installing a glibc scratch build with the upstream fix applied, the original sssd issue is no longer reproducible.

I'm closing this as a duplicate of the (internal) glibc 7.2 bug for this issue.

*** This bug has been marked as a duplicate of bug 1263134 ***