Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1755986

Summary: openssh-server functionality is broken without `setup` package: Privilege separation user sshd does not exist
Product: Red Hat Enterprise Linux 8 Reporter: Andrei Stepanov <astepano>
Component: dnfAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED NOTABUG QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.4CC: ashankar, codonell, dj, fweimer, james.antill, mnewsome, pfrankli, tmraz
Target Milestone: rc   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1757267 (view as bug list) Environment:
Last Closed: 2019-09-30 14:09:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Andrei Stepanov 2019-09-26 14:29:30 UTC
sshd server misbehave without `setup` package

openssh-server-8.0p1-3.el8.x86_64

How reproducible: always

Steps to Reproduce:
1. yum remove -y setup

After this package `sshd` service cannot be started/restared.
Current running server refuses new connections with:

> ssh -vv root.137.174
OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/andrei/.ssh/config
debug1: /Users/andrei/.ssh/config line 71: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug2: resolve_canonicalize: hostname 10.0.137.174 is address
debug2: ssh_connect_direct
debug1: Connecting to 10.0.137.174 [10.0.137.174] port 22.
debug1: Connection established.
debug1: identity file /Users/andrei/.ssh/id_rsa type 0
debug1: identity file /Users/andrei/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9
ssh_exchange_identification: read: Connection reset by peer


Actual results:

[root@ci-vm-10-0-137-174 ~]# journalctl -u sshd
Sep 26 10:14:44 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: Server listening on 0.0.0.0 port 22.
Sep 26 10:14:44 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug2: fd 6 setting O_NONBLOCK
Sep 26 10:14:44 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug3: sock_set_v6only: set socket 6 IPV6_V6ONLY
Sep 26 10:14:44 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug1: Bind to port 22 on ::.
Sep 26 10:14:44 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: Server listening on :: port 22.
Sep 26 10:14:44 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com systemd[1]: Started OpenSSH server daemon.
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug3: fd 7 is not O_NONBLOCK
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug1: Forked child 5275.
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug3: send_rexec_state: entering fd = 10 config len 752
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug3: ssh_msg_send: type 0
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[4824]: debug3: send_rexec_state: done
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[5275]: debug3: oom_adjust_restore
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[5275]: debug1: Set /proc/self/oom_score_adj to 0
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[5275]: debug1: rexec start in 7 out 7 newsock 7 pipe 9 sock 10
Sep 26 10:15:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[5275]: fatal: Privilege separation user sshd does not exist

Comment 1 Dave Cantrell 2019-09-26 14:35:58 UTC
Through some trial and error, we found that removing 'setup' obviously removes /etc/passwd and /etc/shadow which seem to cause havoc with sshd.  My personal thought on this is that openssh should carry a Requires on 'setup' or on the specific files it needs in /etc at a bare minimum.  It seems extraneous, but it's important from a testing standpoint and simulating what users do when they minimize VM images (for example).

Comment 2 Tomas Mraz 2019-09-26 15:09:03 UTC
Does it really make sense? I mean removing /etc/passwd basically means no normal user resolution will work. Should we have dependency on setup for anything that uses user or group names?

On the other hand in theory if nsswitch.conf does not contain files for passwd and group all the users could be stored and resolved by other mechanism and then depending on setup and/or the existence of /etc/passwd and /etc/shadow actually does not make sense at all.

Really just arbitrarily adding dependency on setup into openssh-server does not make much sense to me.

Comment 3 Jakub Jelen 2019-09-26 15:32:20 UTC
Can you provide more information what leads to you removing setup package?

The report as is stated now does not sound like something that would make sense to fix. The user sshd is explicitly stated in the spec file to be created on the installation. If you basically break the system afterward by removing this user, it is nothing sshd can fix, except for adding the explicit dependency. But if you would like to go that way (system-user = dependency on setup package), I would suggest you to start with Fedora and packaging guidelines:

https://docs.fedoraproject.org/en-US/packaging-guidelines/

Comment 4 Andrei Stepanov 2019-09-26 16:05:26 UTC
> Can you provide more information what leads to you removing setup package?

It is very easy

With current: cat /etc/yum.conf  clean_requirements_on_remove=True 

dnf will remove `setup` package in 'Removing unused dependencies' section.

About nsswitch.conf: I doubt that the sshd-system-user-account will be added to remove provider if its USERID is bellow some value.

Comment 5 Andrei Stepanov 2019-09-26 16:06:15 UTC
remote* provider

Comment 6 Tomas Mraz 2019-09-27 06:23:48 UTC
Then I would suggest to put the requires on /etc/passwd on the package that owns /usr/lib64/libnss_files.so.2 which is glibc to avoid the removal of the setup package this way.

Comment 7 Andrei Stepanov 2019-09-27 10:03:21 UTC
I agree with Tomas (comment 6). There are also fails from other system components:
1. 
Sep 26 10:27:31 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com sshd[5283]: fatal: Privilege separation user sshd does not exist
2.
Sep 26 10:19:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com systemd-tmpfiles[5277]: [/usr/lib/tmpfiles.d/var.conf:15] Unknown group 'utmp'.
3.
Sep 26 10:19:52 ci-vm-10-0-137-174.hosted.upshift.rdu2.redhat.com systemd-tmpfiles[5277]: Failed to parse ACL "d:group:adm:r-x,d:group:wheel:r-x": Invalid argument. Ignoring
Whole system functionality is under question.
Re-assign to glibc.

Comment 8 Carlos O'Donell 2019-09-27 20:44:18 UTC
The dependency is already there.

-> openssh-server requires libc.so.6()(64bit)
-> glibc requires basesystem e.g. Requires(pre): basesystem
-> basesystem requires setup e.g. Requires(pre): setup filesystem

So if you remove setup, which owns /etc/passwd, then you have to remove basesystem and glibc, and nothing works.

The bug is that setup was removed.

Let us ask the dnf maintainers what they think. Assigning to them.

Comment 9 Florian Weimer 2019-09-27 20:47:50 UTC
Requires(pre): dependencies are only used for the scriptlet, they can be removed after installation. We probably need both.

However, this change should be prototyped in Fedora. I don't feel like adding new dependency loops after a release.

Comment 10 Carlos O'Donell 2019-09-27 21:37:59 UTC
(In reply to Florian Weimer from comment #9)
> Requires(pre): dependencies are only used for the scriptlet, they can be
> removed after installation. We probably need both.

You are totally right. I had forgotten that Require(pre) could allow removal after install.
 
> However, this change should be prototyped in Fedora. I don't feel like
> adding new dependency loops after a release.

Why does basesystem use Requires(pre) also? Why do we do to the trouble of making them all removable after install. It seems like there is some history here we don't quite understand.

I'm looking to the dnf team to comment on the history here, because it seems like glibc and basesystem have both been using Requires(pre) in this way. Perhaps to avoid dependency loops?

Comment 11 Jaroslav Mracek 2019-09-30 14:09:05 UTC
I believe that many packages uses Require(pre) incorrectly, but mostly there is no problem with it because dependency trees are redundant.

I also think that reproducer is quite strange. Setup is mandatory package of Core group therefore it is top in dependency tree. Removal of core component is more that unexpected from user side.

I also consider that a dependency between the setup package and other packages could be improved. When one package requires for its functionality other package then it must be required or could be conditionally required using richdeps. In case that relation is not clear, weakdeps could be used. But this a subject for discussions between developers and maintainers of related components.

From DNF side this is not a bug.