Bug 1925438

Summary:	sshd sometimes starts too early, before the network is ready, and fails
Product:	Red Hat Enterprise Linux 8	Reporter:	Steffen Froemer <sfroemer>
Component:	openssh	Assignee:	Jakub Jelen <jjelen>
Status:	CLOSED NOTABUG	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	8.2
Target Milestone:	rc
Target Release:	8.0
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-02-08 13:05:59 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Steffen Froemer 2021-02-05 08:07:11 UTC

This bug was initially created as a copy of Bug #1352214

I am copying this bug because:

Description of problem:

systemd sometimes may start openssh before the network interfaces are ready. If openssh-server is configured to listen on a specific IP address(s), it will temporarily fail to bind to it.

Version-Release number of selected component (if applicable):

openssh-server-8.0p1-5.el8.x86_64

How reproducible:

Fairly reliably.

Steps to Reproduce:
1. Configure /etc/ssh/sshd_config to listen on a specific IP address
2. Reboot

Actual results:

The output from journalctl -l /usr/sbin/sshd:

-- Reboot --
Jul 02 09:35:38 shorty.email-scan.com sshd[1007]: WARNING: 'UseLogin yes' is not supported in Fedora and may cause several problems.
Jul 02 09:35:38 shorty.email-scan.com sshd[1077]: error: Bind to port 22 on 192.168.0.1 failed: Cannot assign requested address.
Jul 02 09:35:38 shorty.email-scan.com sshd[1077]: fatal: Cannot bind any address.
Jul 02 09:36:20 shorty.email-scan.com sshd[1494]: WARNING: 'UseLogin yes' is not supported in Fedora and may cause several problems.
Jul 02 09:36:20 shorty.email-scan.com sshd[1495]: Server listening on 192.168.0.1 port 22.

sshd, in this case, initially fails to bind on 192.168.0.1

Looks like sshd bails out and exits initially. After 40 seconds systemd starts it again. By this time the network interfaces have come up, and sshd successfully starts.

Expected results:

sshd should get started by systemd only after all network interfaces are configured.

Additional info:

this can be workarounded by using a systemd-dropin

# mkdir -p /etc/systemd/system/sshd.service.d
# cat << EOF > /etc/systemd/system/sshd.service.d/01-sshd-dependency.conf
[Unit]
After=network-online.target sshd-keygen.target
EOF
# systemctl daemon-reload

Comment 3 Jakub Jelen 2021-02-08 10:11:27 UTC

Its documented in RHEL7 and RHEL8 System administrator guide:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-openssh#s2-ssh-configuration-sshd

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_basic_system_settings/index#configuring-and-starting-an-openssh-server_using-secure-communications-between-two-systems-with-openssh

> To specify different addresses than the default 0.0.0.0 (IPv4) or :: (IPv6) for
> the ListenAddress directive in the /etc/ssh/sshd_config configuration file and
> to use a slower dynamic network configuration, add the dependency on the 
> network-online.target target unit to the sshd.service unit file. To achieve this,
> create the /etc/systemd/system/sshd.service.d/local.conf file with the following content:

>   [Unit]
>   Wants=network-online.target
>   After=network-online.target

Do you have some better way or better place of documenting this?

Comment 4 Steffen Froemer 2021-02-08 13:05:59 UTC

(In reply to Jakub Jelen from comment #3)
> Its documented in RHEL7 and RHEL8 System administrator guide:

Honestly I wasn't aware of this. As it's documented in a well known format, there is no further need. I'm closing this BZ now.