Description of problem:
We are not able to ssh to/between pods using the latest docker version package
1.13.1-108
But older versions such as 1.13.1-104 and 1.13.1-103 appear to work just fine
Version-Release number of selected component (if applicable):
docker-1.13.1-108.git4ef4b30.el7.x86_64
How reproducible:
Fully reproducible
* Test on fresh install fully updated ocp3.11 cluster:
If I rsh in to the mycluster-backrest-shared-repo-xxxxxx pod
And from there I attempt to ssh into the other pod mycluster-xxxxxx.
The results is a "Permission denied (publickey,keyboard-interactive)"
- Details:
sh-4.2$ ssh postgres@mycluster -v
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /home/pgbackrest/.ssh/config
debug1: /home/pgbackrest/.ssh/config line 1: Applying options for *
...
...
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: No more authentication methods to try.
Permission denied (publickey,keyboard-interactive)
* Test with downgraded docker
sh-4.2$ hostname
mycluster-backrest-shared-repo-5f4f56d667-4kvmm
sh-4.2$ ssh postgres@mycluster -v
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /home/pgbackrest/.ssh/config
debug1: /home/pgbackrest/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
...
...
Authenticated to mycluster ([x.x.x.x]:2022).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00 want_reply 0
debug1: Sending environment.
-bash-4.2$ hostname
mycluster-7fb9f74d95-lmst7
Additionally, we did do some additional testing/troubleshooting today, and did find a change that appears to address the issue we are seeing, and therefore results in SSH working properly once again with Docker version 1.13.1-108 Specifically, we noticed that 'UsePAM' was set to 'no' in our SSHD configuration, so we went ahead and changed it to 'yes' (i.e. we updated our 'sshd_config' file with the following setting):
UsePAM yes
With this change in place, SSHD connections started working fine across all version of Docker we were testing with (1.13.1-108, 1.13.1-104 and 1.13.1-103). Further, based on the following message in the 'sshd_config' configuration file, it appears as though this setting should be set to 'yes' to avoid unintended consequences.
Actual results:
ssh does not work without UsePAM set to yes ( Although it is not supported without setting to "yes" ) which was working fine on previous docker versions