Bug 1714816 - [OSP13] sshd running inside nova_migration_target overrides /var/run/sshd.pid on host with pid 1
Summary: [OSP13] sshd running inside nova_migration_target overrides /var/run/sshd.pid...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: z8
: 13.0 (Queens)
Assignee: Piotr Kopec
QA Contact: Martin Schuppert
URL:
Whiteboard:
Depends On:
Blocks: 1724674
TreeView+ depends on / blocked
 
Reported: 2019-05-28 23:45 UTC by Takashi Kajinami
Modified: 2019-09-03 16:55 UTC (History)
5 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.3.1-57.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1724674 (view as bug list)
Environment:
Last Closed: 2019-09-03 16:55:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1830982 0 None None None 2019-05-29 23:08:28 UTC
OpenStack gerrit 663149 0 None MERGED Do not bind /run on host to nova_migration_target 2020-08-14 01:25:30 UTC
OpenStack gerrit 667807 0 None MERGED Add /run/libvirt to nova_migration_target container 2020-08-14 01:25:30 UTC
Red Hat Product Errata RHBA-2019:2624 0 None None None 2019-09-03 16:55:53 UTC

Description Takashi Kajinami 2019-05-28 23:45:43 UTC
Description of problem:

In RHOSOP13, we have 2 sshd instances running on compute nodes.
 1. sshd running on host, to use conventional operation over remote login
 2. sshd running inside nova_migration_target, used for migration in nova

These two instances generally use separated resources, like ports, or conf files,
but they are sharing pid, /var/run/sshd.pid on the host.

This causes that we have "1" in /var/run/sshd.pid, as pid file is overwritten
by the sshd running inside nova_migration_target container with pid 1 inside the container.

Note that currently we do not see any specific problem caused by this,
except for the below error log shown when we restart sshd running on host.
~~~
[heat-admin@compute-0 ~]$ sudo systemctl restart sshd
[heat-admin@compute-0 ~]$ sudo systemctl status sshd
● sshd.service - OpenSSH server daemon
...
May 28 23:31:45 compute-0 sshd[422824]: error: Couldn't create pid file "/var/run/sshd.pid": Permission denied
~~~

Version-Release number of selected component (if applicable):
RHOSP13z6

How reproducible:
Always

Steps to Reproduce:
1. Restart nova_migration_target container
2. Check content in /var/run/sshd.pid

Actual results:
We have pid 1 in the pid file

Expected results:
We have pid for sshd running at host level in the pid file

Additional info:

Comment 1 Keigo Noha 2019-05-29 07:29:11 UTC
The cause of this issue is ./docker/services/nova-migration-target.yaml contains following volume mount.
~~~
    145             volumes:
    146               list_concat:
    152                   - /run:/run
~~~
In host side, /var/run is a symbolic link to /run and the directory is used by the host side sshd also.

Comment 4 Martin Schuppert 2019-06-26 12:52:10 UTC
This introduced an issue with live migration as the nova-migration-wrapper inside the container needs access to the libvirt socket [1].

We track the fix for this in https://bugzilla.redhat.com/show_bug.cgi?id=1724131

[1] https://github.com/rdo-packages/nova-distgit/blob/rpm-master/nova-migration-wrapper#L31

Comment 16 errata-xmlrpc 2019-09-03 16:55:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2624


Note You need to log in before you can comment on or make changes to this bug.