Bug 1466848 - Restart of atomic-openshift-node container terminates pod glusterfs mount
Restart of atomic-openshift-node container terminates pod glusterfs mount
Status: CLOSED DUPLICATE of bug 1472370
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage (Show other bugs)
x86_64 Linux
unspecified Severity medium
: ---
: 3.7.0
Assigned To: Jan Safranek
Jianwei Hou
: NeedsTestCase, Reopened
Depends On:
  Show dependency treegraph
Reported: 2017-06-30 10:23 EDT by Jan Safranek
Modified: 2017-09-11 04:48 EDT (History)
19 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1423640
Last Closed: 2017-09-11 04:48:15 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Comment 1 Jan Safranek 2017-06-30 10:27:10 EDT
I lowered the severity from the original bug, AFAIK no customer is complaining so far.
Comment 2 Jan Safranek 2017-06-30 10:29:42 EDT
Pruned bug dependencies
Comment 3 Jan Safranek 2017-06-30 10:43:34 EDT
I am talking to local systemd guys about escaping a docker container properly so fuse daemon runs really on the host and restart of docker container won't kill it.

Option 1:
Newer systemd (v233?) ships systemd-mount, which creates an transient unit file that mounts. Fuse daemon would probably run in its context. In the container we would probably do 'nsenter --mount=/rootfs/proc/1/ns/mnt -- /bin/systemd-mount -t glusterfs -o <opts> <what> <where>' (testing needed).

Unfortunately, RHEL7 has too old systemd and systemd-mount is not there and rebase is not planned. Backport could be possible though.

Option 2:
systemd in RHEL7 has systemd-run command, which creates a transient service and executes something there. kubelet would do `nsenter --mount=/rootfs/proc/1/ns/mnt -- /bin/systemd-run /bin/mount -t glusterfs -o <opts> <what> <where>'. Again, testing needed as I am not sure if the service would not be killed by systemd when /bin/mount finishes and only glusterfs fuse daemon is running.

I'm investigating these options.

Obviously, both these options will make openshift-node container dependent on the host running systemd. So far that was not hard requirement.

Any other smart ideas how to escape a container are welcome.
Comment 4 Jan Safranek 2017-06-30 11:17:08 EDT
Tested option 2, this looks working:

nsenter --mount=/rootfs/proc/1/ns/mnt -- systemd-run --scope /bin/mount -t glusterfs /var/lib/origin/openshift.local.volumes/xyz

(and nsenter --mount=/rootfs/proc/1/ns/mnt -- umount /var/lib/origin/openshift.local.volumes/xyz)

- glusterfs fuse daemon runs in its own systemd slice (=cgroup) with a random name (run-11615.scope)
- it is not killed when /bin/mount finishes
- it is killed by unmount
- the slice is automatically deleted when the last process dies, i.e. after unmount

That brings us to hard dependency on systemd on the host... In OpenShift it's probably OK, I am not sure about upstream.
Comment 5 Jan Safranek 2017-07-03 09:19:31 EDT
created https://github.com/kubernetes/kubernetes/pull/48430, above systemd-run call is used when it's available on the host, otherwise simple 'nsenter --mount=/rootfs/proc/1/ns/mnt -- mount' is used.
Comment 6 Jan Safranek 2017-09-11 04:48:15 EDT

*** This bug has been marked as a duplicate of bug 1472370 ***

Note You need to log in before you can comment on or make changes to this bug.