Description of problem:
RHEL 7.4 kernel has introduced a new sysctl knob to control kernel behavior. This is called /proc/sys/fs/may_detach_mounts. This knob is set to value 0 by default. Container run times (docker and others) need the new behavior and
want it to be set to 1.
So modify runc package to drop a file say /usr/lib/sysctl.d/99-docker.conf. Contents of this file can be say following.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
We need this change so that we can run docker daemon in host mount namespace. And that will enable shared volume feature of docker where volumes mounted by container can propagate to host (if user has configured it right).
What happens if this file gets placed on a system without the kernel mode? I think it just gets ignored correct? Ie we install the runc package on a RHEL7.3 OS>
I think that it will be ignored. IOW, I think systemd will try to write to this file but it will not be present. I am assuming that systemd will continue to write rest of the knobs.
I tried it on fedora kernel and I see following message in logs.
Apr 12 13:25:28 vm7-f25 systemd-sysctl: Couldn't write '1' to 'fs/may_detach_mounts', ignoring: No such file or directory
Lokesh lets drop /usr/lib/sysctl.d/99-containers.conf with this flag, for rhel7.4.
This RPM install is not behaving as I had expected: the may_detach_mounts option is not taking effect until after a reboot. I believe the specfile is missing a %sysctl_apply directive:
Good point Ed.
Lokesh, we probably need to fix it.
This does not help with docker install. Docker package seems to ship its own runc and does not have dependency to install runc package. That means after installing docker, /proc/sys/fs/may_detach_mounts is not 1.
It is probably easiest to let docker pull in runc package also during installation and that will make sure this knob is turned on.
Docker will depend on this feature so that we have less issues w.r.t mounts and device being busy.
Then we need to add
to docker package.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
Fedora Atomic 26 and 27 are still affected, I'm not sure what need to be done.
I can't set may_detach_mounts to 1 in a fedora atomic 27 host with:
Should I open a bug for fedora?
(In reply to Spyros Trigazis from comment #15)
> Fedora Atomic 26 and 27 are still affected
Can you clarify what you mean by "affected"? This issue affects only RHEL, and the /proc/sys/fs/may_detach_mounts switch exists only in RHEL. Fedora has never been affected by this issue and has never had a need for the may_detach_mounts option.
* run kubernetes (tried with 1.7, 1.8 1.9 and 1.10.alpha) on fedora atomic 26 or 27 with docker as runtime
* have a pod with a secret or a configmap mounted
* try to delete a pod it gets stuck in state terminating
The issue is tracked here:
From the kubelet logs:
E0117 10:44:35.028204 6912 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/secret/e5fdeccf-fb72-11e7-9157-fa163eda350b-default-token-dtcng\" (\"e5fdeccf-fb72-11e7-9157-fa163eda350b\")" failed. No retries permitted until 2018-01-17 10:44:37.028168931 +0000 UTC m=+43.675199102 (durationBeforeRetry 2s). Error: "UnmountVolume.TearDown failed for volume \"default-token-dtcng\" (UniqueName: \"kubernetes.io/secret/e5fdeccf-fb72-11e7-9157-fa163eda350b-default-token-dtcng\") pod \"e5fdeccf-fb72-11e7-9157-fa163eda350b\" (UID: \"e5fdeccf-fb72-11e7-9157-fa163eda350b\") : remove /var/lib/kubelet/pods/e5fdeccf-fb72-11e7-9157-fa163eda350b/volumes/kubernetes.io~secret/default-token-dtcng: device or resource busy"
This is not, somehow, working for rhel 7.5, Lokesh can you take a look? the related bug is: https://bugzilla.redhat.com/show_bug.cgi?id=1637623