Description of problem:
On RHEL 7.4.5 and docker 1.13.1-62, when docker is stopped (no pods running), the /var/lib/docker/overlay2 filesystem is unmounted.
If /var/lib/docker/overlay2 is removed from oci-umount.conf the problem does not occur. My understanding is that oci-umount.conf is to control mounts within pods, not OS mounts.
This problem does not occur with docker 1.12.6-68
Version-Release number of selected component (if applicable): 1.13.1-62
How reproducible: Always
Steps to Reproduce:
1. Install docker 1.13.1-62.
2. Create an XFS filesystem for /var/lib/docker/overlay2. configure docker to use the overlay2 storage driver
3. df -h to verify the fs is mounted
4. systemctl stop docker (or restart)
/var/lib/docker/overlay2 is unmounted
/var/lib/docker/overlay2 fs survives a docker restart
Why not configure your XFS to mount at /var/lib/docker? docker/oci-umount are both mounting and umounting the driver directories. Their are several reasons for this but mainly leaked file descriptors. You could also remove the /var/lib/docker/ovleray2 directory from /etc/oci-umounts.conf.
Both suggestions in comment 2 are valid - thanks. I opened the bz to track the behavior change. Setting severity to medium - is this just a doc issue?
Sure we can make this a docs issue.
Yes, don't mount on /var/lib/docker/overlay2. I suspect docker is the one which unmounts that mount point when it stops (And not oci-umount). Docker creates a private mount there and unmounts when stopping. If mount point is already there, I think then a new mount point is not created but unmounted on stop. Not sure, but that's my current thought process.
I agree it is likely not oci-umount. Removing /var/lib/docker/overlay2 from /etc/oci-umount.conf did not help. We'll change our mount to /var/lib/docker
Mike, we already talked about it on IRC. I am putting same information here so that anybody looking later can find it.
- openshift folks, try to make use of docker-storage-setup to setup a separate volume and put xfs on top of that volume and mount it on /var/lib/docker.
- You can use following /etc/sysconfig/docker-storage-setup for that.
This will create a volume group "docker-vg" from disk /dev/vdb, create a logical volme docker-root-lv from volume group, create xfs filesystem on this logical volume and mount on /var/lib/docker. And over restart, just mounting step happens.
- From command line do following.
- systemctl stop docker
- atomic storage reset
- Edit /etc/sysconfig/docker-storage-setup appropriately
- systemctl start docker
And you should be all set.
I looked at config of another node I think used by openstack folks. I noticed one more things which probably make sense.
This specifies use 100% of free space in VG for the logical volume. By default it uses 40%of free space. So config should look something as follows.
This recommendation is assuming that you don't want to use root filesystem of the host to storage containers and images and want to use a separate disk. If you want to use rootfs of host, then all you need is.
So first decide what are your requirements and then choose the container-storage-setup accordingly.
We have no plans to ship another version of Docker at this time. RHEL7 is in final support stages where only security fixes will get released. Customers should move to use Podman which is available starting in RHEL 7.6.