Description of problem: No directory /var/lib/containers/overlay2 while fixing selinux permissions. Version-Release number of the following components: openshift-ansible-3.10.0-0.16.0.git.0.8925606.el7.noarch.rpm # docker info Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 0 Server Version: 1.13.1 Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Authorization: rhel-push-plugin Swarm: inactive Runtimes: docker-runc runc Default Runtime: docker-runc Init Binary: docker-init containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1) runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f) init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574) Security Options: seccomp WARNING: You're not using the default seccomp profile Profile: /etc/docker/seccomp.json selinux Kernel Version: 3.10.0-693.11.1.el7.x86_64 Operating System: Red Hat Enterprise Linux Server 7.4 (Maipo) OSType: linux Architecture: x86_64 Number of Docker Hooks: 3 CPUs: 1 Total Memory: 3.456 GiB Name: qe-ghuang-master-etcd-1 Docker Root Dir: /var/lib/containers/docker Debug Mode (client): false Debug Mode (server): false How reproducible: always Steps to Reproduce: 1. Trigger installation with rpm cri-o enabled # cat inventory <--snip--> openshift_crio_use_rpm=true <--snip--> Actual results: TASK [container_runtime : Fixup SELinux permissions for docker] **************** Saturday 07 April 2018 23:12:27 -0400 (0:00:01.585) 0:02:00.428 ******** fatal: [qe-ghuang-master-etcd-1.0407-fvf.qe.rhcloud.com]: FAILED! => {"changed": true, "cmd": "semanage fcontext -a -e /var/lib/docker/overlay2 /var/lib/containers/overlay2\n restorecon -R -v /var/lib/containers/overlay2", "delta": "0:00:00.373619", "end": "2018-04-07 23:12:30.368412", "failed": true, "msg": "non-zero return code", "rc": 255, "start": "2018-04-07 23:12:29.994793", "stderr": "restorecon: lstat(/var/lib/containers/overlay2) failed: No such file or directory", "stderr_lines": ["restorecon: lstat(/var/lib/containers/overlay2) failed: No such file or directory"], "stdout": "", "stdout_lines": []} fatal: [qe-ghuang-node-registry-router-1.0407-fvf.qe.rhcloud.com]: FAILED! => {"changed": true, "cmd": "semanage fcontext -a -e /var/lib/docker/overlay2 /var/lib/containers/overlay2\n restorecon -R -v /var/lib/containers/overlay2", "delta": "0:00:00.398757", "end": "2018-04-07 23:12:30.415676", "failed": true, "msg": "non-zero return code", "rc": 255, "start": "2018-04-07 23:12:30.016919", "stderr": "restorecon: lstat(/var/lib/containers/overlay2) failed: No such file or directory", "stderr_lines": ["restorecon: lstat(/var/lib/containers/overlay2) failed: No such file or directory"], "stdout": "", "stdout_lines": []} Expected results: Additional info: # ll /var/lib/containers/overlay2 ls: cannot access /var/lib/containers/overlay2: No such file or directory
Mrunal, You added this[1][2], under which conditions would we expect /var/lib/containers/overlay2 to exist? Should we just check for the existence and not apply those changes if it doesn't exist? 1 - https://github.com/openshift/openshift-ansible/pull/7553 2 - https://github.com/openshift/openshift-ansible/pull/7466
Scott, Yes, we can check for existence and apply only if the directory is present.
https://github.com/openshift/openshift-ansible/pull/7955
I think it's the overlay2 directory in question, which should be context type 'container_share_t' However, the overlay2 directory is not created via RPM, so whatever creates it must be responsible for setting the context. It's not clear what we should do for these cases.
Fix is already in master branch. Tested with master branch, succeeded to spin up cluster with cri-o. Removing TestBlocker.
Verified in openshift-ansible-3.10.0-0.29.0.git.0.8973ee4.el7.noarch.rpm
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1816