Bug 1592488
| Summary: | unable to run containers; exec user process caused "permission denied" | |||
|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Micah Abbott <miabbott> | |
| Component: | libsemanage | Assignee: | Petr Lautrbach <plautrba> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | |
| Severity: | unspecified | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 29 | CC: | adimania, admiller, amurdaca, dwalsh, fkluknav, ichavero, jcajka, lsm5, marianne, mgrepl, nalin, plautrba, santiago, slawomir, vbatts, vmojzis | |
| Target Milestone: | --- | Keywords: | Reopened | |
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | libsemanage-2.8-3.fc29 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1595300 (view as bug list) | Environment: | ||
| Last Closed: | 2018-09-25 14:22:28 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1595300, 1595316 | |||
I would guess /var/lib/docker is mislabeled restorecon -R -v /var/lib/docker Dan, you are correct. This is a problem if /var/lib/docker is getting relabeled after rebasing to Rawhide:
# rpm-ostree status
State: idle; auto updates disabled
Deployments:
● ostree://fedora-atomic:fedora/28/x86_64/atomic-host
Version: 28.20180527.0 (2018-05-27 19:05:29)
Commit: 291ea90da29bc5abe757b5a50813b3de1396b08412939a89b3b671aba9856093
GPGSignature: Valid signature by 128CF232A9371991C8A65695E08E7E629DB62FB1
# ls -lZ /var/lib/docker
total 0
drwx------. 2 root root system_u:object_r:container_var_lib_t:s0 6 Jun 18 19:21 containers
drwx------. 3 root root system_u:object_r:container_var_lib_t:s0 22 Jun 18 19:21 image
drwxr-x---. 3 root root system_u:object_r:container_var_lib_t:s0 19 Jun 18 19:21 network
drwx------. 3 root root system_u:object_r:container_share_t:s0 40 Jun 18 19:21 overlay2
drwx------. 4 root root system_u:object_r:container_var_lib_t:s0 32 Jun 18 19:21 plugins
drwx------. 2 root root system_u:object_r:container_var_lib_t:s0 6 Jun 18 19:21 swarm
drwx------. 2 root root system_u:object_r:container_var_lib_t:s0 6 Jun 18 19:21 tmp
drwx------. 2 root root system_u:object_r:container_var_lib_t:s0 6 Jun 18 19:21 trust
drwx------. 2 root root system_u:object_r:container_var_lib_t:s0 25 Jun 18 19:21 volumes
# rpm-ostree rebase rawhide:fedora/rawhide/x86_64/atomic-host
2244 metadata, 9394 content objects fetched; 365209 KiB transferred in 555 seconds
Copying /etc changes: 20 modified, 0 removed, 50 added
Transaction complete; bootconfig swap: yes; deployment count change: 1
...
# systemctl reboot
$ ssh 10.8.250.36
# rpm-ostree status
lState: idle; auto updates disabled
Deployments:
● ostree://rawhide:fedora/rawhide/x86_64/atomic-host
Version: Rawhide.20180616.n.0 (2018-06-16 09:30:08)
Commit: 1055dea1f99991fb56d5ae9e29cc6ff52fa01970555f82fcc8e929c7f717907f
ostree://fedora-atomic:fedora/28/x86_64/atomic-host
Version: 28.20180527.0 (2018-05-27 19:05:29)
Commit: 291ea90da29bc5abe757b5a50813b3de1396b08412939a89b3b671aba9856093
GPGSignature: Valid signature by 128CF232A9371991C8A65695E08E7E629DB62FB1
[root@micah-f28ah-vm0618b ~]# ls -lZ /var/lib/docker
total 0
drwx------. 2 root root system_u:object_r:unlabeled_t:s0 6 Jun 18 19:21 containers
drwx------. 3 root root system_u:object_r:unlabeled_t:s0 22 Jun 18 19:21 image
drwxr-x---. 3 root root system_u:object_r:unlabeled_t:s0 19 Jun 18 19:21 network
drwx------. 3 root root system_u:object_r:unlabeled_t:s0 40 Jun 18 19:37 overlay2
drwx------. 4 root root system_u:object_r:unlabeled_t:s0 32 Jun 18 19:21 plugins
drwx------. 2 root root system_u:object_r:unlabeled_t:s0 6 Jun 18 19:21 swarm
drwx------. 2 root root system_u:object_r:unlabeled_t:s0 6 Jun 18 19:21 tmp
drwx------. 2 root root system_u:object_r:unlabeled_t:s0 6 Jun 18 19:21 trust
drwx------. 2 root root system_u:object_r:unlabeled_t:s0 25 Jun 18 19:21 volumes
# restorecon -R -v /var/lib/docker
Relabeled /var/lib/docker/tmp from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/containers from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/plugins from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/plugins/tmp from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/plugins/storage from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/plugins/storage/blobs from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/plugins/storage/blobs/tmp from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/overlay2 from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/overlay2/l from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/overlay2/backingFsBlockDev from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2 from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/layerdb from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/imagedb from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/imagedb/content from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/imagedb/content/sha256 from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/imagedb/metadata from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/imagedb/metadata/sha256 from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/distribution from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/image/overlay2/repositories.json from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/volumes from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/volumes/metadata.db from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/trust from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/network from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/network/files from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/docker/network/files/local-kv.db from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_lib_t:s0
Unfortunately, even after `restorecon`, containers are just silently dying:
# docker run -it registry.fedoraproject.org/fedora echo 'hello'
# echo $? 139
# journalctl -b | grep 'avc: denied'
Jun 18 19:43:31 micah-f28ah-vm0618b audit[1673]: AVC avc: denied { read write } for pid=1673 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c37,c365 tcontext=system_u:object_r:container_file_t:s0:c37,c365 tclass=chr_file permissive=0
Jun 18 19:43:31 micah-f28ah-vm0618b audit[1673]: AVC avc: denied { read write } for pid=1673 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c37,c365 tcontext=system_u:object_r:container_file_t:s0:c37,c365 tclass=chr_file permissive=0
Jun 18 19:43:31 micah-f28ah-vm0618b audit[1673]: AVC avc: denied { read write } for pid=1673 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c37,c365 tcontext=system_u:object_r:container_file_t:s0:c37,c365 tclass=chr_file permissive=0
Jun 18 19:43:31 micah-f28ah-vm0618b audit[1673]: AVC avc: denied { read write } for pid=1673 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c37,c365 tcontext=system_u:object_r:container_file_t:s0:c37,c365 tclass=chr_file permissive=0
Jun 18 19:43:31 micah-f28ah-vm0618b audit[1673]: AVC avc: denied { map } for pid=1673 comm="echo" path="/usr/bin/coreutils" dev="dm-0" ino=58727044 scontext=system_u:system_r:container_t:s0:c37,c365 tcontext=system_u:object_r:var_lib_t:s0 tclass=file permissive=0
This shows you have containers with two different labels, It looks like the tty of one container is being leaked into the container of a second On Fedora 28 I get this:
# docker run -it --rm centos /bin/bash
standard_init_linux.go:178: exec user process caused "permission denied"
# echo $?
1
# journalctl -b | grep 'avc: denied'
Jun 20 16:03:17 fenris audit[29545]: AVC avc: denied { entrypoint } for pid=29545 comm="runc:[2:INIT]" path="/usr/bin/bash" dev="dm-8" ino=20710002 scontext=system_u:system_r:container_t:s0:c612,c807 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=0
It works with --privileged
I did:
restorecon -R -v /var/lib/docker
but it didn't change anything
What storage driver are you using? Is the foot file system mounted at /var/lib/docker? It turns out I was using custom value for --graph. After resetting to default the problem went away. Awesome, BTW Have you looked at podman? Dan, this was originally opened against Rawhide and it is still an issue there. Yeah, I even confirmed this on Fedora Server with a fresh install of docker: # cat /etc/os-release NAME=Fedora VERSION="29 (Cloud Edition)" ID=fedora VERSION_ID=29 PLATFORM_ID="platform:f29" PRETTY_NAME="Fedora 29 (Cloud Edition)" ANSI_COLOR="0;34" CPE_NAME="cpe:/o:fedoraproject:fedora:29" HOME_URL="https://fedoraproject.org/" SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=rawhide REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=rawhide PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy" VARIANT="Cloud Edition" VARIANT_ID=cloud # dnf -y install docker ... # systemctl enable docker --now Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service. # docker run -it --rm registry.fedoraproject.org/fedora echo 'hello' Unable to find image 'registry.fedoraproject.org/fedora:latest' locally Trying to pull repository registry.fedoraproject.org/fedora ... sha256:39994db8f1ee63244dc6baa35cd88988eb4be8ac6c026be0570bd618fd84d5af: Pulling from registry.fedoraproject.org/fedora bd02462c6d09: Pull complete Digest: sha256:39994db8f1ee63244dc6baa35cd88988eb4be8ac6c026be0570bd618fd84d5af Status: Downloaded newer image for registry.fedoraproject.org/fedora:latest # echo $? 139 # journalctl -b | grep 'avc: denied' Jun 20 15:11:24 micah-f28c-vm0620a audit[2240]: AVC avc: denied { read write } for pid=2240 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c104,c514 tcontext=system_u:object_r:container_file_t:s0:c104,c514 tclass=chr_file permissive=0 Jun 20 15:11:24 micah-f28c-vm0620a audit[2240]: AVC avc: denied { read write } for pid=2240 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c104,c514 tcontext=system_u:object_r:container_file_t:s0:c104,c514 tclass=chr_file permissive=0 Jun 20 15:11:24 micah-f28c-vm0620a audit[2240]: AVC avc: denied { read write } for pid=2240 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c104,c514 tcontext=system_u:object_r:container_file_t:s0:c104,c514 tclass=chr_file permissive=0 Jun 20 15:11:24 micah-f28c-vm0620a audit[2240]: AVC avc: denied { read write } for pid=2240 comm="echo" path="/1" dev="devpts" ino=4 scontext=system_u:system_r:container_t:s0:c104,c514 tcontext=system_u:object_r:container_file_t:s0:c104,c514 tclass=chr_file permissive=0 Jun 20 15:11:24 micah-f28c-vm0620a audit[2240]: AVC avc: denied { map } for pid=2240 comm="echo" path="/usr/bin/coreutils" dev="vda1" ino=656920 scontext=system_u:system_r:container_t:s0:c104,c514 tcontext=system_u:object_r:var_lib_t:s0 tclass=file permissive=0 # rpm -q container-selinux docker selinux-policy container-selinux-2.64-1.gitdfaf8fd.fc29.noarch docker-1.13.1-59.gitaf6b32b.fc29.x86_64 selinux-policy-3.14.2-25.fc29.noarch # ls -lZ /var/lib/docker total 36 drwx------. 2 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:11 containers drwx------. 3 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:10 image drwxr-x---. 3 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:10 network drwx------. 4 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:11 overlay2 drwx------. 4 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:10 plugins drwx------. 2 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:10 swarm drwx------. 2 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:11 tmp drwx------. 2 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:10 trust drwx------. 2 root root system_u:object_r:var_lib_t:s0 4096 Jun 20 15:10 volumes I did notice this during the install of docker/container-selinux: ... Installing : policycoreutils-python-utils-2.8-3.fc29.noarch 18/27 Installing : container-selinux-2:2.64-1.gitdfaf8fd.fc29.noarch 19/27 Running scriptlet: container-selinux-2:2.64-1.gitdfaf8fd.fc29.noarch 19/27 neverallow check failed at /var/lib/selinux/targeted/tmp/modules/100/base/cil:9194 (neverallow base_typeattr_7 unlabeled_t (file (entrypoint))) <root> allow at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1554 (allow spc_t unlabeled_t (file (entrypoint))) <root> allow at /var/lib/selinux/targeted/tmp/modules/100/sandboxX/cil:866 (allow sandbox_x_domain exec_type (file (entrypoint))) <root> allow at /var/lib/selinux/targeted/tmp/modules/100/virt/cil:1678 (allow virtd_lxc_t exec_type (file (entrypoint))) <root> allow at /var/lib/selinux/targeted/tmp/modules/100/virt/cil:2069 (allow svirt_sandbox_domain exec_type (file (entrypoint))) Failed to generate binary /usr/sbin/semodule: Failed! Installing : python3-pytoml-0.1.16-1.fc29.noarch 20/27 Installing : atomic-registries-1.22.1-22.git5a342e3.fc29.x86_64 ... This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle. Changing version to '29'. |
Using the latest Fedora Rawhide Atomic Host, I was unable to run a container using `docker`. It appears SELinux denied the execution of the container: # rpm-ostree status rpmState: idle; auto updates disabled Deployments: ● ostree://rawhide:fedora/rawhide/x86_64/atomic-host Version: Rawhide.20180616.n.0 (2018-06-16 09:30:08) Commit: 1055dea1f99991fb56d5ae9e29cc6ff52fa01970555f82fcc8e929c7f717907f # rpm -q docker container-selinux selinux-policy docker-1.13.1-59.gitaf6b32b.fc29.x86_64 container-selinux-2.64-1.gitdfaf8fd.fc29.noarch selinux-policy-3.14.2-25.fc29.noarch # docker run -it --rm registry.fedoraproject.org/fedora echo 'hello' Unable to find image 'registry.fedoraproject.org/fedora:latest' locally Trying to pull repository registry.fedoraproject.org/fedora ... sha256:39994db8f1ee63244dc6baa35cd88988eb4be8ac6c026be0570bd618fd84d5af: Pulling from registry.fedoraproject.org/fedora bd02462c6d09: Pull complete Digest: sha256:39994db8f1ee63244dc6baa35cd88988eb4be8ac6c026be0570bd618fd84d5af Status: Downloaded newer image for registry.fedoraproject.org/fedora:latest standard_init_linux.go:178: exec user process caused "permission denied" # journalctl -b | grep 'avc: denied' Jun 18 15:56:19 micah-f28ah-vm0618a audit[1280]: AVC avc: denied { entrypoint } for pid=1280 comm="runc:[2:INIT]" path="/usr/bin/echo" dev="dm-0" ino=58724330 scontext=system_u:system_r:container_t:s0:c256,c1017 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=0