| Summary: | Unable to start container with SELinux enabled | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Johnny Liu <jialiu> |
| Component: | docker | Assignee: | Lokesh Mandvekar <lsm5> |
| Status: | CLOSED ERRATA | QA Contact: | atomic-bugs <atomic-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.3 | CC: | ajia, amurdaca, dma, dwalsh, imcleod, jeder, jpazdziora, lfriedma, lsm5, lsu, lvrabec, mgrepl, miabbott, mifiedle, mnguyen, qcai, tstclair, wehe, xtian |
| Target Milestone: | rc | Keywords: | Extras |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | aos-scalability-34 | ||
| Fixed In Version: | docker-1.10.3-58.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-12-06 17:44:37 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Johnny Liu
2016-10-09 02:23:12 UTC
ps -eZ | grep docker This almost always means docker is running with the wrong label. Whenever you report an SELinux issue, please give us the output of ausearch -m avc -ts recent (In reply to Daniel Walsh from comment #2) > ps -eZ | grep docker > > This almost always means docker is running with the wrong label. # ps -eZ | grep docker system_u:system_r:unconfined_service_t:s0 24549 ? 00:05:53 dockerd-current system_u:system_r:unconfined_service_t:s0 24555 ? 00:00:02 docker-containe > > > Whenever you report an SELinux issue, please give us the output of > > ausearch -m avc -ts recent # ausearch -m avc -ts recent ---- time->Sun Oct 9 05:48:32 2016 type=SYSCALL msg=audit(1476006512.529:26952): arch=c000003e syscall=59 success=no exit=-13 a0=c8200ff778 a1=c820113d20 a2=c820113d40 a3=0 items=0 ppid=58805 pid=58820 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="exe" exe="/usr/libexec/docker/docker-runc" subj=system_u:system_r:unconfined_service_t:s0 key=(null) type=AVC msg=audit(1476006512.529:26952): avc: denied { transition } for pid=58820 comm="exe" path="/usr/bin/bash" dev="dm-5" ino=16797850 scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:system_r:svirt_lxc_net_t:s0:c139,c272 tclass=process BTW, it works well in my RHEL7 system, I will check it w/ unlock mode in atomic host if need. # rpm -q docker selinux-policy libselinux docker-1.12.2-3.el7.x86_64 selinux-policy-3.13.1-102.el7.noarch libselinux-2.5-2.el7.x86_64 # docker run -it --rm centos /bin/bash [root@b6950e1e8e44 /]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b1b57d29485f cockpit/kubernetes:latest "/usr/libexec/cockpit" 30 hours ago Up 30 hours 0.0.0.0:9090->9090/tcp atomic-registry-console da2d4b76e835 openshift/origin:latest "/usr/bin/openshift s" 30 hours ago Up 30 hours 53/tcp, 0.0.0.0:8443->8443/tcp atomic-registry-master # ps -eZ | grep docker system_u:system_r:docker_t:s0 39386 ? 00:00:00 docker-current system_u:system_r:unconfined_service_t:s0 39430 ? 00:00:00 docker-containe system_u:system_r:svirt_lxc_net_t:s0:c173,c871 39447 ? 00:00:00 dockerregistry <defunct> system_u:system_r:unconfined_service_t:s0 134188 ? 00:50:57 dockerd-current system_u:system_r:unconfined_service_t:s0 134195 ? 00:03:18 docker-containe system_u:system_r:docker_t:s0 134320 ? 00:00:02 docker-current system_u:system_r:docker_t:s0 134525 ? 00:00:00 docker-current system_u:system_r:unconfined_service_t:s0 134574 ? 00:00:00 docker-proxy system_u:system_r:unconfined_service_t:s0 134582 ? 00:00:02 docker-containe system_u:system_r:unconfined_service_t:s0 134805 ? 00:00:00 docker-proxy system_u:system_r:unconfined_service_t:s0 134813 ? 00:00:00 docker-containe The problem is /usr/bin/docker is mislabeled. restorecon -R -v /usr/bin/docker* Some how these are mislabeled. Perhaps docker-selinux installation failed? If this is atomic host, you might be in trouble since the labels need to be stored on the server. (In reply to Daniel Walsh from comment #5) > The problem is /usr/bin/docker is mislabeled. > > restorecon -R -v /usr/bin/docker* Seem like this did not change anything, still can not start container. [root@openshift-124 ~]# restorecon -R -v /usr/bin/docker* [root@openshift-124 ~]# ll -Z /usr/bin/docker* -rwxr-xr-x. root root system_u:object_r:docker_exec_t:s0 /usr/bin/docker -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/docker-current -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/dockerd-current -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/docker-proxy -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/docker-storage-setup > > Some how these are mislabeled. Perhaps docker-selinux installation failed? > If this is atomic host, you might be in trouble since the labels need to be > stored on the server. Seem like docker-selinux is renamed to container-selinux now. # yum install docker-selinux Package container-selinux-1.12.2-3.el7.x86_64 already installed and latest version Nothing to do BTW, the test machine is a RHEL system (RHEL-7.3-20161007.n.0). yum reinstall container-selinux See if this fixes the issue. Lokesh, I did not mean for container-selinux to go into RHEL yet, but if everything works ok, it should be fine. Something is going wrong with that package. (In reply to Daniel Walsh from comment #8) > yum reinstall container-selinux > No chance, the same behavior. Then this is some kind of bug in container_selinux package. Here is the labeling I am seeing in Rawhide. # grep container_runtime_exec_t /etc/selinux/targeted/contexts/files/file_contexts /usr/bin/docker.* -- system_u:object_r:container_runtime_exec_t:s0 /usr/bin/container[^/]*plugin -- system_u:object_r:container_runtime_exec_t:s0 /usr/lib/docker/container[^/]*plugin -- system_u:object_r:container_runtime_exec_t:s0 /usr/libexec/docker/docker.* -- system_u:object_r:container_runtime_exec_t:s0 /usr/libexec/docker/container[^/]*plugin -- system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-latest -- system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-current -- system_u:object_r:container_runtime_exec_t:s0 *** Bug 1383813 has been marked as a duplicate of this bug. *** I think the problem is that container-selinux should require the latest selinux-policy package to be installed before running its post install. Lokesh could you confirm that this is happening? Dan, which specific selinux-policy version do we believe they need to be using? selinux-policy-3.13.1-102.el7_3.3 Hi Alex, the NVR is correct. We have moved to 1.12.1 for now. It's quite possible we will ship 1.12.3 once it's released, but for now it's 1.12.1. I see that I have an older selinux-policy on my system -101. Let me get back to you on this. RE: Bug 1385844, skopeo-containers will only conflict with an older version of atomic. atomic 1.13.1 can be installed with skopeo >= 1:0.1.17-0.4. The /etc/containers/registries.d/default.yaml moved from atomic to skopeo-containers and that's why we had to add the Conflicts: atomic <= old_version, but it shouldn't be a problem with newer versions. (In reply to Lokesh Mandvekar from comment #22) > RE: Bug 1385844 I meant Bug 1385584 Lokesh, okay, got it. I gave a try for latest docker-1.12.1-7.el7.x86_64, the issue still hasn't been fixed. Still reproduce with docker-1.12.1-6.el7.x86_64 on a clean system. 1. On a clean system install docker-1.12.1-6.el7.x86_64. docker-selinux-1.10.3-57.el7.x86_64 docker-1.12.1-6.el7.x86_64 docker-rhel-push-plugin-1.12.1-6.el7.x86_64 docker-common-1.12.1-6.el7.x86_64 skopeo-containers-0.1.17-0.4.git550a480.el7.x86_64 subscription-manager-plugin-container-1.17.15-1.el7.x86_64 container-selinux-1.12.1-6.el7.x86_64 2. Try to start a container in enforcing mode, failed. # docker run -it --rm centos /bin/bash Unable to find image 'centos:latest' locally Trying to pull repository registry.access.redhat.com/centos ... Trying to pull repository docker.io/library/centos ... sha256:2ae0d2c881c7123870114fb9cc7afabd1e31f9888dac8286884f6cf59373ed9b: Pulling from docker.io/library/centos 8d30e94188e7: Pull complete Digest: sha256:2ae0d2c881c7123870114fb9cc7afabd1e31f9888dac8286884f6cf59373ed9b Status: Downloaded newer image for docker.io/centos:latest standard_init_linux.go:175: exec user process caused "permission denied" 3. erase docker and docker-container. 4. install "docker-selinux" firstly, then re-install docker and docker-container. and start docker service. docker-selinux-1.10.3-57.el7.x86_64 docker-1.12.1-6.el7.x86_64 docker-rhel-push-plugin-1.12.1-6.el7.x86_64 docker-common-1.12.1-6.el7.x86_64 skopeo-containers-0.1.17-0.4.git550a480.el7.x86_64 subscription-manager-plugin-container-1.17.15-1.el7.x86_64 container-selinux-1.12.1-6.el7.x86_64 5. this time container is started successfully in enforcing mode. # docker run -it --rm centos /bin/bash [root@d8db86bb5fad /]# exit exit The problem with dropping back was that we lost the docker-selinux/container-selinux patch. We are going to build a docker-1.13.3 package which should have the correct fix. Just a update, it seems we shouldn't show debugging message to normal users. # rpm -q docker selinux-policy docker-1.12.3-1.el7.x86_64 selinux-policy-3.13.1-102.el7_3.4.noarch # docker run -it --rm busybox /bin/sh panic: standard_init_linux.go:175: exec user process caused "permission denied" [recovered] panic: standard_init_linux.go:175: exec user process caused "permission denied" goroutine 1 [running, locked to thread]: panic(0x7ec7c0, 0xc82016a8c0) /usr/lib/golang/src/runtime/panic.go:481 +0x3e6 github.com/urfave/cli.HandleAction.func1(0xc8201412e8) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:478 +0x38e panic(0x7ec7c0, 0xc82016a8c0) /usr/lib/golang/src/runtime/panic.go:443 +0x4e9 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization.func1(0xc820140bf8, 0xc820036080, 0xc820140d08) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:259 +0x136 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization(0xc8200806e0, 0x7f6ba4be3470, 0xc82016a8c0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:277 +0x5b1 main.glob.func8(0xc8200a6780, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main_unix.go:26 +0x68 reflect.Value.call(0x750ee0, 0x902d00, 0x13, 0x848d08, 0x4, 0xc820141268, 0x1, 0x1, 0x0, 0x0, ...) /usr/lib/golang/src/reflect/value.go:435 +0x120d reflect.Value.Call(0x750ee0, 0x902d00, 0x13, 0xc820141268, 0x1, 0x1, 0x0, 0x0, 0x0) /usr/lib/golang/src/reflect/value.go:303 +0xb1 github.com/urfave/cli.HandleAction(0x750ee0, 0x902d00, 0xc8200a6780, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:487 +0x2ee github.com/urfave/cli.Command.Run(0x84bbb0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x8e1d40, 0x51, 0x0, ...) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/command.go:191 +0xfec github.com/urfave/cli.(*App).Run(0xc820001980, 0xc82000a100, 0x2, 0x2, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:240 +0xaa4 main.main() /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main.go:137 +0xe24 This is a dump of GOLANG caused by runc crashing. Fixed in docker-1.12.3 alex, johnny can you try with 1.12.3-1 https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=520605 (In reply to Lokesh Mandvekar from comment #30) > alex, johnny can you try with 1.12.3-1 > https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=520605 Lokesh, the docker 1.12.3-1 has the same issue to me, please see Comment 27. Re-test this bug with docker-1.12.3-1.el7.x86_64, still reproduce just like comment 27. # docker run -it --rm centos /bin/bash Unable to find image 'centos:latest' locally Trying to pull repository registry.access.redhat.com/centos ... Trying to pull repository docker.io/library/centos ... sha256:2ae0d2c881c7123870114fb9cc7afabd1e31f9888dac8286884f6cf59373ed9b: Pulling from docker.io/library/centos 8d30e94188e7: Pull complete Digest: sha256:2ae0d2c881c7123870114fb9cc7afabd1e31f9888dac8286884f6cf59373ed9b Status: Downloaded newer image for docker.io/centos:latest panic: standard_init_linux.go:175: exec user process caused "permission denied" [recovered] panic: standard_init_linux.go:175: exec user process caused "permission denied" goroutine 1 [running, locked to thread]: panic(0x7ec7c0, 0xc820128b60) /usr/lib/golang/src/runtime/panic.go:481 +0x3e6 github.com/urfave/cli.HandleAction.func1(0xc8200fd2e8) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:478 +0x38e panic(0x7ec7c0, 0xc820128b60) /usr/lib/golang/src/runtime/panic.go:443 +0x4e9 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization.func1(0xc8200fcbf8, 0xc82001a018, 0xc8200fcd08) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:259 +0x136 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization(0xc820065590, 0x7f599255f738, 0xc820128b60) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:277 +0x5b1 main.glob.func8(0xc820088a00, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main_unix.go:26 +0x68 reflect.Value.call(0x750ee0, 0x902d00, 0x13, 0x848d08, 0x4, 0xc8200fd268, 0x1, 0x1, 0x0, 0x0, ...) /usr/lib/golang/src/reflect/value.go:435 +0x120d reflect.Value.Call(0x750ee0, 0x902d00, 0x13, 0xc8200fd268, 0x1, 0x1, 0x0, 0x0, 0x0) /usr/lib/golang/src/reflect/value.go:303 +0xb1 github.com/urfave/cli.HandleAction(0x750ee0, 0x902d00, 0xc820088a00, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:487 +0x2ee github.com/urfave/cli.Command.Run(0x84bbb0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x8e1d40, 0x51, 0x0, ...) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/command.go:191 +0xfec github.com/urfave/cli.(*App).Run(0xc820001380, 0xc820060060, 0x2, 0x2, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:240 +0xaa4 main.main() /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main.go:137 +0xe24 docker-common-1.12.3-1.el7.x86_64 docker-1.12.3-1.el7.x86_64 docker-rhel-push-plugin-1.12.3-1.el7.x86_64 libselinux-2.5-6.el7.x86_64 libselinux-utils-2.5-6.el7.x86_64 libselinux-python-2.5-6.el7.x86_64 selinux-policy-targeted-3.13.1-102.el7.noarch container-selinux-1.12.3-1.el7.x86_64 selinux-policy-3.13.1-102.el7.noarch Johnny What does ls -lZ /usr/bin/docker* Return? How about matchpatchcon /usr/bin/docker* ps -eZ | grep docker I don't know if this is relevant, but the selinux-policy version in the most recent reproducer is not the one requested here: https://bugzilla.redhat.com/show_bug.cgi?id=1382997#c15 Dan's comment requests: selinux-policy-3.13.1-102.el7_3.3 The most recent test used: selinux-policy-3.13.1-102.el7 It seems selinux-policy-3.13.1-102.el7_3.4 is the one meant to be released. See: https://errata.devel.redhat.com/advisory/25114/builds On the existing env where reproduce comment 33, update selinux-policy to 3.13.1-102.el7_3.4, still reproduce this issue. 1. update selinux-policy and selinux-policy-targeted 2. yum erase docker docker-common container-selinux 3. re-install docker, and start docker service # yum install docker ... Dependencies Resolved ============================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================ Installing: docker x86_64 1:1.12.3-1.el7 rhel7-extra 17 M Installing for dependencies: container-selinux x86_64 1:1.12.3-1.el7 rhel7-extra 79 k docker-common x86_64 1:1.12.3-1.el7 rhel7-extra 9.3 M Transaction Summary ============================================================================================================================================ Install 1 Package (+2 Dependent packages) Total download size: 26 M Installed size: 121 M Is this ok [y/d/N]: y Downloading packages: (1/3): container-selinux-1.12.3-1.el7.x86_64.rpm | 79 kB 00:00:00 (2/3): docker-common-1.12.3-1.el7.x86_64.rpm | 9.3 MB 00:00:03 (3/3): docker-1.12.3-1.el7.x86_64.rpm | 17 MB 00:00:04 -------------------------------------------------------------------------------------------------------------------------------------------- Total 5.6 MB/s | 26 MB 00:00:04 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : 1:container-selinux-1.12.3-1.el7.x86_64 1/3 Failed to create node /usr/sbin/semodule: Failed! Installing : 1:docker-common-1.12.3-1.el7.x86_64 2/3 Installing : 1:docker-1.12.3-1.el7.x86_64 3/3 Verifying : 1:docker-common-1.12.3-1.el7.x86_64 1/3 Verifying : 1:docker-1.12.3-1.el7.x86_64 2/3 Verifying : 1:container-selinux-1.12.3-1.el7.x86_64 3/3 Installed: docker.x86_64 1:1.12.3-1.el7 Dependency Installed: container-selinux.x86_64 1:1.12.3-1.el7 docker-common.x86_64 1:1.12.3-1.el7 Complete! 4. run docker command # docker run -it --rm centos /bin/bash panic: standard_init_linux.go:175: exec user process caused "permission denied" [recovered] panic: standard_init_linux.go:175: exec user process caused "permission denied" goroutine 1 [running, locked to thread]: panic(0x7ec7c0, 0xc820134b60) /usr/lib/golang/src/runtime/panic.go:481 +0x3e6 github.com/urfave/cli.HandleAction.func1(0xc8201092e8) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:478 +0x38e panic(0x7ec7c0, 0xc820134b60) /usr/lib/golang/src/runtime/panic.go:443 +0x4e9 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization.func1(0xc820108bf8, 0xc82001a0b0, 0xc820108d08) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:259 +0x136 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization(0xc820058870, 0x7fed72bb5548, 0xc820134b60) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:277 +0x5b1 main.glob.func8(0xc820076780, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main_unix.go:26 +0x68 reflect.Value.call(0x750ee0, 0x902d00, 0x13, 0x848d08, 0x4, 0xc820109268, 0x1, 0x1, 0x0, 0x0, ...) /usr/lib/golang/src/reflect/value.go:435 +0x120d reflect.Value.Call(0x750ee0, 0x902d00, 0x13, 0xc820109268, 0x1, 0x1, 0x0, 0x0, 0x0) /usr/lib/golang/src/reflect/value.go:303 +0xb1 github.com/urfave/cli.HandleAction(0x750ee0, 0x902d00, 0xc820076780, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:487 +0x2ee github.com/urfave/cli.Command.Run(0x84bbb0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x8e1d40, 0x51, 0x0, ...) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/command.go:191 +0xfec github.com/urfave/cli.(*App).Run(0xc820001680, 0xc82000a100, 0x2, 0x2, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:240 +0xaa4 main.main() /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main.go:137 +0xe24 docker-common-1.12.3-1.el7.x86_64 docker-1.12.3-1.el7.x86_64 docker-rhel-push-plugin-1.12.3-1.el7.x86_64 libselinux-2.5-6.el7.x86_64 libselinux-utils-2.5-6.el7.x86_64 libselinux-python-2.5-6.el7.x86_64 selinux-policy-3.13.1-102.el7_3.4.noarch selinux-policy-targeted-3.13.1-102.el7_3.4.noarch # ls -lZ /usr/bin/docker* -rwxr-xr-x. root root system_u:object_r:docker_exec_t:s0 /usr/bin/docker -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/docker-current -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/dockerd-current -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/docker-proxy -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /usr/bin/docker-storage-setup # matchpathcon /usr/bin/docker* /usr/bin/docker system_u:object_r:docker_exec_t:s0 /usr/bin/docker-current system_u:object_r:bin_t:s0 /usr/bin/dockerd-current system_u:object_r:bin_t:s0 /usr/bin/docker-proxy system_u:object_r:bin_t:s0 /usr/bin/docker-storage-setup system_u:object_r:bin_t:s0 # ps -eZ | grep docker system_u:system_r:unconfined_service_t:s0 2926 ? 00:00:00 dockerd-current system_u:system_r:unconfined_service_t:s0 2933 ? 00:00:00 docker-containe I don't sure if this is relevant, check the above "container-selinux" package install log, see an error: Installing : 1:container-selinux-1.12.3-1.el7.x86_64 1/3 Failed to create node /usr/sbin/semodule: Failed! (In reply to Johnny Liu from comment #37) > > # matchpathcon /usr/bin/docker* > /usr/bin/docker system_u:object_r:docker_exec_t:s0 > /usr/bin/docker-current system_u:object_r:bin_t:s0 > /usr/bin/dockerd-current system_u:object_r:bin_t:s0 > /usr/bin/docker-proxy system_u:object_r:bin_t:s0 > /usr/bin/docker-storage-setup system_u:object_r:bin_t:s0 I notice that the labels are different on my system $ matchpathcon /usr/bin/docker* /usr/bin/docker system_u:object_r:docker_exec_t:s0 /usr/bin/docker-current system_u:object_r:docker_exec_t:s0 /usr/bin/docker-proxy system_u:object_r:docker_exec_t:s0 /usr/bin/docker-storage-setup system_u:object_r:docker_exec_t:s0 /usr/bin/docker-v1.10-migrator-helper system_u:object_r:docker_exec_t:s0 /usr/bin/docker-v1.10-migrator-local system_u:object_r:docker_exec_t:s0 /usr/bin/dockerd-current system_u:object_r:docker_exec_t:s0 All of them have docker_exec_t on mine, while on yours, some have docker_exec_t while others have bin_t. I don't know if that could be the cause. > > # ps -eZ | grep docker > system_u:system_r:unconfined_service_t:s0 2926 ? 00:00:00 dockerd-current > system_u:system_r:unconfined_service_t:s0 2933 ? 00:00:00 docker-containe > > > I don't sure if this is relevant, check the above "container-selinux" > package install log, see an error: > Installing : 1:container-selinux-1.12.3-1.el7.x86_64 > 1/3 > Failed to create node > /usr/sbin/semodule: Failed! I see this on my system too, but it doesn't look like a critical issue. docker works regardless for me. Lokesh, I installed a fresh RHEL7.3-RC-3 then install latest docker and update selinux-policy related RPM packages, the issue still exists. [root@bootp-73-3-203 ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.3 (Maipo) [root@bootp-73-3-203 ~]# rpm -q docker selinux-policy skopeo-containers docker-1.12.3-1.el7.x86_64 selinux-policy-3.13.1-102.el7_3.4.noarch skopeo-containers-0.1.17-0.5.git1f655f3.el7.x86_64 [root@bootp-73-3-203 ~]# matchpathcon /usr/bin/docker* /usr/bin/docker system_u:object_r:docker_exec_t:s0 /usr/bin/docker-current system_u:object_r:bin_t:s0 /usr/bin/dockerd-current system_u:object_r:bin_t:s0 /usr/bin/docker-proxy system_u:object_r:bin_t:s0 /usr/bin/docker-storage-setup system_u:object_r:bin_t:s0 [root@bootp-73-3-203 ~]# docker run -it --rm busybox /bin/sh panic: standard_init_linux.go:175: exec user process caused "permission denied" [recovered] panic: standard_init_linux.go:175: exec user process caused "permission denied" goroutine 1 [running, locked to thread]: panic(0x7ec7c0, 0xc82012d030) /usr/lib/golang/src/runtime/panic.go:481 +0x3e6 github.com/urfave/cli.HandleAction.func1(0xc8200fb2e8) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:478 +0x38e panic(0x7ec7c0, 0xc82012d030) /usr/lib/golang/src/runtime/panic.go:443 +0x4e9 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization.func1(0xc8200fabf8, 0xc82001e0c8, 0xc8200fad08) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:259 +0x136 github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization(0xc82005d590, 0x7fbec8851728, 0xc82012d030) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:277 +0x5b1 main.glob.func8(0xc82007aa00, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main_unix.go:26 +0x68 reflect.Value.call(0x750ee0, 0x902d00, 0x13, 0x848d08, 0x4, 0xc8200fb268, 0x1, 0x1, 0x0, 0x0, ...) /usr/lib/golang/src/reflect/value.go:435 +0x120d reflect.Value.Call(0x750ee0, 0x902d00, 0x13, 0xc8200fb268, 0x1, 0x1, 0x0, 0x0, 0x0) /usr/lib/golang/src/reflect/value.go:303 +0xb1 github.com/urfave/cli.HandleAction(0x750ee0, 0x902d00, 0xc82007aa00, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:487 +0x2ee github.com/urfave/cli.Command.Run(0x84bbb0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x8e1d40, 0x51, 0x0, ...) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/command.go:191 +0xfec github.com/urfave/cli.(*App).Run(0xc820001b00, 0xc82000a100, 0x2, 0x2, 0x0, 0x0) /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/Godeps/_workspace/src/github.com/urfave/cli/app.go:240 +0xaa4 main.main() /builddir/build/BUILD/docker-5759a0805380f1067386e87b64f0e27ed818be27/runc-aa860715c2e8ff4ab736a0168907ea975bf28f0e/main.go:137 +0xe24 Try yum reinstall container-selinux And see if it succeeds? We need to install using semodule command that looks like /usr/sbin/semodule -n -X 200 -s targeted -i $MODULES -X 100 -r docker -X 200 -r docker -X 400 -r docker The current code will only remove docker policy if installed at level 200, this will remove it if it is installed at 100, 200, and 400 levels. It would be nice if semodule had -X 0 which told the command to find docker at all levels and remove it. SELinux guys we need to remove the docker.pp module from selinux-policy or at least replace it with the container.pp module Seem like this issue is fixed on docker-1.12.3-2.el7.x86_64. docker-rhel-push-plugin-1.12.3-2.el7.x86_64 docker-common-1.12.3-2.el7.x86_64 docker-1.12.3-2.el7.x86_64 selinux-policy-targeted-3.13.1-102.el7.noarch selinux-policy-3.13.1-102.el7.noarch libselinux-2.5-6.el7.x86_64 libselinux-utils-2.5-6.el7.x86_64 libselinux-python-2.5-6.el7.x86_64 container-selinux-1.12.3-2.el7.x86_64 # getenforce Enforcing # ls -lZ /usr/bin/docker* -rwxr-xr-x. root root system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker -rwxr-xr-x. root root system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-current -rwxr-xr-x. root root system_u:object_r:container_runtime_exec_t:s0 /usr/bin/dockerd-current -rwxr-xr-x. root root system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-proxy -rwxr-xr-x. root root system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-storage-setup # matchpathcon /usr/bin/docker* /usr/bin/docker system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-current system_u:object_r:container_runtime_exec_t:s0 /usr/bin/dockerd-current system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-proxy system_u:object_r:container_runtime_exec_t:s0 /usr/bin/docker-storage-setup system_u:object_r:container_runtime_exec_t:s0 # ps -eZ | grep docker system_u:system_r:container_runtime_t:s0 10857 ? 00:00:12 dockerd-current system_u:system_r:container_runtime_t:s0 10864 ? 00:00:00 docker-containe # docker run -it --rm centos /bin/bash Unable to find image 'centos:latest' locally Trying to pull repository registry.access.redhat.com/centos ... Trying to pull repository docker.io/library/centos ... sha256:b2f9d1c0ff5f87a4743104d099a3d561002ac500db1b9bfa02a783a46e0d366c: Pulling from docker.io/library/centos 08d48e6f1cff: Pull complete Digest: sha256:b2f9d1c0ff5f87a4743104d099a3d561002ac500db1b9bfa02a783a46e0d366c Status: Downloaded newer image for docker.io/centos:latest [root@1976ae529351 /]# exit exit *** Bug 1390269 has been marked as a duplicate of this bug. *** *** Bug 1391963 has been marked as a duplicate of this bug. *** There still is a bug in container-selinux on removal of the container policy semodule -r container Should be semodule -X 200 -r container -e docker In order to reverse what happens on install. should be fixed in https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=522174 The container with selinux enalbed works fine for me and per comment#44, move to vierifed Since now we are shipping docker-1.10, this needs to be fixed in docker-selinux to pick up the fixes in the comment #43 and comment #47. Otherwise, we easily ends up with the wrong labeling as in the BZ. CAI, are you saying you installed the new packages and you saw them mislabeled. Most of the problems reported here were caused by changing docker-selinux -> container-selinux. selinux-policy package ships a docker.pp package and we were replacing it with a container.pp. This was causing the issue. Going back to docker-selinux should eliminate the issue until we switch to docker-1.12 in the next release. Yes, I still saw those mislabeling using docker-selinux from 1.10. The policy was disabled. # semodule -lfull | grep docker 200 docker pp disabled 100 docker pp disabled # semodule -n --priority=200 -s targeted -i /usr/share/selinux/packages/docker.pp.bz2 libsemanage.semanage_direct_install_info: docker module will be disabled after install due to default enabled status. I have to manually remove those before everything works again. Also, the policy is still left over after packages uninstalled. # semodule -lfull | grep docker 200 docker pp 100 docker pp Container-selinux cause docker.pp to be disabled. semodule -e -X200 docker Should enable it. Since our customers have never gotten container-selinux this should not be a problem for them. Per comment#54, move to verified in docker-1.10.3-58.el7 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2859.html (In reply to Daniel Walsh from comment #54) > Container-selinux cause docker.pp to be disabled. > > semodule -e -X200 docker > > Should enable it. Since our customers have never gotten container-selinux > this should not be a problem for them. Daniel, need users to manually configure selinux policy like the following CLI [1]? but I got a question, the /usr is read only in Atomic Host, although I switched Atomic Host to development mode, but I can't still generate policy under the /usr/share/selinux/packages [2]. [1] # semodule -n --priority=200 -s targeted -i /usr/share/selinux/packages/docker.pp.bz2 # semodule -e -X200 docker [2] [root@atomic-00 cloud-user]# atomic host status State: idle Deployments: ● rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard Version: 7.3.2 (2016-12-15 20:05:50) Commit: e6ec3ba29da50d459e54dfa988480498b408125d674db55baef355c4ef0b636d OSName: rhel-atomic-host Unlocked: development [root@atomic-00 cloud-user]# semodule -n --priority=200 -s targeted -i /usr/share/selinux/packages/docker.pp.bz2 libsemanage.map_file: Unable to open /usr/share/selinux/packages/docker.pp.bz2 (No such file or directory). libsemanage.semanage_direct_install_file: Unable to read file /usr/share/selinux/packages/docker.pp.bz2 (No such file or directory). semodule: Failed on /usr/share/selinux/packages/docker.pp.bz2! docker.pp is shipped by the distribution (selinux-policy-targeted package), so their is no pp file, I believe. semodule -n -e docker Should enable the docker policy if it is installed. |