Bug 1433223

Summary: Docker refuses to start containers on RHEL 7.2
Product: Red Hat Enterprise Linux 7 Reporter: Gan Huang <ghuang>
Component: container-selinuxAssignee: Lokesh Mandvekar <lsm5>
Status: CLOSED ERRATA QA Contact: Martin Jenner <mjenner>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: ajia, amurdaca, anli, cevich, ddarrah, dwalsh, ghuang, lsm5, lsu, santiago, wmeng
Target Milestone: rcKeywords: Extras
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: container-selinux-2.10-2.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-04-12 14:50:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Gan Huang 2017-03-17 05:46:58 UTC
Description of problem:
Docker refuses to start containers on RHEL 7.2

Version-Release number of selected component (if applicable):
3.10.0-327.49.2.el7.x86_64
Red Hat Enterprise Linux Server release 7.2 (Maipo)

docker-1.12.6-13.el7.x86_64
docker-client-1.12.6-13.el7.x86_64
docker-common-1.12.6-13.el7.x86_64
docker-rhel-push-plugin-1.12.6-13.el7.x86_64

libselinux-2.2.2-6.el7.x86_64
libselinux-python-2.2.2-6.el7.x86_64
container-selinux-2.10-1.el7.noarch
selinux-policy-targeted-3.13.1-60.el7_2.13.noarch
libselinux-utils-2.2.2-6.el7.x86_64
selinux-policy-3.13.1-60.el7_2.13.noarch

How reproducible:
always

Steps to Reproduce:
# docker run --rm busybox echo 'hello'
Unable to find image 'busybox:latest' locally
Trying to pull repository registry.access.redhat.com/busybox ... 
Trying to pull repository docker.io/library/busybox ... 
latest: Pulling from docker.io/library/busybox
7520415ce762: Pull complete 
Digest: sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
panic: standard_init_linux.go:178: exec user process caused "permission denied" [recovered]
    panic: standard_init_linux.go:178: exec user process caused "permission denied"

goroutine 1 [running, locked to thread]:
panic(0x6f3000, 0xc42012e9a0)
    /usr/lib/golang/src/runtime/panic.go:500 +0x1a1
github.com/urfave/cli.HandleAction.func1(0xc42007f748)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/app.go:478 +0x247
panic(0x6f3000, 0xc42012e9a0)
    /usr/lib/golang/src/runtime/panic.go:458 +0x243
github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization.func1(0xc42007f198, 0xc42001e098, 0xc42007f238)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:259 +0x18f
github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization(0xc42004e780, 0xaac9c0, 0xc42012e9a0)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:277 +0x353
main.glob..func8(0xc420082780, 0x0, 0x0)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/main_unix.go:26 +0x66
reflect.Value.call(0x6ddd80, 0x769ce8, 0x13, 0x73c1c9, 0x4, 0xc42007f708, 0x1, 0x1, 0x4d17a8, 0x732020, ...)
    /usr/lib/golang/src/reflect/value.go:434 +0x5c8
reflect.Value.Call(0x6ddd80, 0x769ce8, 0x13, 0xc42007f708, 0x1, 0x1, 0xac2700, 0xc42007f6e8, 0x4da786)
    /usr/lib/golang/src/reflect/value.go:302 +0xa4
github.com/urfave/cli.HandleAction(0x6ddd80, 0x769ce8, 0xc420082780, 0x0, 0x0)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/app.go:487 +0x1e0
github.com/urfave/cli.Command.Run(0x73c395, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x74d9d9, 0x51, 0x0, ...)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/command.go:191 +0xc3b
github.com/urfave/cli.(*App).Run(0xc4200c6000, 0xc42000c120, 0x2, 0x2, 0x0, 0x0)
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/app.go:240 +0x611
main.main()
    /builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/main.go:137 +0xbd6



Actual results:


Expected results:


Additional info:
# ps -eZ | grep docker
system_u:system_r:unconfined_service_t:s0 10311 ? 00:00:01 dockerd-current
system_u:system_r:unconfined_service_t:s0 10318 ? 00:00:00 docker-containe

# ausearch -m avc -ts recent
<no matches>

# docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.6
Storage Driver: devicemapper
 Pool Name: docker-253:1-42027658-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 11.8 MB
 Data Space Total: 107.4 GB
 Data Space Available: 41.28 GB
 Metadata Space Used: 581.6 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Thin Pool Minimum Free Space: 10.74 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.135-RHEL7 (2016-11-16)
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: bridge null host overlay
 Authorization: rhel-push-plugin
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Security Options: seccomp selinux
Kernel Version: 3.10.0-327.49.2.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 2
Total Memory: 3.702 GiB
Name: jenkins-service-openstack-images-rhel-7.2-updated-165-073d9a5ca
ID: SSDJ:PTV2:STVF:LT3C:HKZI:BIJG:RH3A:NEKD:EGIB:OO6J:G46A:GO7V
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://registry.access.redhat.com/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Insecure Registries:
 127.0.0.0/8
Registries: registry.access.redhat.com (secure), docker.io (secure)

Comment 2 Alex Jia 2017-03-17 10:51:41 UTC
(In reply to Gan Huang from comment #0)
> Description of problem:
> Docker refuses to start containers on RHEL 7.2
> 
> Version-Release number of selected component (if applicable):
> 3.10.0-327.49.2.el7.x86_64
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
> 
> docker-1.12.6-13.el7.x86_64
> docker-client-1.12.6-13.el7.x86_64
> docker-common-1.12.6-13.el7.x86_64
> docker-rhel-push-plugin-1.12.6-13.el7.x86_64
> 
> libselinux-2.2.2-6.el7.x86_64
> libselinux-python-2.2.2-6.el7.x86_64
> container-selinux-2.10-1.el7.noarch
> selinux-policy-targeted-3.13.1-60.el7_2.13.noarch
> libselinux-utils-2.2.2-6.el7.x86_64
> selinux-policy-3.13.1-60.el7_2.13.noarch
> 

Please update your libselinux and selinux-policy version, the docker-1.12.6-13.el7.x86_64 and docker-1.12.6-14.el7.x86_64 both works well for me.

[root@dell-per630-02 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.3 (Maipo)

[root@dell-per630-02 ~]# rpm -q docker container-selinux selinux-policy libselinux
docker-1.12.6-13.el7.x86_64
container-selinux-2.10-1.el7.noarch
selinux-policy-3.13.1-130.el7.noarch
libselinux-2.5-10.el7.x86_64
[root@dell-per630-02 ~]# docker run -it --rm busybox /bin/sh
/ # exit

Comment 3 Daniel Walsh 2017-03-17 12:36:06 UTC
Yes this looks like the 7.2 update problem we have been seeing.
Usually reinstalling container-selinux fixes the issue.

yum reinstall container-selinux
systemctl restart docker

The problem is caused by the extras branch which we are shipping docker and friends in.  Extras does not follow the standard Z Stream efforts, it is more of a rolling release.  So when you update a 7.2 version to the latest docker, it pulls in the 7.3 selinux-policy and container-selinux, this seems to cause container-selinux installation to fail, reinstalling it fixes the issue, and then docker will run with the proper labels.

Comment 5 Gan Huang 2017-03-17 14:02:02 UTC
(In reply to Daniel Walsh from comment #3)
> Yes this looks like the 7.2 update problem we have been seeing.
> Usually reinstalling container-selinux fixes the issue.
> 
> yum reinstall container-selinux
> systemctl restart docker
> 

Unfortunately hit the issue after reinstalling container-selinux, "/usr/sbin/semodule: invalid option -- 'X'" was seen when reinstalling container-selinux.

# yum reinstall container-selinux
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
fast-datapath                                                                                                                                         | 4.0 kB  00:00:00     
pulp-rhel-7.2-server-rpms                                                                                                                             | 3.5 kB  00:00:00     
rhel7                                                                                                                                                 | 4.1 kB  00:00:00     
rhel7-extra                                                                                                                                           | 3.0 kB  00:00:00     
rhel7-extra/primary_db                                                                                                                                |  40 kB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package container-selinux.noarch 2:2.10-1.el7 will be reinstalled
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================
 Package                                        Arch                                Version                                   Repository                                Size
=============================================================================================================================================================================
Reinstalling:
 container-selinux                              noarch                              2:2.10-1.el7                              rhel7-extra                               26 k

Transaction Summary
=============================================================================================================================================================================
Reinstall  1 Package

Total download size: 26 k
Installed size: 32 k
Is this ok [y/d/N]: y
Downloading packages:
container-selinux-2.10-1.el7.noarch.rpm                                                                                                               |  26 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : 2:container-selinux-2.10-1.el7.noarch                                                                                                                     1/1 
/usr/sbin/semodule: invalid option -- 'X'
  Verifying  : 2:container-selinux-2.10-1.el7.noarch                                                                                                                     1/1 

Installed:
  container-selinux.noarch 2:2.10-1.el7                                                                                                                                      

Complete!
[root@jenkins-service-openstack-images-rhel-7 ~]# systemctl restart docker
[root@jenkins-service-openstack-images-rhel-7 ~]# docker run -it --rm busybox /bin/sh
panic: standard_init_linux.go:178: exec user process caused "permission denied" [recovered]
	panic: standard_init_linux.go:178: exec user process caused "permission denied"

goroutine 1 [running, locked to thread]:
panic(0x6f3000, 0xc42011e9a0)
	/usr/lib/golang/src/runtime/panic.go:500 +0x1a1
github.com/urfave/cli.HandleAction.func1(0xc42007f748)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/app.go:478 +0x247
panic(0x6f3000, 0xc42011e9a0)
	/usr/lib/golang/src/runtime/panic.go:458 +0x243
github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization.func1(0xc42007f198, 0xc42001e098, 0xc42007f238)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:259 +0x18f
github.com/opencontainers/runc/libcontainer.(*LinuxFactory).StartInitialization(0xc42004f400, 0xaac9c0, 0xc42011e9a0)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/factory_linux.go:277 +0x353
main.glob..func8(0xc420082a00, 0x0, 0x0)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/main_unix.go:26 +0x66
reflect.Value.call(0x6ddd80, 0x769ce8, 0x13, 0x73c1c9, 0x4, 0xc42007f708, 0x1, 0x1, 0x4d17a8, 0x732020, ...)
	/usr/lib/golang/src/reflect/value.go:434 +0x5c8
reflect.Value.Call(0x6ddd80, 0x769ce8, 0x13, 0xc42007f708, 0x1, 0x1, 0xac2700, 0xc42007f6e8, 0x4da786)
	/usr/lib/golang/src/reflect/value.go:302 +0xa4
github.com/urfave/cli.HandleAction(0x6ddd80, 0x769ce8, 0xc420082a00, 0x0, 0x0)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/app.go:487 +0x1e0
github.com/urfave/cli.Command.Run(0x73c395, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x74d9d9, 0x51, 0x0, ...)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/command.go:191 +0xc3b
github.com/urfave/cli.(*App).Run(0xc4200ac000, 0xc42000c120, 0x2, 0x2, 0x0, 0x0)
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/Godeps/_workspace/src/github.com/urfave/cli/app.go:240 +0x611
main.main()
	/builddir/build/BUILD/docker-3a094bd354f91c08f76655da01921369b04714b5/runc-81b254244390bc636b20c87c34a3d9e1a8645069/main.go:137 +0xbd6

Comment 6 Gan Huang 2017-03-17 14:12:52 UTC
(In reply to Alex Jia from comment #2)

> Please update your libselinux and selinux-policy version, the
> docker-1.12.6-13.el7.x86_64 and docker-1.12.6-14.el7.x86_64 both works well
> for me.
> 
Hit another issue after updating these packages to latest. Note: we only hit this issue on RHEL 7.2. I suggest you re-test it on RHEL 7.2 if you don't mind.

# rpm -q docker container-selinux selinux-policy libselinux
docker-1.12.6-13.el7.x86_64
container-selinux-2.10-1.el7.noarch
selinux-policy-3.13.1-129.el7.noarch
libselinux-2.5-10.el7.x86_64

# systemctl restart docker

# docker run -it --rm busybox /bin/sh
/usr/bin/docker-current: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:334: running prestart hook 1 caused \\\"error running hook: exit status 1, stdout: , stderr: \\\"\"\n".

Comment 7 Daniel Walsh 2017-03-17 14:26:33 UTC
So this looks like we need to add a policycoreutils update requirement to container-selinux.

The problem is we need the semanage from RHEL7.3 which has the new -X Flag.

Comment 10 Chris Evich 2017-03-20 13:49:46 UTC
New card: https://trello.com/c/id2OeNeL

Comment 11 Daniel Walsh 2017-03-20 14:33:22 UTC
Ok I just kicked off builds of container-selinux 
container-selinux-2.10-2.1.el7 for 7.4
container-selinux-2.10-2.el7 for 7.3

Which require an updated version of policycoreutils.  This will hopefully solve the issue we are seeing on container-selinux installation on 7.2 boxes.

Comment 12 Daniel Walsh 2017-03-20 14:34:56 UTC
Fixed in container-selinux-2.10-2.el7

When testing make sure we test a 7.2 box updating to latest extras packages.

Comment 25 Alex Jia 2017-04-01 09:19:16 UTC
I can reproduce the bug, and it works well in container-selinux-2.10-2.el7 w/ the following RPMs:

[root@localhost ~]# rpm -q docker container-selinux oci-register-machine oci-systemd-hook skopeo-containers policycoreutils libselinux libsemanage libsepol systemd
docker-1.12.6-13.el7.x86_64
container-selinux-2.10-2.el7.noarch
oci-register-machine-0-3.11.1.gitdd0daef.el7.x86_64
oci-systemd-hook-0.1.7-1.git1788cf2.el7.x86_64
skopeo-containers-0.1.18-1.2.el7.x86_64
policycoreutils-2.5-11.el7_3.x86_64
libselinux-2.5-6.el7.x86_64
libsemanage-2.5-5.el7_3.x86_64
libsepol-2.5-6.el7.x86_64
systemd-219-20.el7.x86_64

[root@localhost ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)

Comment 27 errata-xmlrpc 2017-04-12 14:50:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0942