Bug 1186669 - Failed to start Docker Application Container Engine.
Summary: Failed to start Docker Application Container Engine.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: selinux-policy
Version: 21
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Miroslav Grepl
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-01-28 09:50 UTC by colin
Modified: 2015-03-31 21:46 UTC (History)
19 users (show)

Fixed In Version: selinux-policy-3.13.1-105.9.fc21
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-31 21:46:27 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
grep -i 'docker' /var/log/audit/audit.log | tail (2.05 KB, text/plain)
2015-01-28 09:50 UTC, colin
no flags Details
'yum localinstall selinux-policy-* | tee yum_localinstall_koji-selinux-policy-3.13.1-105.9' (8.50 KB, text/plain)
2015-03-24 15:42 UTC, colin
no flags Details

Description colin 2015-01-28 09:50:08 UTC
Created attachment 985059 [details]
grep -i 'docker' /var/log/audit/audit.log | tail

Description of problem:

I yum-updated to F21 latest and now docker service fails to start on boot.
This update included reboot to new kernel 3.18

[root@vm117 ~]# uname -a
Linux vm117 3.18.3-201.fc21.x86_64 #1 SMP Mon Jan 19 15:59:31 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Maybe this could be related to the other new warning that I see in 
joutnalctl -b

SELinux is preventing sm-notify from write access on the file nlm_end_grace. For complete SELinux messages. run sealert

and
Dependency failed for Network Manager Wait Online.

Which I assume is new with 3.18 since web search found this:
https://www.kernel.org/pub/linux/utils/nfs-utils/1.3.1/1.3.1-Changelog

I don't see anything notable in /var/log/messages

Will attach a grep of /var/log/audit/audit.log

Version-Release number of selected component (if applicable):

[root@vm117 ~]# yum info docker-io.x86_64
Loaded plugins: langpacks
Installed Packages
Name        : docker-io
Arch        : x86_64
Version     : 1.4.1
Release     : 5.fc21
Size        : 19 M
Repo        : installed
From repo   : updates


How reproducible:

100% Service Fails to start on every boot.

Steps to Reproduce:
1. Reboot Fedora
2. systemctl status docker.service
3.

Actual results:


Expected results:


Additional info:

Comment 1 colin 2015-02-06 21:57:04 UTC
I yum updated to latest 
docker-io.x86_64 0:1.4.1-8.fc21

(and updated SELinux for RHBZ1181338) 
see: https://bugzilla.redhat.com/show_bug.cgi?id=1181338

But docker still fails to start on boot.

Is there anything that I can do to investigate this issue further?

thanks
Colin.

Comment 2 Lokesh Mandvekar 2015-02-07 04:22:04 UTC
Hi Colin, 

The docker rpm doesn't use systemd socket activation anymore. 

Please check if your /etc/sysconfig/docker has '-H fd://' included as part of OPTIONS. If yes, please remove it and restart docker. If it still fails, please post here what 'systemctl status docker.service' spits out.

Also, docker group has been gotten rid of in the defaults. But you can still manually create docker group and add users to it if you prefer to avoid sudo (though it'd be a security risk).

Comment 3 Lokesh Mandvekar 2015-02-21 00:05:47 UTC
Colin, does this work for you now?

Comment 4 colin 2015-02-23 10:04:45 UTC
Hi Lokesh.

So I did a yum-update and rebooted the VM

upon login to the (SPICE) RemoteViewer MATE desktop
I see 2 SELinux Alerts for openvswitch.

[root@vm117 yum]# docker images 
FATA[0000] Cannot connect to the Docker daemon. Is 'docker -d' running on this host? 

[root@vm117 yum]# systemctl status -l docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)
   Active: failed (Result: timeout) since Mon 2015-02-23 09:38:27 GMT; 22min ago
     Docs: http://docs.docker.com
  Process: 1612 ExecStart=/usr/bin/docker -d $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $INSECURE_REGISTRY (code=exited, status=0/SUCCESS)
 Main PID: 1612 (code=exited, status=0/SUCCESS)

Feb 23 09:36:59 vm117 docker[1612]: time="2015-02-23T09:36:59Z" level="info" msg="+job serveapi(unix:///var/run/docker.sock)"
Feb 23 09:36:59 vm117 docker[1612]: time="2015-02-23T09:36:59Z" level="info" msg="Listening for HTTP on unix (/var/run/docker.sock)"
Feb 23 09:38:04 vm117 python[2253]: SELinux is preventing docker from getattr access on the directory /run/openvswitch.
                                    
                                    *****  Plugin catchall (100. confidence) suggests   **************************
                                    
                                    If you believe that docker should be allowed getattr access on the openvswitch directory by default.
                                    Then you should report this as a bug.
                                    You can generate a local policy module to allow this access.
                                    Do
                                    allow this access for now by executing:
                                    # grep docker /var/log/audit/audit.log | audit2allow -M mypol
                                    # semodule -i mypol.pp
                                    
Feb 23 09:38:04 vm117 python[2253]: SELinux is preventing docker from getattr access on the sock_file /run/openvswitch/db.sock.
                                    
                                    *****  Plugin catchall (100. confidence) suggests   **************************
                                    
                                    If you believe that docker should be allowed getattr access on the db.sock sock_file by default.
                                    Then you should report this as a bug.
                                    You can generate a local policy module to allow this access.
                                    Do
                                    allow this access for now by executing:
                                    # grep docker /var/log/audit/audit.log | audit2allow -M mypol
                                    # semodule -i mypol.pp
                                    
Feb 23 09:38:17 vm117 python[2253]: SELinux is preventing docker from getattr access on the directory /run/openvswitch.
                                    
                                    *****  Plugin catchall (100. confidence) suggests   **************************
                                    
                                    If you believe that docker should be allowed getattr access on the openvswitch directory by default.
                                    Then you should report this as a bug.
                                    You can generate a local policy module to allow this access.
                                    Do
                                    allow this access for now by executing:
                                    # grep docker /var/log/audit/audit.log | audit2allow -M mypol
                                    # semodule -i mypol.pp
                                    
Feb 23 09:38:27 vm117 systemd[1]: docker.service start operation timed out. Terminating.
Feb 23 09:38:27 vm117 docker[1612]: time="2015-02-23T09:38:27Z" level="info" msg="Received signal 'terminated', starting shutdown of docker..."
Feb 23 09:38:27 vm117 systemd[1]: Failed to start Docker Application Container Engine.
Feb 23 09:38:27 vm117 systemd[1]: Unit docker.service entered failed state.
Feb 23 09:38:27 vm117 systemd[1]: docker.service failed.

I do a manual docker start now and this does work.

[root@vm117 yum]# systemctl stop docker.service
[root@vm117 yum]# systemctl start docker.service
[root@vm117 yum]# systemctl status -l docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)
   Active: active (running) since Mon 2015-02-23 10:03:28 GMT; 5s ago
     Docs: http://docs.docker.com
 Main PID: 2550 (docker)
   CGroup: /system.slice/docker.service
           └─2550 /usr/bin/docker -d --selinux-enabled

Feb 23 10:03:27 vm117 python[2507]: SELinux is preventing docker from getattr access on the sock_file /run/openvswitch/db.sock.
                                    
                                    *****  Plugin catchall (100. confidence) suggests   **************************
                                    
                                    If you believe that docker should be allowed getattr access on the db.sock sock_file by default.
                                    Then you should report this as a bug.
                                    You can generate a local policy module to allow this access.
                                    Do
                                    allow this access for now by executing:
                                    # grep docker /var/log/audit/audit.log | audit2allow -M mypol
                                    # semodule -i mypol.pp
                                    
Feb 23 10:03:27 vm117 docker[2550]: time="2015-02-23T10:03:27Z" level="info" msg="-job init_networkdriver() = OK (0)"
Feb 23 10:03:27 vm117 docker[2550]: time="2015-02-23T10:03:27Z" level="info" msg="Loading containers: start."
Feb 23 10:03:28 vm117 docker[2550]: ........time="2015-02-23T10:03:28Z" level="error" msg="Warning: error unmounting device 66c4719685e9ca574429ebed6c16816efd3bba70b9cb19dd81209003a6f8d7b5: UnmountDevice: device not-mounted id 66c4719685e9ca574429ebed6c16816efd3bba70b9cb19dd81209003a6f8d7b5"
Feb 23 10:03:28 vm117 docker[2550]: .time="2015-02-23T10:03:28Z" level="error" msg="Warning: error unmounting device 7d0bef932cf04222a6417685a64f8ed87b1af9a2ce962d4c7439937a671f00fd: UnmountDevice: device not-mounted id 7d0bef932cf04222a6417685a64f8ed87b1af9a2ce962d4c7439937a671f00fd"
Feb 23 10:03:28 vm117 docker[2550]: .....time="2015-02-23T10:03:28Z" level="error" msg="Warning: error unmounting device d3123de348b17df04fc08607bf826ff1d18b3b4fe2c31f521e846749f2624c2f: UnmountDevice: device not-mounted id d3123de348b17df04fc08607bf826ff1d18b3b4fe2c31f521e846749f2624c2f"
Feb 23 10:03:28 vm117 docker[2550]: time="2015-02-23T10:03:28Z" level="info" msg="Loading containers: done."
Feb 23 10:03:28 vm117 docker[2550]: time="2015-02-23T10:03:28Z" level="info" msg="docker daemon: 1.5.0 a8a31ef/1.5.0; execdriver: native-0.2; graphdriver: devicemapper"
Feb 23 10:03:28 vm117 docker[2550]: time="2015-02-23T10:03:28Z" level="info" msg="+job acceptconnections()"
Feb 23 10:03:28 vm117 docker[2550]: time="2015-02-23T10:03:28Z" level="info" msg="-job acceptconnections() = OK (0)"
[root@vm117 yum]#

Comment 5 colin 2015-02-23 10:07:54 UTC
Re: SELinux alert from Comment #4
https://bugzilla.redhat.com/show_bug.cgi?id=1195159

Comment 6 colin 2015-02-23 10:16:19 UTC
Re: comment #2

> Please check if your /etc/sysconfig/docker has '-H fd://' included as part of > OPTIONS. If yes, please remove it and restart docker. If it still fails, 
> please post here what 'systemctl status docker.service' spits out.

[root@vm117 yum]# cat /etc/sysconfig/docker
OPTIONS=--selinux-enabled
DOCKER_CERT_PATH=/etc/docker

# Location used for temporary files, such as those created by
# docker load and build operations. Default is /var/lib/docker/tmp
# Can be overriden by setting the following environment variable.
# DOCKER_TMPDIR=/var/tmp
[root@vm117 yum]#

Comment 7 Daniel Walsh 2015-03-09 18:45:38 UTC
0a70c020b334a2fd2c0f92a3528087ba5bbcae59 fixes this in the selinux-policy git 

We need to get this backported to f22 and f21.

Comment 8 Lukas Vrabec 2015-03-10 09:26:50 UTC
commit bfed9799e203a94b0c7d0ebd19ec7f9844c8b675
Author: Dan Walsh <dwalsh>
Date:   Mon Mar 9 14:43:56 2015 -0400

    Allow docker to relablefrom/to sockets and docker_log_t

Added in F22 and F21

Comment 9 Fedora Update System 2015-03-23 16:47:51 UTC
selinux-policy-3.13.1-105.9.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/selinux-policy-3.13.1-105.9.fc21

Comment 10 colin 2015-03-24 15:33:31 UTC
Hi Lukas

So I went ahead and downloaded

selinux-policy-3.13.1-105.9.fc21.noarch.rpm (info) (download)
	selinux-policy-devel-3.13.1-105.9.fc21.noarch.rpm (info) (download)
	selinux-policy-doc-3.13.1-105.9.fc21.noarch.rpm (info) (download)
	selinux-policy-minimum-3.13.1-105.9.fc21.noarch.rpm (info) (download)
	selinux-policy-mls-3.13.1-105.9.fc21.noarch.rpm (info) (download)
	selinux-policy-sandbox-3.13.1-105.9.fc21.noarch.rpm (info) (download)
	selinux-policy-targeted-3.13.1-105.9.fc21.noarch.rpm (info) (download)

from your Koji test build and installed them with:
'yum localinstall selinux-policy-* | tee yum_localinstall_koji-selinux-policy-3.13.1-105.9'

There were 3 complaints from yum so I will upload the log as attachment.

I rebooted and am happy to report that this time No New AVC warnings have been reported. :-)

I can now manually start docker and run my scripts that start openvpn and openvswitch without causing SELinux warnings.
Brilliant! 

(The only minor niggle is that docker.service still won't start from boot- but I want to move onto a fresh VM now so will wait for all your changes to land for F22)

Thankyou for the support and fixes. :-)
Colin

Comment 11 colin 2015-03-24 15:42:02 UTC
Created attachment 1005909 [details]
'yum localinstall selinux-policy-* | tee yum_localinstall_koji-selinux-policy-3.13.1-105.9'

installation log for manual install of koji test build

http://koji.fedoraproject.org/koji/buildinfo?buildID=622719

 selinux-policy-3.13.1-105.9.fc21

https://admin.fedoraproject.org/updates/selinux-policy-3.13.1-105.9.fc21?_csrf_token=66a10c66c0164aece5ff7e37cf39cd548f2d7172

works for me. :-)
+1 krama left

Comment 12 Fedora Update System 2015-03-26 21:28:03 UTC
Package selinux-policy-3.13.1-105.9.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing selinux-policy-3.13.1-105.9.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-4492/selinux-policy-3.13.1-105.9.fc21
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2015-03-31 21:46:27 UTC
selinux-policy-3.13.1-105.9.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.