Description of problem: Between docker-io-0.10.0-2.fc20 and docker-io-0.11.1-1.fc20, the following has started failing: $ docker run -t centos /usr/sbin/useradd test useradd: failure while writing changes to /etc/passwd 'centos' is the official CentOS 6 image (0b443ba03958). The Fedora 20 host has SELinux enforcing, and the same issue occurs when set to permissive. No AVCs are seen. Version-Release number of selected component (if applicable): docker-io-0.11.1-1.fc20.x86_64 kernel-3.14.2-200.fc20.x86_64 How reproducible: Always Steps to Reproduce: 1. docker pull centos 2. docker run -t centos /usr/sbin/useradd test Actual results: useradd: failure while writing changes to /etc/passwd Expected results: no output Additional info: On 0.10.0, an strace of useradd shows: open("/etc/group", O_RDONLY|O_CLOEXEC) = 11 fstat(11, {st_mode=S_IFREG|0644, st_size=379, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feb5efe5000 read(11, "root:x:0:\nbin:x:1:bin,daemon\ndae"..., 4096) = 379 close(11) = 0 munmap(0x7feb5efe5000, 4096) = 0 fchown(10, 500, 12) = 0 fchmod(10, 0660) = 0 fsync(10) = 0 close(10) = 0 fstat(6, {st_mode=S_IFREG|0644, st_size=670, ...}) = 0 gettid() = 14 open("/proc/self/task/14/attr/fscreate", O_RDONLY) = 10 read(10, "", 4095) = 0 close(10) = 0 gettid() = 14 open("/proc/self/task/14/attr/fscreate", O_RDWR) = 10 write(10, "system_u:object_r:file_t:s0\0", 28) = 28 close(10) = 0 fstat(6, {st_mode=S_IFREG|0644, st_size=670, ...}) = 0 umask(077) = 022 open("/etc/passwd-", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 10 umask(022) = 077 lseek(6, 0, SEEK_SET) = 0 read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 670 fstat(10, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feb5efe5000 read(6, "", 4096) = 0 write(10, "root:x:0:0:root:/root:/bin/bash\n"..., 670) = 670 While on 0.11.1, strace shows: open("/etc/group", O_RDONLY|O_CLOEXEC) = 10 fstat(10, {st_mode=S_IFREG|0644, st_size=379, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2905a38000 read(10, "root:x:0:\nbin:x:1:bin,daemon\ndae"..., 4096) = 379 close(10) = 0 munmap(0x7f2905a38000, 4096) = 0 fchown(9, 500, 12) = 0 fchmod(9, 0660) = 0 fsync(9) = 0 close(9) = 0 fstat(5, {st_mode=S_IFREG|0644, st_size=675, ...}) = 0 gettid() = 30 open("/proc/self/task/30/attr/fscreate", O_RDONLY) = 9 read(9, "", 4095) = 0 close(9) = 0 gettid() = 30 open("/proc/self/task/30/attr/fscreate", O_RDWR) = -1 EROFS (Read-only file system) write(2, "useradd: failure while writing c"..., 54useradd: failure while writing changes to /etc/passwd ) = 54
useradd is just calling libselinux's setfscreatecon, which is being blocked. On 0.11.1, this library call fails, on 0.10.0/0.9 it works.
when selinux=disabled, i cannot reproduce the bug. when selinux is enabled (permissive), the bug is always reproducible. i reproduced using two docker service configs: * first, use default systemd unit * second, add `--selinux-enabled` as described at http://blog.docker.io/2014/05/docker-0-11-release-candidate-for-1-0/ override default systemd unit... # cp /usr/lib/systemd/system/docker.service /etc/systemd/system/ # vim /etc/systemd/system/docker.service # cat !$ [Unit] Description=Docker Application Container Engine Documentation=http://docs.docker.io After=network.target [Service] ExecStart=/usr/bin/docker -d --selinux-enabled Restart=on-failure LimitNOFILE=1048576 LimitNPROC=1048576 [Install] WantedBy=multi-user.target # systemctl restart docker.service # systemctl status docker.service docker.service - Docker Application Container Engine Loaded: loaded (/etc/systemd/system/docker.service; enabled) Active: active (running) since Fri 2014-05-09 09:52:25 EDT; 4s ago Docs: http://docs.docker.io Main PID: 997 (docker) CGroup: /system.slice/docker.service └─997 /usr/bin/docker -d --selinux-enabled other system info: $ rpm -q selinux-policy-targeted selinux-policy-targeted-3.12.1-158.fc20.noarch $ uname -a Linux f20-01.example.com 3.13.9-200.fc20.x86_64 #1 SMP Fri Apr 4 12:13:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux $ rpm -q docker-io docker-io-0.11.1-1.fc20.x86_64 $ docker version Client version: 0.11.1 Client API version: 1.11 Go version (client): go1.2.1 Git commit (client): fb99f99/0.11.1 Server version: 0.11.1 Server API version: 1.11 Git commit (server): fb99f99/0.11.1 Go version (server): go1.2.1 Last stable version: 0.11.1 $ docker images | grep '^centos' centos centos6 0b443ba03958 3 weeks ago 297.6 MB centos latest 0b443ba03958 3 weeks ago 297.6 MB centos 6.4 539c0211cd76 13 months ago 300.6 MB
I have a new scratch build http://kojipkgs.fedoraproject.org//work/tasks/2230/6832230/docker-io-0.11.1-3.fc20.x86_64.rpm with this build, this error doesn't occur with fedora:20, but still does with centos dwalsh, comments?
Comparing the straces between el6 and fedora:20, I don't see any of the same accesses to attr/fscreate on f20 that are in the bug description. The source of shadow-utils between el6 & f20 looks very different too, I can't see any setfscreatecon calls in the useradd code path.
The problem is inside the container it sees SELinux as being enabled, which is the bug. If you do id -Z, does it complain inside the container? docker run --rm -t -i fedora sh sh-4.2# id -Z id: --context (-Z) works only on an SELinux-enabled kernel sh-4.2# mount | grep /sys sysfs on /sys type sysfs (ro,relatime,seclabel) SELinux sees the container as being disabled since /sys/fs/selinux is mounted as read/only, this will tell useradd NOT to try to do any SELinux stuff while in the container.
docker-io-0.11.1-3.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/docker-io-0.11.1-3.fc20
(In reply to Daniel Walsh from comment #5) > The problem is inside the container it sees SELinux as being enabled, which > is the bug. > > If you do id -Z, does it complain inside the container? No, it runs and reports a context. > docker run --rm -t -i fedora sh > sh-4.2# id -Z > id: --context (-Z) works only on an SELinux-enabled kernel > sh-4.2# mount | grep /sys > sysfs on /sys type sysfs (ro,relatime,seclabel) $ rpm -q docker-io docker-io-0.9.1-1.fc20.x86_64 $ docker run -i -t centos /bin/bash bash-4.1# id -Z system_u:system_r:docker_t:s0 bash-4.1# mount | grep sys sysfs on /sys type sysfs (rw,seclabel,nosuid,nodev,noexec,relatime) $ rpm -q docker-io docker-io-0.11.1-3.fc20.x86_64 $ docker run -i -t centos /bin/bash bash-4.1# id -Z system_u:system_r:svirt_lxc_net_t:s0:c231,c400 bash-4.1# mount | grep /sys sysfs on /sys type sysfs (ro,seclabel,relatime) > SELinux sees the container as being disabled since /sys/fs/selinux is > mounted as read/only, this will tell useradd NOT to try to do any SELinux > stuff while in the container. /sys is correctly read-only as you expected, but it seems useradd's still doing SELinux stuff then. These packages are installed inside the EL6 container: libselinux-2.0.94-5.3.el6_4.1.x86_64 libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 shadow-utils-4.1.4.2-13.el6.x86_64 Calling is_selinux_enabled() on Fedora is returning 0, while on EL6 it's returning 1. Another difference - on Fedora, getenforce returns "Disabled" but on EL6 it prints: # getenforce getenforce: getenforce() failedbash-4.1# /selinux exists within the container, but nothing is actually mounted there. It appears to be simply a directory on the root filesystem (/selinux/booleans exists as an empty dir). No other SELinux mounts are visible. Looking at libselinux-2.0.94, I think it's seeing selinuxfs listed in /proc/filesystems and assuming SELinux is enabled because of this. libselinux-2.2.1 on F20 doesn't seem to have this code. libselinux-2.0.94/src/enabled.c: /* Drop back to detecting it the long way. */ fp = fopen("/proc/filesystems", "r"); if (!fp) return -1; __fsetlocking(fp, FSETLOCKING_BYCALLER); while ((num = getline(&buf, &len, fp)) != -1) { if (strstr(buf, "selinuxfs")) { enabled = 1; break; } } # grep selinux /proc/filesystems nodev selinuxfs (All the above was tested with docker-io-0.11.1-3.fc20)
Package docker-io-0.11.1-3.fc20: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing docker-io-0.11.1-3.fc20' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-6281/docker-io-0.11.1-3.fc20 then log in and leave karma (feedback).
(In reply to Dominic Cleal from comment #7) > Looking at libselinux-2.0.94, I think it's seeing selinuxfs listed in > /proc/filesystems and assuming SELinux is enabled because of this. > libselinux-2.2.1 on F20 doesn't seem to have this code. Bug #835146 (against EL6) seems to confirm this, suggesting a backport of the patch that removes the /proc/filesystems based check.
Hi All, I had a similar problem while running a stable docker-io-0.11.1-1.fc20. Then following the updates here i installed the package from koji (yum update --enablerepo=updates-testing docker-io-0.11.1-3.fc20). Now i run into this issue [prudhvi@tau ~]$ cat /etc/redhat-release Fedora release 20 (Heisenbug) [prudhvi@tau ~]$ rpm -qa | grep docker docker-io-0.11.1-3.fc20.x86_64 [prudhvi@tau ~]$ docker run -i -t centos:latest /bin/bash /bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory From the audit.log with selinux enabled type=NETFILTER_CFG msg=audit(1400024895.340:918): table=filter family=2 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=raw family=2 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=security family=2 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=mangle family=2 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=nat family=2 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=filter family=10 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=raw family=10 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=security family=10 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=mangle family=10 entries=0 type=NETFILTER_CFG msg=audit(1400024895.340:918): table=nat family=10 entries=0 type=SYSCALL msg=audit(1400024895.340:918): arch=c000003e syscall=56 success=yes exit=11327 a0=6c020011 a1=0 a2=0 a3=0 items=0 ppid=1 pid=11262 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid =0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="docker" exe="/usr/bin/docker" subj=system_u:system_r:docker_t:s0 key=(null) type=ANOM_PROMISCUOUS msg=audit(1400024895.509:919): dev=vethd2bd prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295 type=SYSCALL msg=audit(1400024895.509:919): arch=c000003e syscall=16 success=yes exit=0 a0=11 a1=89a2 a2=c2102057a0 a3=0 items=0 ppid=1 pid=11258 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsu id=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="docker" exe="/usr/bin/docker" subj=system_u:system_r:docker_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.548:920): table=filter family=2 entries=124 type=SYSCALL msg=audit(1400024895.548:920): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=1352630 items=0 ppid=699 pid=11339 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.552:921): table=filter family=10 entries=122 type=SYSCALL msg=audit(1400024895.552:921): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=1c8db50 items=0 ppid=699 pid=11340 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.556:922): table=filter family=2 entries=125 type=SYSCALL msg=audit(1400024895.556:922): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=1b497a0 items=0 ppid=699 pid=11341 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.560:923): table=filter family=10 entries=123 type=SYSCALL msg=audit(1400024895.560:923): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=8ccd40 items=0 ppid=699 pid=11342 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=AVC msg=audit(1400024895.562:924): avc: denied { read } for pid=11327 comm="bash" name="ld.so.cache" dev="sdb1" ino=4163 scontext=system_u:system_r:svirt_lxc_net_t:s0:c702,c947 tconte xt=system_u:object_r:file_t:s0 tclass=file type=SYSCALL msg=audit(1400024895.562:924): arch=c000003e syscall=2 success=no exit=-13 a0=7f0ee5883201 a1=0 a2=1 a3=0 items=0 ppid=0 pid=11327 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid =0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=4294967295 comm="bash" exe="/bin/bash" subj=system_u:system_r:svirt_lxc_net_t:s0:c702,c947 key=(null) type=AVC msg=audit(1400024895.562:925): avc: denied { read } for pid=11327 comm="bash" name="libtinfo.so.5" dev="sdb1" ino=4655 scontext=system_u:system_r:svirt_lxc_net_t:s0:c702,c947 tcon text=system_u:object_r:file_t:s0 tclass=lnk_file type=SYSCALL msg=audit(1400024895.562:925): arch=c000003e syscall=2 success=no exit=-13 a0=7fff43237600 a1=0 a2=0 a3=ffffffff items=0 ppid=0 pid=11327 auid=4294967295 uid=0 gid=0 euid=0 suid= 0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=4294967295 comm="bash" exe="/bin/bash" subj=system_u:system_r:svirt_lxc_net_t:s0:c702,c947 key=(null) type=NETFILTER_CFG msg=audit(1400024895.565:926): table=filter family=2 entries=126 type=SYSCALL msg=audit(1400024895.565:926): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=96b920 items=0 ppid=699 pid=11343 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.571:927): table=filter family=10 entries=124 type=SYSCALL msg=audit(1400024895.571:927): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=2048f30 items=0 ppid=699 pid=11345 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.575:928): table=mangle family=2 entries=52 type=SYSCALL msg=audit(1400024895.575:928): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=1c6c2d0 items=0 ppid=699 pid=11346 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.580:929): table=mangle family=10 entries=52 type=SYSCALL msg=audit(1400024895.580:929): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=11f32a0 items=0 ppid=699 pid=11347 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.583:930): table=nat family=2 entries=81 type=SYSCALL msg=audit(1400024895.583:930): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=13df4b0 items=0 ppid=699 pid=11348 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.587:931): table=nat family=10 entries=76 type=SYSCALL msg=audit(1400024895.587:931): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=1a994f0 items=0 ppid=699 pid=11349 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.591:932): table=nat family=2 entries=82 type=SYSCALL msg=audit(1400024895.591:932): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=1173630 items=0 ppid=699 pid=11350 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.597:933): table=nat family=10 entries=77 type=SYSCALL msg=audit(1400024895.597:933): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=21eb6e0 items=0 ppid=699 pid=11351 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=ANOM_PROMISCUOUS msg=audit(1400024895.603:934): dev=vethd2bd prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295 type=NETFILTER_CFG msg=audit(1400024895.614:935): table=filter family=2 entries=127 type=SYSCALL msg=audit(1400024895.614:935): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=118f8e0 items=0 ppid=699 pid=11359 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.618:936): table=filter family=10 entries=125 type=SYSCALL msg=audit(1400024895.618:936): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=ce7ef0 items=0 ppid=699 pid=11360 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.624:937): table=filter family=2 entries=126 type=SYSCALL msg=audit(1400024895.624:937): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=1b86770 items=0 ppid=699 pid=11362 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.629:938): table=filter family=10 entries=124 type=SYSCALL msg=audit(1400024895.629:938): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=c6bd00 items=0 ppid=699 pid=11363 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.634:939): table=filter family=2 entries=125 type=SYSCALL msg=audit(1400024895.634:939): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=12975f0 items=0 ppid=699 pid=11364 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.637:940): table=filter family=10 entries=123 type=SYSCALL msg=audit(1400024895.637:940): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=14a3b10 items=0 ppid=699 pid=11365 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.641:941): table=mangle family=2 entries=53 type=SYSCALL msg=audit(1400024895.641:941): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=aed2a0 items=0 ppid=699 pid=11366 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.644:942): table=mangle family=10 entries=53 type=SYSCALL msg=audit(1400024895.644:942): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=253b260 items=0 ppid=699 pid=11367 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.648:943): table=nat family=2 entries=83 type=SYSCALL msg=audit(1400024895.648:943): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=1c8d5f0 items=0 ppid=699 pid=11368 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.652:944): table=nat family=10 entries=78 type=SYSCALL msg=audit(1400024895.652:944): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=18436a0 items=0 ppid=699 pid=11370 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.656:945): table=nat family=2 entries=82 type=SYSCALL msg=audit(1400024895.656:945): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=0 a2=40 a3=19c0480 items=0 ppid=699 pid=11371 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null) type=NETFILTER_CFG msg=audit(1400024895.660:946): table=nat family=10 entries=77 type=SYSCALL msg=audit(1400024895.660:946): arch=c000003e syscall=54 success=yes exit=0 a0=3 a1=29 a2=40 a3=1d504b0 items=0 ppid=699 pid=11372 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid= 0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:iptables_t:s0 key=(null)
(In reply to Prudhvi Surapaneni from comment #11) > Hi All, > > I had a similar problem while running a stable docker-io-0.11.1-1.fc20. Then > following the updates here i installed the package from koji (yum update > --enablerepo=updates-testing docker-io-0.11.1-3.fc20). Now i run into this > issue I noticed some AVCs when using an older selinux-policy, so ensure this is up to date from updates-testing (the changelog suggests 3.12.1-159 or higher). Perhaps run restorecon -R against /var/lib/docker too.
Prudhvi there are no AVC's in that log. You can gather AVC's with the following command ausearch -m AVC,USER_AVC -ts recent -i
Hi Daniel, Here is the list of AVC's ---- type=SYSCALL msg=audit(05/14/2014 10:26:30.241:1174) : arch=x86_64 syscall=open success=no exit=-13(Permission denied) a0=0x7f4e74733201 a1=O_RDONLY a2=0x1 a3=0x0 items=0 ppid=0 pid=6246 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts3 ses=unset comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c596 c698 key=(null) type=AVC msg=audit(05/14/2014 10:26:30.241:1174) : avc: denied { read } for pid=6246 comm=bash name=ld.so.cache dev="sdb1" ino=4163 scontext=system_u:system_r:svirt_lxc_net_t:s0:c596 c698 tcontext=system_u:object_r:file_t:s0 tclass=file ---- type=SYSCALL msg=audit(05/14/2014 10:26:30.241:1175) : arch=x86_64 syscall=open success=no exit=-13(Permission denied) a0=0x7fffe3de5d00 a1=O_RDONLY a2=0x0 a3=0xffffffff items=0 ppid=0 pid=6246 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts3 ses=unset comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c596 c698 key=(null) type=AVC msg=audit(05/14/2014 10:26:30.241:1175) : avc: denied { read } for pid=6246 comm=bash name=libtinfo.so.5 dev="sdb1" ino=4655 scontext=system_u:system_r:svirt_lxc_net_t:s0:c596 c698 tcontext=system_u:object_r:file_t:s0 tclass=lnk_file Please note that i have /dev/sdb1 mounted on /containers which is a symlinked to /var/lib/docker Thanks
(In reply to Prudhvi Surapaneni from comment #14) > Please note that i have /dev/sdb1 mounted on /containers which is a > symlinked to /var/lib/docker try: semanage fcontext -a -e /var/lib/docker /containers restorecon -R /containers
I tried the above. But i still see AVC denials. However, i remove the symlink and the docker package. Re-installed docker-io and tried it with vanilla /var/lib/docker and still run into similar problems. here are the interesting bits. ---- type=SYSCALL msg=audit(05/14/2014 11:18:50.593:1397) : arch=x86_64 syscall=mmap success=no exit=-13(Permission denied) a0=0x0 a1=0x221000 a2=PROT_READ|PROT_EXEC a3=MAP_PRIVATE|MAP_DENYWRITE items=0 ppid=0 pid=22550 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts5 ses=unset comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c171 c898 key=(null) type=AVC msg=audit(05/14/2014 11:18:50.593:1397) : avc: denied { execute } for pid=22550 comm=bash path=/lib64/libtinfo.so.5.7 dev="sdb1" ino=4656 scontext=system_u:system_r:svirt_lxc_net_t:s0:c171 c898 tcontext=system_u:object_r:docker_var_lib_t:s0 tclass=file
Why not mount it directly on /var/lib/containers? What backend are you using?
Hi Daniel, I tried both separate mount and using the default /var/lib/docker. The results are the same. I'm not using any specialized backends. Just the defaults in the systemd unit file. The reason i was using a custom mount is i wanted to have all my containers use a separate disk. But, like i said before. I tried the default directories too. I still get the same problem.
To add more details. I tried to do this on a fresh installation of fedora 20 with docker-io-0.11.1-3.fc20 and i am able to replicate this problem [prudhvi@aldebaran ~]$ cat /etc/redhat-release Fedora release 20 (Heisenbug) [prudhvi@aldebaran ~]$ rpm -qa | grep docker docker-io-0.11.1-3.fc20.x86_64 [prudhvi@aldebaran ~]$ docker run -i -t centos:latest /bin/bash /bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory
Could you paste ausearch -m avc -ts recent -i What backend are you using devmapper? What file system is /var/lib/docker running on?
Hi Daniel, Im using the btrfs filesystem for / which also includes /var/lib/docker on the same fs. In my docker unit file i don't see any special backend being used. So, i am not sure what the default is. Here is the output of ausearch -m avc -ts recent -i type=SYSCALL msg=audit(05/15/2014 12:13:12.467:2038) : arch=x86_64 syscall=open success=no exit=-13(Permission denied) a0=0x7fe63f743201 a1=O_RDONLY a2=0x1 a3=0x0 items=0 ppid=0 pid=19684 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts5 ses=unset comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c239 c640 key=(null) type=AVC msg=audit(05/15/2014 12:13:12.467:2038) : avc: denied { read } for pid=19684 comm=bash name=ld.so.cache dev="sda3" ino=4163 scontext=system_u:system_r:svirt_lxc_net_t:s0:c239 c640 tcontext=system_u:object_r:file_t:s0 tclass=file ---- type=SYSCALL msg=audit(05/15/2014 12:13:12.467:2039) : arch=x86_64 syscall=open success=no exit=-13(Permission denied) a0=0x7fff97aef9c0 a1=O_RDONLY a2=0x0 a3=0xffffffff items=0 ppid=0 pid=19684 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts5 ses=unset comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c239 c640 key=(null) type=AVC msg=audit(05/15/2014 12:13:12.467:2039) : avc: denied { read } for pid=19684 comm=bash name=libtinfo.so.5 dev="sda3" ino=4655 scontext=system_u:system_r:svirt_lxc_net_t:s0:c239 c640 tcontext=system_u:object_r:file_t:s0 tclass=lnk_file
Ok that explains it, if you could change the /var/lib/docker to be ext4 it should work, or remove the --selinux-enabled flag from the docker unit file. We do not have good support for btrfs yet.
docker-io-0.11.1-7.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/docker-io-0.11.1-7.fc20
docker-io-0.11.1-7.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.
I am able to start containers that were running before upgrade/reboot, but running any new ones fails. After a reboot, with any new containers: $ cat /etc/redhat-release Fedora release 20 (Heisenbug) $ rpm -qa | grep docker python-docker-py-0.2.3-8.fc20.x86_64 fedora-dockerfiles-0-0.5.git122ef5d.fc20.noarch docker-io-0.11.1-7.fc20.x86_64 $ mount | grep docker /dev/sda3 on /var/lib/docker type btrfs (rw,relatime,seclabel,space_cache) $ docker run -i -t fedora:latest /bin/bash /bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: Permission denied $ docker run -i -t centos:latest /bin/bash /bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory $ docker run -i -t ubuntu:latest /bin/bash /bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory $ sudo grep docker /var/log/yum.log* /var/log/yum.log-20140611:Feb 18 17:09:32 Installed: docker-io-0.7.6-4.fc20.x86_64 /var/log/yum.log-20140611:Feb 24 13:40:04 Updated: docker-io-0.8.0-3.fc20.x86_64 /var/log/yum.log-20140611:Mar 10 14:00:24 Updated: docker-io-0.8.1-1.fc20.x86_64 /var/log/yum.log-20140611:Mar 31 13:03:26 Updated: docker-io-0.9.0-3.fc20.x86_64 /var/log/yum.log-20140611:Apr 17 11:52:07 Updated: docker-io-0.9.1-1.fc20.x86_64 /var/log/yum.log-20140611:May 12 15:27:54 Updated: docker-io-0.11.1-1.fc20.x86_64 /var/log/yum.log-20140611:Jun 02 10:40:46 Installed: python-docker-py-0.2.3-8.fc20.x86_64 /var/log/yum.log-20140611:Jun 02 10:43:25 Installed: fedora-dockerfiles-0-0.5.git122ef5d.fc20.noarch /var/log/yum.log-20140611:Jun 10 07:25:17 Updated: docker-io-0.11.1-7.fc20.x86_64 $
I'm also running into the same issue. From the above comments, i suspect that you are trying to run docker on a btrfs fs?. It looks like SELinux + btrfs + docker isn't ready yet.
Also, i just tried docker-io-1.0.0 from updates-testing. It seems to not have the problem you are describing.
I'm using Fedora 20 with standard repo. I have these error using docker.io 1.0.0 1.fc20 sudo docker run -t -i debian:6.0.9 /bin/bash 2014/06/17 04:24:43 finalize namespace drop capabilities read /proc/1/status: bad file descriptor [michael@mininet-vm screenFetch]$ sudo docker run -t -i debian:6.0.9 /bin/bash /bin/bash: error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory I just done touch /.autorelabel too.
I have there selinux error too : type=SYSCALL msg=audit(2014-06-17 00:24:50.243:2194) : arch=x86_64 syscall=open success=no exit=-13(Permission non accordée) a0=0x7fcfd99afcd4 a1=O_RDONLY a2=0x1 a3=0xffffffffffffffb0 items=0 ppid=0 pid=13070 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=pts4 comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c455 c570 key=(null) type=AVC msg=audit(2014-06-17 00:24:50.243:2194) : avc: denied { read } for pid=13070 comm=bash name=ld.so.cache dev="sda4" ino=446 scontext=system_u:system_r:svirt_lxc_net_t:s0:c455 c570 tcontext=system_u:object_r:file_t:s0 tclass=file ---- type=SYSCALL msg=audit(2014-06-17 00:24:50.243:2195) : arch=x86_64 syscall=open success=no exit=-13(Permission non accordée) a0=0x7ffff71be890 a1=O_RDONLY a2=0x0 a3=0xffffffff items=0 ppid=0 pid=13070 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=pts4 comm=bash exe=/bin/bash subj=system_u:system_r:svirt_lxc_net_t:s0:c455 c570 key=(null) type=AVC msg=audit(2014-06-17 00:24:50.243:2195) : avc: denied { read } for pid=13070 comm=bash name=libncurses.so.5 dev="sda4" ino=630 scontext=system_u:system_r:svirt_lxc_net_t:s0:c455 c570 tcontext=system_u:object_r:file_t:s0 tclass=lnk_file
If I create a module with the selinux log in my last comment (Comment 31), using | audit2allow -M docker. And I load this module, I have this new Selinux audit error after a new try with this cmd line «sudo docker run -t -i debian:6.0.9 /bin/bash» : type=SYSCALL msg=audit(2014-06-17 00:32:03.683:2289) : arch=x86_64 syscall=connect success=no exit=-13(Permission non accordée) a0=0x3 a1=0xc21005ac90 a2=0x17 a3=0x0 items=0 ppid=13456 pid=13458 auid=michael uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=1 tty=pts1 comm=docker exe=/usr/bin/docker subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(2014-06-17 00:32:03.683:2289) : avc: denied { connectto } for pid=13458 comm=docker path=/run/docker.sock scontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=unix_stream_socket The new output in stderr : 2014/06/17 00:35:32 Post http:///var/run/docker.sock/v1.12/containers/create: dial unix /var/run/docker.sock: permission denied
If I try to create a new module again using my new selinux error from comment 32, it say : sudo ausearch -m AVC,USER_AVC -ts recent | audit2allow -M docker libsepol.context_from_record: type docker_t is not defined libsepol.context_from_record: could not create context structure libsepol.context_from_string: could not create context structure libsepol.sepol_context_to_sid: could not convert system_u:system_r:docker_t:s0 to sid (3x repetitions) ******************** IMPORTANT ***********************
Michael are you running docker on a BTRFS system? Miroslav, we should not run docker with selinux-enabled on rhel6. Since we do not have all of the policy in place, I believe.
I confirm the bug exists on Fedora 20, even after doing a yum upgrade so that docker is at version 1.0, and it very likely comes from SE-Linux. The way I was able to repeat the issue: $ getenforce Enforcing $ docker -v Docker version 1.0.0, build 63fe64c/1.0.0 $ docker run -t -i -p 80:80 -p 20022:22 oskarhane/docker-wordpress-nginx-ssh 140620 15:36:13 mysqld_safe Logging to syslog. 140620 15:36:13 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql mysql root password: Iecalood4uoh wordpress password: eechoo7Uj4bo ssh password: ahMaisohSh8i usermod: failure while writing changes to /etc/passwd ^ |--------------------- this is the issue % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 58606 100 58606 0 0 51933 0 0:00:01 0:00:01 --:--:-- 53278 Archive: nginx-helper.1.8.1.zip [...] If I change to non-enforcing # setenforce 0 $ getenforce Permissive Then the "docker run" line above works with no issues. Oddly, on CentOS 6.5, I don't need to change the enforcement policy. CentOS6.5 $ getenforce Enforcing CentOS6.5 $ docker -v Docker version 1.0.0, build 63fe64c/1.0.0 CentOS6.5 $ docker run -t -i -p 80:80 -p 20022:22 oskarhane/docker-wordpress-nginx-ssh [no issues]
This has nothing to do with Fedora, it is a Centos and RHEL6 issue. I have updated the libselinux we ship with RHEL6 to handle this properly and centos is supposed to have a version with the updated libselinux also.
Thanks Daniel for the message. The reason I posted it here is throughout the comments and even on the bug description, it says it's for Fedora, in particular Fedora 20. The problem is definitively on Fedora and the way the selinux policy is written, as the code I posted earlier has demonstrated: Enforce selinux -> docker containers can't update *their* /etc/passwd file, which is a problem as one core expectation of docker is that it runs the same regardless of platform. If you want I can create a register a new bug, but here seem to be the proper place.
I am so confused by this bug at this point. parts cover rhel6/centos6. Parts Fedora. The original bug was caused by programs using libselinux attempting to write to /proc which was mounted read/only. If you are getting an error in enforcing mode, but not in permissive mode, then you are seeing a different issue. What AVC were you getting? I believe your bug has nothing to do with the rest of this bugzilla.
Then I will be happy to create a new bug report. In the meantime, to answer your question, in one terminal I ran: # tail -f /var/log/audit/audit.log and in another, I launched: $ docker run -t -i -p 80:80 -p 20021:22 oskarhane/docker-wordpress-nginx-ssh Beside a bunch of NETFILTER entries related to xtables-multi, there was nothing at all showing in the audit log, even around the moments when the update to /etc/passwd failed within the container. However, in the console were the container runs, I saw this quite a bit later: sed: warning: failed to set default file creation context to system_u:object_r:svirt_sandbox_file_t:s0:c369,c792: Permission denied (This message appears even if I run docker as root) I don't know if it's related to the failure to update /etc/passwd or not. I will create a new bug report and will provide more details there.
I created the new bug report. Available at https://bugzilla.redhat.com/show_bug.cgi?id=1112748