Bug 1461571

Same problem exists in Fedora 25 Atomic Host:

# docker ps -a
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS                     PORTS               NAMES
57a50a35fdfc        apache_httpd        "httpd-foreground"   3 minutes ago       Exited (1) 2 minutes ago                       apache_httpd

# docker logs 57a50a35fdfc
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00091: httpd: could not open error log file /proc/self/fd/2.
AH00015: Unable to open logs
[root@micah-f25ah-vm0613a ~]# journalctl -b | grep -i avc
Jun 14 19:54:25 micah-f25ah-vm0613a.localdomain audit[6381]: AVC avc:  denied  { open } for  pid=6381 comm="httpd" path="pipe:[57170]" dev="pipefs" ino=57170 scontext=system_u:system_r:container_t:s0:c64,c482 tcontext=system_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0

# rpm -q docker container-selinux
docker-1.12.6-6.gitae7d637.fc25.x86_64
container-selinux-2.14-1.fc25.noarch

# atomic host status
State: idle
Deployments:
● fedora-atomic:fedora-atomic/25/x86_64/docker-host
             Version: 25.137 (2017-06-04 23:31:40)
              Commit: 0ed61d7441eddf96e6a98de4f10f4675268c7888b6d2b8a405b8c21fe6c92d23



+++ This bug was initially created as a clone of Bug #1460341 +++

One of our automated tests that builds 'httpd' images using various base images found a problem when trying to run an image built from 'docker.io/httpd'.  

This was testing against RHELAH 7.3.6 with 'docker-1.12.6-31.git3a6eaeb.el7.x86_64'


Using the following Dockerfile:

https://github.com/projectatomic/atomic-host-tests/blob/master/tests/docker-build-httpd/files/apache_httpd/Dockerfile

The image was built with:

# docker build --pull --rm -t apache_httpd -f Dockerfile .

Then run with:

# docker run -d -p 80:80 --name apache_httpd apache_httpd

But the container exits immediately.  The 'docker logs' show:

# docker logs apache_httpd
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00091: httpd: could not open error log file /proc/self/fd/2.
AH00015: Unable to open logs



This same container runs fine on RHELAH 7.3.5 using 'docker-1.12.6-28.git1398f24.el7.x86_64'


See the log below showing the results with both versions:


# atomic host status
State: idle
Deployments:
● 7.3_latest:rhel-atomic-host/7/x86_64/standard
             Version: 7.3.5 (2017-05-22 22:00:44)
              Commit: 0ccf9138962e5c2c3794969a228e751d13bb780f5b0a1f15f4a9649df06ba80a

# rpm -q docker
docker-1.12.6-28.git1398f24.el7.x86_64

# docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
apache_httpd                              latest              a251680b211f        17 minutes ago      177.5 MB
docker.io/httpd                           latest              e0645af13ada        4 weeks ago         177.5 MB

# docker run -d -p 80:80 --name apache_httpd apache_httpd
37c293f787b8360e774047f118f22c5f65defdd52003816abf9116d569e59a57

# docker ps -a
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS              PORTS                NAMES
37c293f787b8        apache_httpd        "httpd-foreground"   7 seconds ago       Up 5 seconds        0.0.0.0:80->80/tcp   apache_httpd

# curl http://localhost:80
SUCCESS apache_httpd


....upgrade/reboot....


# atomic host status
State: idle
Deployments:
● 7.3_latest:rhel-atomic-host/7/x86_64/standard
             Version: 7.3.6 (2017-06-07 21:38:15)
              Commit: a88c603af2a5c6b052b32c12b92c5578f71a8088e077781d17330275c63d03bd

  7.3_latest:rhel-atomic-host/7/x86_64/standard
             Version: 7.3.5 (2017-05-22 22:00:44)
              Commit: 0ccf9138962e5c2c3794969a228e751d13bb780f5b0a1f15f4a9649df06ba80a

# rpm -q docker
docker-1.12.6-31.git3a6eaeb.el7.x86_64

# docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
apache_httpd                              latest              a251680b211f        21 minutes ago      177.5 MB
docker.io/httpd                           latest              e0645af13ada        4 weeks ago         177.5 MB

# docker run -d -p 80:80 --name apache_httpd apache_httpd
1e12902555a72e0839ae68ac07ee9b1ee2e853d2704abe1e3fafaf059b0dd16a

# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

# docker ps -a
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS                     PORTS               NAMES
1e12902555a7        apache_httpd        "httpd-foreground"   8 seconds ago       Exited (1) 7 seconds ago                       apache_httpd

# docker logs apache_httpd 
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00091: httpd: could not open error log file /proc/self/fd/2.
AH00015: Unable to open logs

--- Additional comment from Red Hat Bugzilla Rules Engine on 2017-06-09 14:23:41 EDT ---

Since this bug report was entered in Red Hat Bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from Red Hat Bugzilla Rules Engine on 2017-06-09 14:23:41 EDT ---

This bug report has Keywords: Regression or TestBlocker.

Since no regressions or test blockers are allowed between releases, it is also being [proposed|marked] as a blocker for this release.

Please resolve ASAP.

--- Additional comment from Antonio Murdaca on 2017-06-09 14:28:05 EDT ---

Can you disable selinux and re-try?

--- Additional comment from Micah Abbott on 2017-06-09 15:22:19 EDT ---

Yeah, disabling SELinux works around it.

--- Additional comment from Micah Abbott on 2017-06-09 15:27:42 EDT ---

Similar problems with a 'httpd' container based on 'docker.io/nginx'

Build with this Dockerfile:

https://github.com/projectatomic/atomic-host-tests/blob/master/tests/docker-build-httpd/files/nginx_httpd/Dockerfile

The logs from that container:

# docker logs nginx_httpd 
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2017/06/09 19:14:34 [emerg] 1#1: open() "/var/log/nginx/error.log" failed (13: Permission denied)


Disabling SELinux in this case also worked around the error.

--- Additional comment from Daniel Walsh on 2017-06-10 08:20:51 EDT ---

I believe this is an issue in the latest container-selinux package.


Whenever you report an error with SELinux always attach the AVC messages.

ausearch -m avc -ts recent.



Also give me

ps -eZ | grep docker


I suspect that docker is running with something like
container_runtime_t:s0-s0:c10123

It used to run as
container_runtime_t:s0


I believe this is related to a problem we have seen in Fedora about docker leaking fds into the container to be used for stdin, stdout and stderr. I changed the way that the container runtimes run which is causing a new SELinux issue.

--- Additional comment from Micah Abbott on 2017-06-12 09:42:18 EDT ---

There's no `ausearch` on RHELAH, so the best I can give you is some grepped journal entries.


# journalctl -b | grep -e avc -e audit -e denied
Jun 12 13:39:56 micah-rhelah-vm0609a.localdomain kernel: type=1401 audit(1497274796.569:12): op=security_compute_av reason=bounds scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=unconfined_u:system_r:svirt_lxc_net_t:s0-s0:c0.c1023 tclass=process perms=transition,sigchld,sigstop,signull,signal,getattr
Jun 12 13:39:56 micah-rhelah-vm0609a.localdomain kernel: type=1401 audit(1497274796.603:13): op=security_compute_av reason=bounds scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:svirt_lxc_net_t:s0-s0:c0.c1023 tclass=process perms=transition,sigchld,sigstop,signull,signal,getattr
Jun 12 13:40:29 micah-rhelah-vm0609a.localdomain kernel: type=1400 audit(1497274829.974:14): avc:  denied  { open } for  pid=14526 comm="httpd" path="pipe:[78235]" dev="pipefs" ino=78235 scontext=system_u:system_r:svirt_lxc_net_t:s0:c533,c834 tcontext=system_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file
Jun 12 13:40:29 micah-rhelah-vm0609a.localdomain dockerd-current[5726]: (13)Permission denied: AH00091: httpd: could not open error log file /proc/self/fd/2.



# ps -eZ | grep docker
system_u:system_r:container_runtime_t:s0-s0:c0.c1023 5726 ? 00:00:45 dockerd-current
system_u:system_r:container_runtime_t:s0-s0:c0.c1023 5765 ? 00:00:03 docker-containe

--- Additional comment from Daniel Walsh on 2017-06-14 14:41:32 EDT ---

grep avc /var/log/audit/audit.log

--- Additional comment from Micah Abbott on 2017-06-14 15:37:49 EDT ---

(In reply to Daniel Walsh from comment #8)
> grep avc /var/log/audit/audit.log

That doesn't exist on RHELAH systems.

--- Additional comment from Daniel Walsh on 2017-06-14 15:39:17 EDT ---

Yes well you gave me the AVC's anyways.

journalctl -b | grep -i avc

:^)

--- Additional comment from Micah Abbott on 2017-06-14 15:39:31 EDT ---

(In reply to Micah Abbott from comment #9)
> (In reply to Daniel Walsh from comment #8)
> > grep avc /var/log/audit/audit.log
> 
> That doesn't exist on RHELAH systems.

Well, I won't make a blanket statement.  But it doesn't exist on the system I reproduced this on.