|Summary:||/run mounted as tmpfs in container and causes changes in the directory to not be persistent|
|Product:||Red Hat Enterprise Linux 7||Reporter:||Chen Chang <cchang>|
|Component:||docker||Assignee:||Daniel Walsh <dwalsh>|
|Status:||CLOSED NOTABUG||QA Contact:||Virtualization Bugs <virt-bugs>|
|Version:||7.1||CC:||ablum, cchang, cpelland, ghacker, sbonnevi, sct|
|Fixed In Version:||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2015-02-20 19:02:53 UTC||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Cloudforms Team:||---||Target Upstream Version:|
Description Chen Chang 2015-02-20 06:24:51 UTC
Description of problem: Directory /run/httpd created in a container by the installation of the httpd rpm is lost once container is exited. When container is started again, httpd fails to run because it can't create /run/httpd/authdigest.shm.8 since the /run/httpd directory no longer exists. Version-Release number of selected component (if applicable): -bash-4.2# cat /etc/redhat-release Red Hat Atomic Host release 7.1 -bash-4.2# rpm -q docker docker-1.4.1-37.el7.x86_64 How reproducible: Can be produced every time. Steps to Reproduce: On Atomic host, run: 1. docker pull rhel7 2. docker run -i -t rhel7 /bin/bash - runs image ID bef54b8f8a2f In container, run: 1. yum install -y httpd 2. rpm -V httpd - verification passes with no output 3. ls -lad /run/httpd - confirms directory was created by httpd rpm install 4. exit On Atomic host, run: 1. docker ps -a - displays ID of exited container 2. docker start CONTAINERID 3. docker attach CONTAINERID In container, run: 1. rpm -V httpd - reports: missing /run/httpd missing /run/httpd/htcacheclean 2. ls -lad /run/httpd - fails and reports: ls: cannot access /run/httpd: No such file or directory 3. /usr/sbin/httpd -DFOREGROUND; echo $? - httpd exits with status 1 - /var/log/httpd/error_log reports error "No such file or directory: AH01762: Failed to create shared memory segment on file /run/httpd/authdigest_shm.8" Actual results: /run contents are lost after container is exited. When container is started again, absence of the /run/httpd directory causes httpd to fail to start. Expected results: /run contents should persist after container is exited. When container is started again, directory /run/httpd should still exist and httpd should start without issues. Additional info: Same steps did not result in failure when run on prior release: -bash-4.2# cat /etc/redhat-release Red Hat Atomic Host Beta release 7.0 Beta -bash-4.2# rpm -q docker docker-1.3.2-4.el7.x86_64 In the above release, when running the same rhel7 image ID bef54b8f8a2f, the running container does not have /run mounted separately as tmpfs. In the problematic release, the running container has /run mounted separately as tmpfs. Looks like it might be related to issue mentioned here: https://github.com/docker/docker/pull/5975
Comment 1 Steve Bonneville 2015-02-20 14:53:44 UTC
Since /run is mounted as tmpfs in regular RHEL 7, that's really weird that the httpd package owns files in there and is installing files in there. I'm noticing that rpm -ql reports httpd owns /run/httpd and /run/httpd/htcacheclean. I think you're right, there must be startup scripts that make sure those files are recreated properly in a traditional environment that aren't run in a container.
Comment 2 Andrew Blum 2015-02-20 16:26:17 UTC
I agree this is an issue we need some help with. Running the httpd container like the following isn't a good workaround [although this works]: -bash-4.2# docker run -d -p 8080:80 rht-training:rhel7_httpd /bin/bash -c "mkdir /run/httpd; httpd -D FOREGROUND" Also to note that containers built on older atomic versions [pre docker-1.4], don't have any issues running on the latest atomic build [2015-02-19 20:26:26 5799825b36]...but they also don't have tmpfs mounted /run either: -bash-4.2# docker exec -it 77f0c3c187f0 df -h /run Filesystem Size Used Avail Use% Mounted on /dev/mapper/docker-253:1-16842786-77f0c3c187f0d969807bd235f58749150ddb35d91186863e3dc8dd137c01021c 10G 303M 9.7G 3% /
Comment 4 Stephen Tweedie 2015-02-20 16:56:26 UTC
/run is non-persistent on RHEL and Fedora too, this isn't container-specific. In httpd's case, these files are created on boot via /usr/lib/tmpfiles.d/httpd.conf which causes the appropriate /run/httpd and /run/httpd/htcacheclean directories to be created at each boot. A container start script for httpd will need to replicate this action.
Comment 5 Andrew Blum 2015-02-20 17:21:38 UTC
Ok. based on Stephen's comments and a quick chat on irc...I think this will need to be the httpd "hello world": FROM https://github.com/fedora-cloud/Fedora-Dockerfiles/tree/master/apache -bash-4.2# docker run -it registry.access.redhat.com/rhel7 /bin/bash bash-4.2# yum install httpd -y bash-4.2# echo kc5 > /var/www/html/index.html Create startup script: ---------------------- bash-4.2# vi /usr/bin/my_httpd_startup.bash #!/bin/bash # Make sure we're not confused by old, incompletely-shutdown httpd # context after restarting the container. httpd won't start correctly # if it thinks it is already running. rm -rf /run/httpd/* # need to create directories per /usr/lib/tmpfiles.d/httpd.conf ## d /run/httpd 710 root apache ## d /run/httpd/htcacheclean 700 apache apache mkdir -p /run/httpd/htcacheclean chmod 710 /run/httpd chmod 700 /run/httpd/htcacheclean chown root:apache /run/httpd chown apache:apache /run/httpd/htcacheclean exec /usr/sbin/apachectl -D FOREGROUND -------------------- bash-4.2# chmod 755 /usr/bin/my_httpd_startup.bash -bash-4.2# docker run -p 8080:80 -d mytest /usr/bin/my_httpd_startup.bash 8518f03a28782745c4148667b1a936e7a052284231bde4c16d82b4ce3a0d0f72 -bash-4.2# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8518f03a2878 mytest:latest "/usr/bin/my_httpd_s 3 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp goofy_tesla -bash-4.2# curl localhost:8080 kc5 -bash-4.2# docker stop 8518f03a2878 8518f03a2878 NOTE: docker stop will now take 10 sec, the default time before docker sends the sigkill since our wrapper script can't handle the sigterm.
Comment 6 Daniel Walsh 2015-02-20 19:02:53 UTC
You could also work with systemd in the container. If you build an image using a docker file, this should work correctly also. The /run patch will copy the content under /run on the image to /run on the image.