RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1412728 - Bind mount on /var/log seems to be over shadowed
Summary: Bind mount on /var/log seems to be over shadowed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: oci-systemd-hook
Version: 7.3
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: rc
: ---
Assignee: Mrunal Patel
QA Contact: Martin Jenner
URL:
Whiteboard:
Depends On:
Blocks: 1412281
TreeView+ depends on / blocked
 
Reported: 2017-01-12 16:10 UTC by Mohamed Ashiq
Modified: 2023-09-14 03:37 UTC (History)
11 users (show)

Fixed In Version: oci-systemd-hook-1:0.1.4-9.git671c428.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-17 20:46:33 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1419040 0 unspecified CLOSED The change to /var/log mounting breaks the running of services that require a folder in /var/log created at docker build... 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHBA-2017:0117 0 normal SHIPPED_LIVE oci-systemd-hook bug fix update 2017-01-18 01:40:01 UTC

Internal Links: 1419040

Description Mohamed Ashiq 2017-01-12 16:10:50 UTC 
Description of problem:
We have a systemd container on which we have to have a bind mount on the '/var/log/<something>'. This worked fine before this release. This bind mount is strict requirement for us because of which we are hitting issue in our setups. 

Version-Release number of selected component (if applicable):
# rpm -qa | grep docker
cockpit-docker-126-1.el7.x86_64
docker-common-1.12.5-14.el7.x86_64
docker-1.12.5-14.el7.x86_64
docker-client-1.12.5-14.el7.x86_64
docker-rhel-push-plugin-1.12.5-14.el7.x86_64

# rpm -qa | grep systemd
systemd-219-30.el7_3.7.x86_64
systemd-sysv-219-30.el7_3.7.x86_64
oci-systemd-hook-0.1.4-8.git45455fe.el7.x86_64
systemd-libs-219-30.el7_3.7.x86_64

# rpm -qa | grep systemd
systemd-libs-219-30.el7_3.6.x86_64
systemd-219-30.el7_3.6.x86_64

How reproducible:
Always

Steps to Reproduce:
# docker run -d -v /var/log/something:/var/log/something:z rhel /usr/sbin/init
b8a202e69e364f0cee13a7127bc97a7ad9f55e5827e5462b5bc4ed09b1f12f74
# docker exec -it b8a202e69e364f0cee13a7127bc97a7ad9f55e5827e5462b5bc4ed09b1f12f74 bash
/]# df -h
Filesystem                                                                                         Size  Used Avail Use% Mounted on
/dev/mapper/docker-8:17-67158286-1c93270178ee4a8845a0b69c0729a516ffd9dc712d0bf7c2cb160021f02c94b0   10G  247M  9.8G   3% /
tmpfs                                                                                               24G     0   24G   0% /dev
tmpfs                                                                                               24G     0   24G   0% /sys/fs/cgroup
/dev/sdb1                                                                                           40G  1.2G   39G   3% /etc/hosts
shm                                                                                                 64M     0   64M   0% /dev/shm
tmpfs                                                                                               64M  236K   64M   1% /run
tmpfs                                                                                              4.0E     0  4.0E   0% /tmp
tmpfs                                                                                              4.0E  8.0K  4.0E   1% /var/log
/]# ls /var/log/
btmp     journal/ wtmp

/]# mount       
/dev/mapper/docker-8:17-67158286-1c93270178ee4a8845a0b69c0729a516ffd9dc712d0bf7c2cb160021f02c94b0 on / type xfs (rw,relatime,seclabel,nouuid,attr2,inode64,sunit=1024,swidth=1024,noquota)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev type tmpfs (rw,nosuid,seclabel,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=666)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/net_prio,net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpuacct,cpu type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
/dev/sdb1 on /etc/resolv.conf type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/sdb1 on /etc/hostname type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/sdb1 on /etc/hosts type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,size=65536k)
/dev/sdb1 on /run/secrets type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/sdb1 on /var/log/something type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,size=65536k,mode=755)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=4503599627370496k)
tmpfs on /var/log type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=4503599627370496k)
/dev/sdb1 on /var/log/journal/552de008be2c0a1364cbacfc32ef526f type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/pids type cgroup (ro,relatime,pids)
cgroup on /sys/fs/cgroup/blkio type cgroup (ro,relatime,blkio)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (ro,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/cpuset type cgroup (ro,relatime,cpuset)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (ro,relatime,hugetlb)
cgroup on /sys/fs/cgroup/memory type cgroup (ro,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (ro,relatime,perf_event)
cgroup on /sys/fs/cgroup/devices type cgroup (ro,relatime,devices)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (ro,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/freezer type cgroup (ro,relatime,freezer)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=28,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
configfs on /sys/kernel/config type configfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)


Please let me know if you need more information.
Comment 1 Daniel Walsh 2017-01-12 16:12:55 UTC 
What version of oci-systemd-hook do you have installed

rpm -q oci-systemd-hook
Comment 2 Mohamed Ashiq 2017-01-12 16:16:25 UTC 
# rpm -qa | grep oci
oci-register-machine-0-1.11.gitdd0daef.el7.x86_64
oci-systemd-hook-0.1.4-8.git45455fe.el7.x86_64


In the description First systemd version is on the host and second one is on container.
Comment 3 Daniel Walsh 2017-01-12 16:36:16 UTC 
Mrunal, we are overmounting the directories on top of existing volume mounts and on /run/secrets.

First question?  Do you remember why we are mounting a tmpfs on /var/log?

When we mount over /run we should probably tar up the contents from the directory and put them into the tmpfs of at least mv any mount points off of /run into the newly mounted tmpfs.

# docker run -ti -v /var/log/dan:/var/log/dan:z fedora mount | grep /var/log
/dev/sda2 on /var/log/dan type ext4 (rw,relatime,seclabel,data=ordered)
tmpfs on /var/log type tmpfs (rw,nosuid,nodev,relatime,context="system_u:object_r:container_file_t:s0:c118,c249")
/dev/sda2 on /var/log/journal/184b22d455aafb6c9d56ce1f79cf3d20 type ext4 (rw,relatime,seclabel,data=ordered)

I can work on a fix, but I want  your opinion.
Comment 4 Mrunal Patel 2017-01-12 17:53:21 UTC 
Dan, the /var/log tmpfs was for journald logs. I think we can do the same dance of MS_MOVE that we do for /run mounts. We move mounts temporarily using MS_MOVE and then put them back in place for anything over /run. We can do the same for mounts specified by the user over /var/log.
Comment 6 Guohua Ouyang 2017-01-13 02:54:10 UTC 
tested oci-systemd-hook-0.1.4-9.git671c428.el7.x86_64.rpm, the fix works.

1. Reproduced the bug 
# mkdir /var/log/test
# docker run -d -v /var/log/test:/var/log/test:z rhel /usr/sbin/init
44f4d3a237c94b6f0b870f972f9adf638407827103283a19fccb55433495fd7b
# docker exec -it 44f4d3a237 bash
# ls /var/log
btmp  journal  wtmp
# docker stop 44f4
44f4
# docker rm 44f4
44f4

2. 
# rpm -Uvh oci-systemd-hook-0.1.4-9.git671c428.el7.x86_64.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:oci-systemd-hook-1:0.1.4-9.git671################################# [ 50%]
Cleaning up / removing...
   2:oci-systemd-hook-1:0.1.4-8.git454################################# [100%]

3. 
# docker run -d -v /var/log/test:/var/log/test:z rhel /usr/sbin/init
2c46308f2981e72fd19378da393e8719faaee85cdeec597377bb73b13c5b8133
# docker exec -it 2c46308 bash
# ls /var/log
btmp  journal  test  wtmp

The /var/log/test dir is there.
Comment 8 Humble Chirammal 2017-01-13 06:46:09 UTC 
Thanks Dan for your quick help on this!! We are also validating the fix from our side and will update this bug accordingly.

Brew link : https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=12348885
Comment 9 Daniel Walsh 2017-01-13 12:31:55 UTC 
Awesome.
Comment 10 Humble Chirammal 2017-01-13 14:59:02 UTC 
Good news!! Gluster Container deployment is working as expected and we confirm that, the reported issue at our end is fixed with above mentioned build. Once again thanks a lot for quick help on this, much appreciated.
Comment 13 errata-xmlrpc 2017-01-17 20:46:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0117.html
Comment 14 Red Hat Bugzilla 2023-09-14 03:37:20 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days
Note You need to log in before you can comment on or make changes to this bug.