Red Hat Bugzilla – Bug 1491566
[3.6] Checks during installation fails, due to /var being split out on different filesystems.
Last modified: 2018-01-15 12:51:15 EST
Description of problem: customer have setup the filesystems to have /var /var/log /var/lib/origin /var/lib/docker in order to split out where to put data. During the installation the checks for storage will fail on /var as /var is only 10G but /var/lib/origin are 80GB and so is /var/lib/docker, we can abvoid this check with openshift_disable_check. however in future settings we should check on /var/lib/docker and /var/lib/origin as that is most likely the places to get filled up. How reproducible: All the time Steps to Reproduce: 1. Split out filesystems on /var with /var being below 40G 2. Start install 3. Actual results: Installation fails with Storage requirement not met even though we have enough storage in underlying mounts. Expected results: Installation succeed Additional info: Please attach logs from ansible-playbook with the -vvv flag
I agree it would make sense to look at subtree mounts for data that OCP uses and adjust the requirement for /var accordingly. At the moment the documented requirements simply specify /var so that's what the check follows (https://docs.openshift.org/latest/install_config/install/prerequisites.html#system-requirements). It shouldn't be too hard to just include filesystems underneath /var in the total calculation for /var. I think that would make the most sense here. It would also be a good idea to add "sanity" thresholds in the check for specific subtrees; for instance if a /var/lib/docker volume fills up it doesn't really matter that there's plenty of room in /var, docker will still fail.
https://github.com/openshift/openshift-ansible/pull/5816
https://github.com/openshift/openshift-ansible/pull/5819 is to backport to 3.6
Commit pushed to master at https://github.com/openshift/openshift-ansible https://github.com/openshift/openshift-ansible/commit/4fb35642a47550021fed3e455bedc9b6fdf2b9c9 disk_availability check: include submount storage Fixes bug 1491566 https://bugzilla.redhat.com/show_bug.cgi?id=1491566 In order to determine how much storage is under a path, include any mounts that are below it in addition to the path itself.
Change status since no build in errata system.
Waiting for build existing in Errata System to verify this.
Verified with version openshift-ansible-3.6.173.0.83-1.git.0.84c5eff.el7, create split filesystems under /var with more than 40G size, health check passed. # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/rhel-root 10G 1.4G 8.7G 14% / devtmpfs 1.8G 0 1.8G 0% /dev tmpfs 1.8G 0 1.8G 0% /dev/shm tmpfs 1.8G 17M 1.8G 1% /run tmpfs 1.8G 0 1.8G 0% /sys/fs/cgroup /dev/xvda2 497M 159M 339M 32% /boot /dev/mapper/rhel-docker 40G 33M 40G 1% /var/lib/docker tmpfs 361M 0 361M 0% /run/user/0
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3438
*** Bug 1506265 has been marked as a duplicate of this bug. ***