Bug 1629392
| Summary: | cannot install undercloud because / is too small | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Amit Ugol <augol> |
| Component: | openstack-tripleo-validations | Assignee: | Alex Schultz <aschultz> |
| Status: | CLOSED ERRATA | QA Contact: | Andrey Fiodorov <afiodoro> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 14.0 (Rocky) | CC: | ahrechan, aschultz, augol, hbrock, jjoyce, jschluet, jslagle, mburns, mlammon, slinaber, tvignaud, ukalifon, yobshans |
| Target Milestone: | z1 | Keywords: | Triaged, ZStream |
| Target Release: | 14.0 (Rocky) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-validations-9.3.1-0.20180831205306.el7ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-03-18 13:03:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Amit Ugol
2018-09-15 16:22:17 UTC
In 14, we're not using instack-undercloud any more. This might be a bug in the validations for the new containerized undercloud deployment. Please provide a sosreport. Reproduced: [stack@undercloud-0 ~]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 55G 32G 24G 58% / devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 8.6M 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1001 [stack@undercloud-0 ~]$ cat install-undercloud.log 2018-09-18 06:57:20.183 24892 INFO tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] Checking Hostname... 2018-09-18 06:57:20.183 24892 INFO tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] Checking for a FQDN hostname... 2018-09-18 06:57:20.250 24892 INFO tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] Static hostname detected as undercloud-0.redhat.local 2018-09-18 06:57:20.266 24892 INFO tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] Transient hostname detected as undercloud-0.redhat.local 2018-09-18 06:57:20.266 24892 INFO tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] Checking Memory... 2018-09-18 06:57:20.266 24892 INFO tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] Checking Disk space... 2018-09-18 06:57:24.595 24892 ERROR tripleoclient.v1.undercloud_preflight.UndercloudSetup [ ] An error occurred during configuration validation, please check your host configuration and try again. Error message: Task 'loop on existing volumes, check available space' failed: Host: undercloud Message: Minimum required for /: 25G - volume free space: 23.4 Failure! The validation failed for all hosts: * undercloud [stack@undercloud-0 tmp]$ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/vda1 57659264 31457956 26201308 55% / devtmpfs 8111684 0 8111684 0% /dev tmpfs 8133464 0 8133464 0% /dev/shm tmpfs 8133464 8808 8124656 1% /run tmpfs 8133464 0 8133464 0% /sys/fs/cgroup tmpfs 1626696 0 1626696 0% /run/user/0 tmpfs 1626696 0 1626696 0% /run/user/1001 [stack@undercloud-0 tmp]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 55G 31G 25G 55% / devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 8.7M 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1001 [stack@undercloud-0 ~]$ bash -xe undercloud_deploy.sh + openstack undercloud install [stack@undercloud-0 ~]$ bash -xe undercloud_deploy.sh cca^C [stack@undercloud-0 ~]$ cat undercloud_install.log An error occurred during configuration validation, please check your host configuration and try again. Error message: Task 'loop on existing volumes, check available space' failed: Host: undercloud Message: Minimum required for /: 25G - volume free space: 25.0 Failure! The validation failed for all hosts: * undercloud [stack@undercloud-0 ~]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 55G 30G 26G 55% / devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 8.6M 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1001 [stack@undercloud-0 ~]$ bash -xe undercloud_deploy.sh + openstack undercloud install [stack@undercloud-0 ~]$ cat undercloud_install.log An error occurred during configuration validation, please check your host configuration and try again. Error message: Task 'loop on existing volumes, check available space' failed: Host: undercloud Message: Minimum required for /: 25G - volume free space: 25.0 Failure! The validation failed for all hosts: * undercloud [stack@undercloud-0 tmp]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 55G 30G 26G 55% / devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 8.7M 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1001 [stack@undercloud-0 tmp]$ df -h^C [stack@undercloud-0 tmp]$ cd [stack@undercloud-0 ~]$ bash -xe undercloud_deploy.sh + openstack undercloud install [stack@undercloud-0 ~]$ cat undercloud_install.log An error occurred during configuration validation, please check your host configuration and try again. Error message: Task 'loop on existing volumes, check available space' failed: Host: undercloud Message: Minimum required for /: 25G - volume free space: 25.2 Failure! The validation failed for all hosts: * undercloud [stack@undercloud-0 ~]$ rpm -qa "*validation*" openstack-tripleo-validations-9.3.1-0.20180831205305.fbfd253.el7ost.noarch [stack@undercloud-0 ~]$ rpm -qa "*openstack*" openstack-tripleo-validations-9.3.1-0.20180831205305.fbfd253.el7ost.noarch openstack-heat-common-11.0.1-0.20180901130821.680a515.el7ost.noarch openstack-tripleo-image-elements-9.0.0-0.20180831210308.2dc678a.el7ost.noarch openstack-tripleo-common-9.3.1-0.20180831204016.bb0582a.el7ost.noarch openstack-heat-agents-1.7.1-0.20180829044839.24f9e9c.el7ost.noarch openstack-heat-monolith-11.0.1-0.20180901130821.680a515.el7ost.noarch openstack-tripleo-puppet-elements-9.0.0-0.20180831205939.0641fdc.el7ost.noarch openstack-tripleo-common-containers-9.3.1-0.20180831204016.bb0582a.el7ost.noarch python-openstackclient-lang-3.16.0-0.20180809175603.f77ca68.el7ost.noarch python2-openstackclient-3.16.0-0.20180809175603.f77ca68.el7ost.noarch openstack-tripleo-heat-templates-9.0.0-0.20180831204457.17bb71e.0rc1.el7ost.noarch openstack-heat-engine-11.0.1-0.20180901130821.680a515.el7ost.noarch puppet-openstack_extras-13.3.1-0.20180831173811.9fc5de6.el7ost.noarch python2-openstacksdk-0.17.2-0.20180809182656.3ad9dab.el7ost.noarch openstack-selinux-0.8.15-0.20180823061238.b63283a.el7ost.noarch openstack-heat-api-11.0.1-0.20180901130821.680a515.el7ost.noarch puppet-openstacklib-13.3.1-0.20180822220049.72521cd.el7ost.noarch python-tripleoclient-10.5.1-0.20180901082351.6d7aa74.el7ost.noarch Failed because I have only 8GB on my root, and the validator passed anyways. $ df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 40G 32G 8.8G 79% / devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 758M 7.1G 10% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs 500M 43M 458M 9% /var/log/heat-launcher tmpfs 1.6G 0 1.6G 0% /run/user/1001 tmpfs 1.6G 0 1.6G 0% /run/user/0 In addition to that - the validator has a broken link in the description: http://tripleo.org/environments/environments.html#id5. Please update the url. I also hit this issue trying to do a undercloud re-install while trying to enable autodiscovery puddle 14 -p 2018-12-03.2 Environment: openstack-tripleo-validations-9.3.1-0.20181008110754.4064fb7.el7ost.noarch Checking Hostname... Checking for a FQDN hostname... Static hostname detected as undercloud-0.redhat.local Transient hostname detected as undercloud-0.redhat.local Checking Memory... Checking Disk space... An error occurred during configuration validation, please check your host configuration and try again. Error message: Task 'Fail if any of the volumes are too small' failed: Host: undercloud Message: Minimum free space required for /: 25G - current free space: 24.8G Failure! The validation failed for all hosts: * undercloud END return value: 1 (overcloud) [stack@undercloud-0 ~]$ df --total Filesystem 1K-blocks Used Available Use% Mounted on /dev/vda1 57659264 31651732 26007532 55% / devtmpfs 10174200 0 10174200 0% /dev tmpfs 10197064 0 10197064 0% /dev/shm tmpfs 10197064 923244 9273820 10% /run tmpfs 10197064 0 10197064 0% /sys/fs/cgroup tmpfs 2039416 0 2039416 0% /run/user/0 total 100464072 32574976 67889096 33% - NOTE: I propose you also address the error message while fixing this bug unless you want a separate one. Its very cosmetic. "Failure! The validation failed for all hosts" => Failure! The validation failed for the following hosts:" So moving back to modified as I've verified it works. Now the issue that comes into play around sizing is the determination of what is 1GB. In validation we're using the 1024 b * 1024 kb * 1024 mb = 1 gb (1073741824) rather than the way disks sizes are calculated from manufacturers (1000 b * 1000 kb * 1000 mb). So the way to verify this is to use df with a block-size of 1 and then divide by 1073741824. For example: [stack@undercloud ~]$ df --block-size 1 Filesystem 1B-blocks Used Available Use% Mounted on /dev/vda1 64412954624 38178009088 26234945536 60% / devtmpfs 4077133824 0 4077133824 0% /dev tmpfs 4100538368 0 4100538368 0% /dev/shm tmpfs 4100538368 17260544 4083277824 1% /run tmpfs 4100538368 0 4100538368 0% /sys/fs/cgroup tmpfs 820109312 0 820109312 0% /run/user/1000 This will fail because 26234945536 / 1073741824 = 24.433197021 GB which is less than 25 GB. If this bug requires doc text for errata release, please set the 'Doc Type' and provide draft text according to the template in the 'Doc Text' field. The documentation team will review, edit, and approve the text. If this bug does not require doc text, please set the 'requires_doc_text' flag to -. It is reproduced on OSP 15 with RHEL 8 also compose: RHOS_TRUNK-15.0-RHEL-8-20190311.n.1 [stack@site-undercloud-0 ~]$ rpm -qa "*validation*" openstack-tripleo-validations-10.2.1-0.20190218150113.e6490b3.el8ost.noarch [stack@site-undercloud-0 ~]$ cat undercloud_install.log An error occurred during configuration validation, please check your host configuration and try again. Error message: b"Task 'Fail if any of the volumes are too small' failed:\nHost: undercloud\nMessage: Minimum free space required for /var: 20G - current free space: 5.6G\n\nTask 'Fail if any of the volumes are too small' failed:\nHost: undercloud\nMessage: Minimum free space required for /: 25G - current free space: 5.6G\n\nFailure! The validation failed for all hosts:\n* undercloud\n" [stack@site-undercloud-0 ~]$ df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 9.7G 0 9.7G 0% /dev tmpfs 9.8G 0 9.8G 0% /dev/shm tmpfs 9.8G 17M 9.8G 1% /run tmpfs 9.8G 0 9.8G 0% /sys/fs/cgroup /dev/vda3 8.0G 2.4G 5.7G 30% / /dev/vda1 61G 713M 61G 2% /boot tmpfs 2.0G 0 2.0G 0% /run/user/1001 @Yuri, your partitioning is bad. All your space is assigned to /boot (61G) not / (5.7G) Yes, I've already found that was problem with image http://rhos-qe-mirror-tlv.usersys.redhat.com/brewroot/packages/rhel-guest-image/8.0/1776/images/rhel-guest-image-8.0-1776.x86_64.qcow2 No problem with image http://download-node-02.eng.bos.redhat.com/rel-eng/latest-RHEL-8/compose/BaseOS/x86_64/images/rhel-guest-image-8.0-1807.x86_64.qcow2 Please, ignore my comment. Sorry for disturbing all. Yuri Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0446 |