Bug 1891543 - OpenShift 4.6/OSP install fails when node flavor has less than 25GB, even with dedicated storage
Summary: OpenShift 4.6/OSP install fails when node flavor has less than 25GB, even wit...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.7
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 4.7.0
Assignee: Emilien Macchi
QA Contact: weiwei jiang
URL:
Whiteboard:
Depends On:
Blocks: 1899161
TreeView+ depends on / blocked
 
Reported: 2020-10-26 15:06 UTC by Ben Pritchett
Modified: 2024-03-25 16:49 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Installer checks that the minimum 25GB disk is available per node. However, this validation only checks the osp flavor, and does not validate if separate rootDisk has been attached from dedicated storage. Consequence: When using a small flavor in combination with sufficient rootDisk, Installer refuses to install. Fix: With this patch, the additional rootDisk is taken into consideration when validating the required disk space. Result: A successful install can be carried out using a rootDisk in combination with a flavor with a small disk.
Clone Of:
: 1899161 (view as bug list)
Environment:
Last Closed: 2021-02-24 15:28:00 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
example install-config (1.00 KB, text/plain)
2020-10-26 15:06 UTC, Ben Pritchett
no flags Details
flavor combinations test report (252.12 KB, text/plain)
2020-11-11 06:34 UTC, weiwei jiang
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4323 0 None closed Bug 1891543: openstack: consider volumes for storage requirements checks 2021-02-21 10:25:32 UTC
Github openshift installer pull 4383 0 None closed Bug 1891543: openstack: remove platform flavor validation 2021-02-21 10:25:33 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:28:23 UTC

Description Ben Pritchett 2020-10-26 15:06:38 UTC
Created attachment 1724186 [details]
example install-config

Thanks for opening a bug report!
Before hitting the button, please fill in as much of the template below as you can.
If you leave out information, it's harder to help you.
Be ready for follow-up questions, and please respond in a timely manner.
If we can't reproduce a bug we might close your issue.
If we're wrong, PLEASE feel free to reopen it and explain why.

Version:

$ openshift-install version
$ ./openshift-install-pre version
./openshift-install-pre 4.6.0-0.nightly-2020-10-03-051134
built from commit bff124c9941762d2532490774b3f910241bd63f6
release image quay.io/openshift-release-dev/ocp-release-nightly@sha256:9663f178a9a5bf87fad0d4e2dabeaef32110d4c9c3d400eededd4f6bff5109fc

Platform:

#Please specify the platform type: aws, libvirt, openstack or baremetal etc.
openstack

Please specify:
* IPI (automated install with `openshift-install`. If you don't know, then it's IPI)
* UPI (semi-manual installation on customized infrastructure)
IPI

What happened?

With OCP 4.6 and the IPI install on OpenStack, it looks like there are new validations added to check necessary disk/memory/compute preinstall. One of the checks is to ensure the minimum 25GB disk is available per node. However, this validation only checks the osp flavor, and does not validate if separate rootDisk has been attached from dedicated storage. This means if a osp flavor is used with less than 25GB on ephemeral storage, the installer will error out, even if dedicated rootDisk is requested for more than 25GB.

#See the troubleshooting documentation (https://github.com/openshift/installer/blob/master/docs/user/troubleshooting.md) for ideas about what information to collect.

$ cat .openshift_install.log 
time="2020-10-26T14:54:50Z" level=debug msg="OpenShift Installer 4.6.0-0.nightly-2020-10-03-051134"
time="2020-10-26T14:54:50Z" level=debug msg="Built from commit bff124c9941762d2532490774b3f910241bd63f6"
time="2020-10-26T14:54:50Z" level=debug msg="Fetching Metadata..."
time="2020-10-26T14:54:50Z" level=debug msg="Loading Metadata..."
time="2020-10-26T14:54:50Z" level=debug msg="  Loading Cluster ID..."
time="2020-10-26T14:54:50Z" level=debug msg="    Loading Install Config..."
time="2020-10-26T14:54:50Z" level=debug msg="      Loading SSH Key..."
time="2020-10-26T14:54:50Z" level=debug msg="      Loading Base Domain..."
time="2020-10-26T14:54:50Z" level=debug msg="        Loading Platform..."
time="2020-10-26T14:54:50Z" level=debug msg="      Loading Cluster Name..."
time="2020-10-26T14:54:50Z" level=debug msg="        Loading Base Domain..."
time="2020-10-26T14:54:50Z" level=debug msg="        Loading Platform..."
time="2020-10-26T14:54:50Z" level=debug msg="      Loading Pull Secret..."
time="2020-10-26T14:54:50Z" level=debug msg="      Loading Platform..."
time="2020-10-26T14:54:52Z" level=fatal msg="failed to fetch Metadata: failed to load asset \"Install Config\": platform.openstack.flavorName: Invalid value: \"osp.2xlarge\": Flavor did not meet the following minimum requirements: Must have minimum of 25 GB Disk, had 20 GB"


What did you expect to happen?

With attached dedicated storage, the validation for 25GB minimum disk is not checked only on the osp flavor.

How to reproduce it (as minimally and precisely as possible)?

$ openshift-install create cluster --dir=my-cluster/ --log-level=debug
In this case the install-config.yaml references a osp flavor with less than 25GB, but has rootDisk attributed to the node from dedicated storage offering (see example install-config.yaml)

Comment 8 weiwei jiang 2020-11-10 07:42:33 UTC
Checked with 
$ ./openshift-install-4.7 version 
./openshift-install-4.7 4.7.0-0.nightly-2020-11-09-190845
built from commit 519f21bfada103752bb78929f170580d505b92c0
release image registry.svc.ci.openshift.org/ocp/release@sha256:f7bf10489f11c05927e901310ad7d824599b385f69f12c2de61cf8c437c85fc4

and found that rootVolume do not take the first priority to decide the failure.


apiVersion: v1           
baseDomain: 10.0.103.240.nip.io
compute:   
- architecture: amd64
  hyperthreading: Enabled
  name: worker     
  platform:       
    rootVolume:
      size: 25
      type: tripleo  
    type: ci.usm-qe-02   
  replicas: 2 
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master     
  platform:                                                                                                                                                                                                                                                                     
    rootVolume:
      size: 25
      type: tripleo 
    type: ci.usm-qe-02
  replicas: 3    
metadata:              
  name: wj45ios1019a
networking:      
  clusterNetwork:       
  - cidr: 10.128.0.0/14    
    hostPrefix: 23
  machineNetwork:
  - cidr: 192.168.0.0/18
  networkType: OpenShiftSDN
  serviceNetwork: 
  - 172.30.0.0/16              
platform:                              
  openstack:                  
    cloud: upshift     
    computeFlavor: ci.usm-qe-02
    externalNetwork: provider_net_cci_8
    lbFloatingIP: 10.0.103.240
    octaviaSupport: '0'
    region: regionOne                                                                                                                                                                                                                                                           
    trunkSupport: '1'                                                                                                                   
publish: External        
pullSecret: Hidden                                                                                                                      
sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa3
7P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D
  openshift-qe
Running: ./openshift-install-4.7 create manifests --dir /tmp/tmp.cbKWELvH3Y
FATAL failed to fetch Master Machines: failed to load asset "Install Config": platform.openstack.computeFlavor: Invalid value: "ci.usm-qe-02": Flavor did not meet the following minimum requirements: Must have minimum of 25 GB Disk, had 20 GB

Comment 12 weiwei jiang 2020-11-11 06:34:20 UTC
Created attachment 1728265 [details]
flavor combinations test report

Comment 32 David Hernández Fernández 2020-12-08 09:46:30 UTC
For reference, backport to 4.6.z: https://bugzilla.redhat.com/show_bug.cgi?id=1899161  | https://github.com/openshift/installer/pull/4394

Comment 35 egarcia 2021-02-04 20:12:39 UTC
What disk size does the image you are using have? Could we also see the install config?

Comment 36 Emilien Macchi 2021-02-04 20:50:24 UTC
also please provide details on the flavor m6.xlarge.8 (specs).

Comment 41 errata-xmlrpc 2021-02-24 15:28:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633

Comment 43 Pierre Prinetti 2021-11-11 07:48:24 UTC
Blake,
Please create a new BZ detailing:
* the Installer version
* the error output 
* a redacted copy-paste of your install-config
* the output of `openstack flavor show ${flavor}` for the master or the worker flavor, depending on what's causing problem

Thank you!


Note You need to log in before you can comment on or make changes to this bug.