Bug 1989917 - OpenStack inconsistency reports on limits numbers for network quota check
Summary: OpenStack inconsistency reports on limits numbers for network quota check
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.9
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: ---
: 4.9.0
Assignee: Eric Duen
QA Contact: Itay Matza
URL:
Whiteboard:
Depends On:
Blocks: 1978213
TreeView+ depends on / blocked
 
Reported: 2021-08-04 10:42 UTC by Itay Matza
Modified: 2021-10-18 17:44 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-18 17:44:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 5140 0 None None None 2021-08-10 01:05:28 UTC
Red Hat Product Errata RHSA-2021:3759 0 None None None 2021-10-18 17:44:43 UTC

Description Itay Matza 2021-08-04 10:42:08 UTC
Version:
OSP: RHOS-16.1-RHEL-8-20210604.n.0

Openshift-install:
openshift-install 4.9.0-0.nightly-2021-08-03-200806
built from commit f531b868b0907fb5506e5c87393057085ba048ab
release image registry.ci.openshift.org/ocp/release@sha256:b4694cf1ea51c37a7956a916d724a1738f86ad546f260d9614a776dd18866e99


Platform:
OpenShift on OpenStack with Kuryr.


Please specify:
IPI


What happened?
Quota check is failing when using Kuryr:
>(overcloud) [stack@undercloud-0 ~]$ openstack quota show shiftstack | grep "port\|secgroup"
>| ports                 | 1500                                                                                                                                                                                       >|
>| secgroup-rules        | 1000                                                                                                                                                                                       >|
>| secgroups             | 250                                                                                                                                                                                        >|
>(shiftstack) [stack@undercloud-0 ~]$ openshift-install create cluster --dir ostest/
>FATAL failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Quota Check": error(MissingQuota): Port is not available because the required number >of resources (1500) is more than the limit of 1498, SecurityGroup is not available because the required number of resources (250) is more than the limit of 249, SecurityGroupRule is not >available because the required number of resources (1000) is more than the limit of 996 

There is an inconsistency report on limits numbers, for example -
The limit of the number of ports is 1498 even though that the number of ports on quota is configured as 1500.


What did you expect to happen?
I expect that the quota configuration will be compatible with the limits numbers.


How to reproduce it?
$ openstack quota set shiftstack --ports 1500 --secgroup-rules 1000 --secgroups 250
$ openshift-install create cluster --dir <ocp_installation_dir>

Comment 1 Martin André 2021-08-04 11:43:06 UTC
This might be one of few things:
- we run the wrong API call to get networking quota;
- a bug on openstack side when returning networking quota;
- the intended, but confusing, behavior of openstack.

Comment 2 Emilien Macchi 2021-08-04 16:31:00 UTC
Itay,

Before running the OpenShift deployment, please run the following commands with the same tenant that is used to deploy OCP:

$ openstack port list
$ openstack security group list


I think that you already have ports & SGs resources, which would cause this message to happen.

During a meeting today, Matt and Martin proposed that we change the wording:
e.g. because the required number of resources (1000) is superior than the available resources (996) 

Thanks

Comment 4 Udi Shkalim 2021-08-05 08:36:17 UTC
(In reply to Emilien Macchi from comment #2)
> Itay,
> 
> Before running the OpenShift deployment, please run the following commands
> with the same tenant that is used to deploy OCP:
> 
> $ openstack port list
> $ openstack security group list
> 
> 
> I think that you already have ports & SGs resources, which would cause this
> message to happen.
> 
> During a meeting today, Matt and Martin proposed that we change the wording:
> e.g. because the required number of resources (1000) is superior than the
> available resources (996) 
> 
> Thanks

Hi Emilien,

Just to confirm, the installer is looking for the Available or Defined quota?

Comment 5 Emilien Macchi 2021-08-06 21:49:20 UTC
(In reply to Udi Shkalim from comment #4)
> (In reply to Emilien Macchi from comment #2)
> > Itay,
> > 
> > Before running the OpenShift deployment, please run the following commands
> > with the same tenant that is used to deploy OCP:
> > 
> > $ openstack port list
> > $ openstack security group list
> > 
> > 
> > I think that you already have ports & SGs resources, which would cause this
> > message to happen.
> > 
> > During a meeting today, Matt and Martin proposed that we change the wording:
> > e.g. because the required number of resources (1000) is superior than the
> > available resources (996) 
> > 
> > Thanks
> 
> Hi Emilien,
> 
> Just to confirm, the installer is looking for the Available or Defined quota?

the available resources from quotas are calculated with: quota.Limit - quota.InUse - quota.Reserved

Comment 6 Emilien Macchi 2021-08-06 21:56:17 UTC
We need to relax the security group as well because OpenStack provides a default one, so to be fully compatible we need to relax the number to 249 as well.

I'll send a PR.

Comment 7 Itay Matza 2021-08-09 12:59:17 UTC
(In reply to Emilien Macchi from comment #6)
> We need to relax the security group as well because OpenStack provides a
> default one, so to be fully compatible we need to relax the number to 249 as
> well.
> 
> I'll send a PR.

Hi Emilien,

I think that we need to relax the security group rule as well.
It seems like we have also default security group rules - Comment 3:

>(shiftstack) [stack@undercloud-0 ~]$ openstack security group list                                                                                                                                                 +--------------------------------------+---------+------------------------+----------------------------------+------+                                                                                              | ID                                   | Name    | Description            | Project                          | Tags |
>+--------------------------------------+---------+------------------------+----------------------------------+------+
>| 56a9feb5-097d-4ab1-a37c-b3635812c1fe | default | Default security group | da323a9db99142c48250c871bc6fb5ee | []   |
>+--------------------------------------+---------+------------------------+----------------------------------+------+
>(shiftstack) [stack@undercloud-0 ~]$ openstack security group rule list
>+--------------------------------------+-------------+-----------+-----------+------------+--------------------------------------+--------------------------------------+
>| ID                                   | IP Protocol | Ethertype | IP Range  | Port Range | Remote Security Group                | Security Group                       |
>+--------------------------------------+-------------+-----------+-----------+------------+--------------------------------------+--------------------------------------+
>| 5574b63a-9b01-4385-ad96-6a1b6bc5fc9f | None        | IPv4      | 0.0.0.0/0 |            | None                                 | 56a9feb5-097d-4ab1-a37c-b3635812c1fe |
>| 692e6c00-1174-402e-a137-443f6fa0d732 | None        | IPv4      | 0.0.0.0/0 |            | 56a9feb5-097d-4ab1-a37c-b3635812c1fe | 56a9feb5-097d-4ab1-a37c-b3635812c1fe |
>| 6edf29f7-cf00-4535-aa98-3463d255b13f | None        | IPv6      | ::/0      |            | 56a9feb5-097d-4ab1-a37c-b3635812c1fe | 56a9feb5-097d-4ab1-a37c-b3635812c1fe |
>| efbd254e-a44b-40e9-9c70-38b9f2d1f26f | None        | IPv6      | ::/0      |            | None                                 | 56a9feb5-097d-4ab1-a37c-b3635812c1fe |
>+--------------------------------------+-------------+-----------+-----------+------------+--------------------------------------+--------------------------------------+

Comment 13 Itay Matza 2021-08-22 09:27:10 UTC
Verified successfully on openshift-install version:
openshift-install 4.9.0-0.nightly-2021-08-18-033031
built from commit cd2c598ae11c088eb65d8f84bc673e4c8b12f09c
release image registry.ci.openshift.org/ocp/release@sha256:29fe71a2d014c3fc34dfa040766d8c335eb4b3f385ac3f5667c3ff7a0e9a9d4e


1) For Kuryr:
>(shiftstack) [stack@undercloud-0 ~]$ grep type install-config.yaml
>  type: "Kuryr"
>(overcloud) [stack@undercloud-0 ~]$ openstack quota show shiftstack | grep "port\|secgroup"
>| ports                 | 1500                                                                                                                                                                                    
>| secgroup-rules        | 1000                                                                                                                                                                                    
>| secgroups             | 250                                                                                                                                                                                     
>(overcloud) [stack@undercloud-0 ~]$ openshift-install create cluster --dir ostest/                                                                                                                                
>INFO Credentials loaded from file "/home/stack/clouds.yaml"
>INFO Consuming Install Config from target directory
>INFO Obtaining RHCOS image file from 'https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.9/49.84.202107010027-0/x86_64/rhcos-49.84.202107010027-0-openstack.x86_64.qcow2.gz?sha256=00cb56c8711686255744646394e22a8ca5f27e059016f6758f14388e5a0a14cb'
>INFO The file was found in cache: /home/stack/.cache/openshift-installer/image_cache/rhcos-49.84.202107010027-0-openstack.x86_64.qcow2. Reusing...                                                                
>WARNING Following quotas SecurityGroup, RAM, Port, Subnet, Network, SecurityGroupRule are available but will be completely used pretty soon.                                                                      
>INFO Creating infrastructure resources... 


2) For other types of networks:
>(overcloud) [stack@undercloud-0 ~]$ grep type install-config.yaml                                                                                                                            
>  type: "OpenshiftSDN" 
>(overcloud) [stack@undercloud-0 ~]$ openstack quota set shiftstack --ports 15 --secgroup-rules 60 --secgroups 3                                                                                                    
>(overcloud) [stack@undercloud-0 ~]$ openstack quota show shiftstack | grep "port\|secgroup"                                              
>| ports                 | 15                                                                                                                                                                                      
>| secgroup-rules        | 60                                                                                                                                                                                       
>| secgroups             | 3 
>(overcloud) [stack@undercloud-0 ~]$ openshift-install create cluster --dir ostest/                                                      
>INFO Credentials loaded from file "/home/stack/clouds.yaml"                                                                                                                                                       
>INFO Consuming Install Config from target directory                                                                                                                                                                
>INFO Obtaining RHCOS image file from 'https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.9/49.84.202107010027-0/x86_64/rhcos-49.84.202107010027-0-openstack.x86_64.qcow2.gz?sha256=00cb56c8711686255744646394e22a8ca5f27e059016f6758f14388e5a0a14cb'                                                                                                                                                       
>INFO The file was found in cache: /home/stack/.cache/openshift-installer/image_cache/rhcos-49.84.202107010027-0-openstack.x86_64.qcow2. Reusing...                                                                 
>WARNING Following quotas RAM, Port, SecurityGroupRule, SecurityGroup are available but will be completely used pretty soon.   
>INFO Creating infrastructure resources...

Comment 16 errata-xmlrpc 2021-10-18 17:44:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3759


Note You need to log in before you can comment on or make changes to this bug.