Bug 1815019 - Kuryr quota calculations are inefficient
Summary: Kuryr quota calculations are inefficient
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.3.z
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.3.z
Assignee: Michał Dulko
QA Contact: GenadiC
URL:
Whiteboard:
Depends On: 1810591
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-19 10:38 UTC by Michał Dulko
Modified: 2023-09-07 22:28 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1810591
Environment:
Last Closed: 2020-04-01 19:10:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift kuryr-kubernetes pull 194 0 None closed [release-4.3] Bug 1815019: Ensures accurate quota calculation during the readiness checks 2020-03-30 15:09:15 UTC
Red Hat Product Errata RHBA-2020:0930 0 None None None 2020-04-01 19:12:14 UTC

Comment 8 Jon Uriarte 2020-03-24 11:32:31 UTC
Verified in 4.3.0-0.nightly-2020-03-23-130439 on top of OSP 13 2020-03-10.1 puddle.

Checked tenant quotas:

$ openstack quota list --network --detail --project shiftstack

+----------------------+--------+----------+-------+
| Resource             | In Use | Reserved | Limit |
+----------------------+--------+----------+-------+
| subnets              |     55 |        0 |   250 |
| routers              |      1 |        0 |    10 |
| security_group_rules |    175 |        0 |  1000 |
| subnet_pools         |      1 |        0 |    -1 |
| security_groups      |     46 |        0 |   250 |
| rbac_policies        |      0 |        0 |    10 |
| floating_ips         |      1 |        0 |    50 |
| networks             |     55 |        0 |   250 |
| ports                |    442 |        0 |  1500 |
+----------------------+--------+----------+-------+

When limiting the quota to the currently used quota there will be no more quota available
for resource creation. It will be detected by Kury-controller and set as Unhealthy due to
readiness probe failure.

ports quota:
-----------

$ openstack quota set --ports 442 shiftstack                                                                                                                                     
$ openstack quota list --network --detail --project shiftstack
+----------------------+--------+----------+-------+
| Resource             | In Use | Reserved | Limit |
+----------------------+--------+----------+-------+
| subnets              |     55 |        0 |   250 |
| routers              |      1 |        0 |    10 |
| security_group_rules |    175 |        0 |  1000 |
| subnet_pools         |      1 |        0 |    -1 |
| security_groups      |     46 |        0 |   250 |
| rbac_policies        |      0 |        0 |    10 |
| floating_ips         |      1 |        0 |    50 |
| networks             |     55 |        0 |   250 |
| ports                |    442 |        0 |   442 |
+----------------------+--------+----------+-------+


2020-03-24 11:23:46.210 1 ERROR kuryr_kubernetes.utils [-] Quota exceeded for resource: ports
2020-03-24 11:23:46.216 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:23:46] "GET /ready HTTP/1.1" 500 -

Events:
  Type     Reason     Age                 From                            Message
  ----     ------     ----                ----                            -------
  Warning  Unhealthy  13s (x4 over 23m)   kubelet, ostest-kzd9b-master-1  Readiness probe failed: HTTP probe failed with statuscode: 500


$ openstack quota set --ports 1500 shiftstack                                                                                                                                    

2020-03-24 11:25:16.253 1 INFO kuryr_kubernetes.controller.managers.health [-] Kuryr Controller readiness verified.
2020-03-24 11:25:16.254 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:25:16] "GET /ready HTTP/1.1" 200 -


subnets quota:
-------------
$ openstack quota set --subnets 55 shiftstack

2020-03-24 11:26:16.684 1 ERROR kuryr_kubernetes.utils [-] Quota exceeded for resource: subnets
2020-03-24 11:26:16.684 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:26:16] "GET /ready HTTP/1.1" 500 -


$ openstack quota set --subnets 250 shiftstack

2020-03-24 11:26:46.489 1 INFO kuryr_kubernetes.controller.managers.health [-] Kuryr Controller readiness verified.
2020-03-24 11:26:46.490 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:26:46] "GET /ready HTTP/1.1" 200 -


networks quota:
--------------
$ openstack quota set --networks 55 shiftstack

2020-03-24 11:27:16.238 1 ERROR kuryr_kubernetes.utils [-] Quota exceeded for resource: networks
2020-03-24 11:27:16.239 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:27:16] "GET /ready HTTP/1.1" 500 -

$ openstack quota set --networks 250 shiftstack

2020-03-24 11:27:46.594 1 INFO kuryr_kubernetes.controller.managers.health [-] Kuryr Controller readiness verified.
2020-03-24 11:27:46.596 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:27:46] "GET /ready HTTP/1.1" 200 -

sec groups quota:
----------------
$ openstack quota set --secgroups 46 shiftstack

2020-03-24 11:28:46.193 1 ERROR kuryr_kubernetes.utils [-] Quota exceeded for resource: security_groups
2020-03-24 11:28:46.194 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:28:46] "GET /ready HTTP/1.1" 500 -

$ openstack quota set --secgroups 250 shiftstack

2020-03-24 11:29:16.316 1 INFO kuryr_kubernetes.controller.managers.health [-] Kuryr Controller readiness verified.
2020-03-24 11:29:16.317 1 INFO werkzeug [-] 10.196.0.12 - - [24/Mar/2020 11:29:16] "GET /ready HTTP/1.1" 200 -

Comment 10 errata-xmlrpc 2020-04-01 19:10:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0930


Note You need to log in before you can comment on or make changes to this bug.