Bug 1937460 - [CI] Network quota pre-flight checks are failing the installation
Summary: [CI] Network quota pre-flight checks are failing the installation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: ---
: 4.8.0
Assignee: Martin André
QA Contact: Udi Shkalim
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-10 17:08 UTC by Pierre Prinetti
Modified: 2021-07-27 22:53 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: In some conditions, Neutron will JSON-encode quota values as strings instead of integers breaking their API contract. Consequence: Installer fails to parse Neutron's response and fails pre-flight validations. Fix: Make gophercloud, the installer's underlying library talking to OpenStack, tolerate the badly formated Neutron response. Result: Pre-flight validations are now returning expected results.
Clone Of:
Environment:
Last Closed: 2021-07-27 22:52:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github gophercloud gophercloud pull 2126 0 None closed Neutron quota: int-as-string workaround 2021-03-19 17:41:46 UTC
Github openshift installer pull 4741 0 None closed Bug 1937460: Update gophercloud module from master 2021-03-19 17:41:51 UTC
Launchpad 1918565 0 None None None 2021-03-11 03:53:30 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:53:05 UTC

Description Pierre Prinetti 2021-03-10 17:08:10 UTC 
Some (parallel) 4.8 periodic jobs are failing with:

```
failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Quota Check": failed to get cloud info: failed to generate OpenStack cloud info: failed to load Quota: failed to load quota limits: failed to get network quota records: failed to get QuotaSets from OpenStack Network API: json: cannot unmarshal string into Go struct field QuotaDetail.quota.network.reserved of type int
```

Failed builds so far:

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-openstack-4.8/1369468189917843456

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-openstack-4.8/1369588989123629056
Comment 1 Emilien Macchi 2021-03-11 02:52:43 UTC 
Related to https://bugs.launchpad.net/neutron/+bug/1918565
Comment 2 Martin André 2021-03-11 08:19:29 UTC 
Looks like https://review.opendev.org/c/openstack/neutron/+/779878 is going to fix the issue on Neutron side, but until then we'll need to make the quota validation tolerant to the badly formatted json and ignore the failure, allowing to go on with the installation.

Ideally the mitigation happens in gophercloud, where we unmarshal the json but I don't think that's possible.
Comment 3 Pierre Prinetti 2021-03-11 09:46:12 UTC 
I have filed a Gophercloud patch[1]; if accepted, we will need to bump Gophercloud in the installer.

[1]: https://github.com/gophercloud/gophercloud/pull/2126
Comment 5 Udi Shkalim 2021-05-06 10:59:10 UTC 
Verified on:
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-04-30-201824   True        False         21h     Cluster version is 4.8.0-0.nightly-2021-04-30-201824
Comment 8 errata-xmlrpc 2021-07-27 22:52:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438
Note You need to log in before you can comment on or make changes to this bug.