Bug 1856354 - New APIServices missing from OpenAPI definitions
Summary: New APIServices missing from OpenAPI definitions
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.5
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: 4.7.0
Assignee: Jon Jackson
QA Contact: Yanping Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-13 13:00 UTC by Paul Weil
Modified: 2021-02-24 15:15 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: OpenAPI definitions were only updated on initial page load. Consequence: New OpenAPI definitions that were added by CRD were not reflected in the app until a page refresh. Fix: Update OpenAPI definitions on a 5-minute interval and whenever we fetch models from the API. Result: OpenAPI definitions stay up to date without a page refresh.
Clone Of:
Environment:
Last Closed: 2021-02-24 15:13:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
auto-complete-suggestion (17.46 KB, image/png)
2020-11-11 09:03 UTC, Yanping Zhang
no flags Details
Mock CRD with structural schema (512 bytes, text/plain)
2020-11-16 19:52 UTC, Jon Jackson
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift console pull 6227 0 None closed Bug 1866114: Don't store OpenAPI definitions in localStorage 2021-01-29 13:13:48 UTC
Github openshift console pull 6857 0 None closed Bug 1856354: Keep swagger definitions up to date 2021-01-29 13:13:48 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:15:02 UTC

Description Paul Weil 2020-07-13 13:00:25 UTC
When new CustomResourceDefinitions or APIServices are added to the cluster (for instance by installing an Operator), their JSONSchema will be added to the /openapi/v2 endpoint. Console will register the new models using API discovery, but does not refresh the OpenAPI definitions in localStorage (used for YAML editor autocomplete, etc). This document is only refreshed once per day because it is very large to download.

Solution is to refresh the openapi/v2 endpoint whenever new models are added (can compare GroupVersionKinds for registered models versus stored OpenAPI definitions).

Comment 1 Jon Jackson 2020-07-31 19:47:07 UTC
Will address next sprint.

Comment 2 Jon Jackson 2020-08-11 02:07:54 UTC
OpenAPI definitions are no longer stored in local storage as of https://github.com/openshift/console/pull/6227. We now rely on browser cache for optimization.

Comment 3 Samuel Padgett 2020-08-11 13:26:58 UTC
This should still be an issue since we keep the definitions in memory and only refresh on console load. We'll need to fetch the document again after running API discovery.

Comment 4 Jon Jackson 2020-08-11 15:11:51 UTC
My mistake. At first glance at your PR, it looked like it would fix this, but now I see that it still only gets updated when `fetchSwagger` is called.

Comment 5 Jon Jackson 2020-08-21 19:13:23 UTC
Briefly looked into this, but haven't had a chance to dig in to it. Will address in upcoming sprint.

Comment 6 Jon Jackson 2020-08-21 19:13:46 UTC
Briefly looked into this, but haven't had a chance to dig in to it. Will address in upcoming sprint.

Comment 7 Jon Jackson 2020-09-11 21:06:24 UTC
Still haven't gotten to this yet. Did a little bit of exploration for possible approaches. Essentially, I think we are going to need a global watch to make sure that the schema definitions stay up to date.

Comment 8 Jon Jackson 2020-10-02 13:39:07 UTC
Still haven't had a chance to research a solution for this yet.

Comment 9 Samuel Padgett 2020-10-02 14:15:37 UTC
(In reply to Jon Jackson from comment #7)
> Still haven't gotten to this yet. Did a little bit of exploration for
> possible approaches. Essentially, I think we are going to need a global
> watch to make sure that the schema definitions stay up to date.

Hey, Jon. We do have a global watch on API services that triggers an update to API discovery:

https://github.com/spadgett/console/blob/d3fa4db55a58c1e4dbfc5d2c59b3b073653dbcfd/frontend/public/actions/k8s.ts#L281-L315

We might just want to refresh the OpenAPI doc whenever we call `getResources`. The endpoint uses ETags and browser caching, so there should be no significant overhead when the document hasn't changed.

Comment 10 Jon Jackson 2020-10-02 15:37:26 UTC
(In reply to Samuel Padgett from comment #9)
> (In reply to Jon Jackson from comment #7)
> > Still haven't gotten to this yet. Did a little bit of exploration for
> > possible approaches. Essentially, I think we are going to need a global
> > watch to make sure that the schema definitions stay up to date.
> 
> Hey, Jon. We do have a global watch on API services that triggers an update
> to API discovery:
> 
> https://github.com/spadgett/console/blob/
> d3fa4db55a58c1e4dbfc5d2c59b3b073653dbcfd/frontend/public/actions/k8s.ts#L281-
> L315
> 
> We might just want to refresh the OpenAPI doc whenever we call
> `getResources`. The endpoint uses ETags and browser caching, so there should
> be no significant overhead when the document hasn't changed.

Great, thanks for the pointer.

Comment 11 Jon Jackson 2020-10-23 15:09:51 UTC
PR pending

Comment 13 Yanping Zhang 2020-11-11 09:02:57 UTC
Checked on ocp 4.7 with payload: 4.7.0-0.nightly-2020-11-11-033756.
Install etcd operator in project, then there is new api: EtcdCluster, EtcdBackup, EtcdRestore. In yaml creation, try to set kind using auto complete, but it doesn't show the new api in suggestion list. Refer to the screenshot.
@Jon, if my steps are correct to check the bug?

Comment 14 Yanping Zhang 2020-11-11 09:03:53 UTC
Created attachment 1728278 [details]
auto-complete-suggestion

Comment 15 Jon Jackson 2020-11-16 19:52:50 UTC
Created attachment 1729883 [details]
Mock CRD with structural schema

Comment 16 Jon Jackson 2020-11-16 19:56:24 UTC
(In reply to Yanping Zhang from comment #13)
> Checked on ocp 4.7 with payload: 4.7.0-0.nightly-2020-11-11-033756.
> Install etcd operator in project, then there is new api: EtcdCluster,
> EtcdBackup, EtcdRestore. In yaml creation, try to set kind using auto
> complete, but it doesn't show the new api in suggestion list. Refer to the
> screenshot.
> @Jon, if my steps are correct to check the bug?

Yanping, I've attached a YAML file with a CRD that has a valid structural schema. You should be able to create this resource, then after a 10 second delay, it should populate in the suggestions for "kind" as "MockResource".

The etcd CRDs do not have valid structural schemas, which is why they were not showing up.

Comment 17 Yanping Zhang 2020-11-17 07:55:49 UTC
Thanks, Jon, the file works.
Checked on ocp 4.7 cluster with payload 4.7.0-0.nightly-2020-11-12-200927.
Created CRD "mock-resources.test.tectonic.com" with yaml in attachment, then create yaml, for "kind", input "Mo", and use ctrl+space key to show suggestion, there is "MockResource" shown in the list.
The bug is fixed.

Comment 20 errata-xmlrpc 2021-02-24 15:13:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.