Bug 1825207 - [4.5] 504 error for Prometheus API in administrator console with kuryr network
Summary: [4.5] 504 error for Prometheus API in administrator console with kuryr network
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.5
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.5.0
Assignee: Maysa Macedo
QA Contact: Jon Uriarte
URL:
Whiteboard:
Depends On:
Blocks: 1825215
TreeView+ depends on / blocked
 
Reported: 2020-04-17 11:54 UTC by Junqi Zhao
Modified: 2020-07-13 17:28 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1825215 (view as bug list)
Environment:
Last Closed: 2020-07-13 17:28:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
504 error for the Prometheus APIs for "Home -> Overview" page (232.53 KB, image/png)
2020-04-17 11:54 UTC, Junqi Zhao
no flags Details
failed to load metrics list due to 504 error (80.70 KB, image/png)
2020-04-17 12:03 UTC, Junqi Zhao
no flags Details
GET requests from console GUI to prometheus API (226.57 KB, image/png)
2020-04-30 13:48 UTC, Jon Uriarte
no flags Details
"Monitoring -> Metrics" menu from console GUI (71.66 KB, image/png)
2020-04-30 13:49 UTC, Jon Uriarte
no flags Details
kube_pod_info_node_coun metric retrieval from console GUI (77.18 KB, image/png)
2020-04-30 13:50 UTC, Jon Uriarte
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift kuryr-kubernetes pull 215 0 None closed Bug 1824258: Ensure LB state annotation sg matches the SG on the LB 2020-06-23 10:05:34 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:28:54 UTC

Description Junqi Zhao 2020-04-17 11:54:53 UTC
Created attachment 1679633 [details]
504 error for the Prometheus APIs for "Home -> Overview" page

Created attachment 1679633 [details]
504 error for the Prometheus APIs for "Home -> Overview" page

Description of problem:
This bug is from Bug 1822861, use it to track the 504 error for the Prometheus APIs in the administrator console.

Login the administrator console, check in "Home -> Overview" page, "Monitoring -> Alerting", or "Monitoring -> Metrics" page, the Prometheus APIs in the administrator console returns 504 error in the kuryr OCP 4.5.0-0.nightly-2020-04-17-053057 cluster, see the attached picture 



Version-Release number of selected component (if applicable):
kuryr OCP 4.5.0-0.nightly-2020-04-17-05305

How reproducible:
always

Steps to Reproduce:
1. See the description
2.
3.

Actual results:
504 error for Prometheus API in administrator console 

Expected results:
no error

Additional info:

Comment 1 Junqi Zhao 2020-04-17 12:03:22 UTC
Created attachment 1679637 [details]
failed to load metrics list due to 504 error

Comment 2 Jon Uriarte 2020-04-30 13:47:13 UTC
Verified in 4.5.0-0.nightly-2020-04-29-144201 on top of RHOS_TRUNK-16.0-RHEL-8-20200427.n.0 OSP 16 compose.
OSP 16 deployment with OVS neutron backend.

OCP 4.5 installation with Kuryr completed successfully.

There are no 504 error responses to Prometheus metrics requests in the GUI "Home -> Overview" page.
The metrics are retrieved correctly in "Monitoring -> Metrics"

The metrics can be retrieved from the CLI as well:

$ token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://10.196.2.22:9100/metrics'

Comment 3 Jon Uriarte 2020-04-30 13:48:33 UTC
Created attachment 1683314 [details]
GET requests from console GUI to prometheus API

Comment 4 Jon Uriarte 2020-04-30 13:49:25 UTC
Created attachment 1683315 [details]
"Monitoring -> Metrics" menu from console GUI

Comment 5 Jon Uriarte 2020-04-30 13:50:11 UTC
Created attachment 1683316 [details]
kube_pod_info_node_coun metric retrieval from console GUI

Comment 7 errata-xmlrpc 2020-07-13 17:28:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.