Bug 1703777 - sometimes cluster console is not showing projects list when user logged in
Summary: sometimes cluster console is not showing projects list when user logged in
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.11.z
Assignee: Samuel Padgett
QA Contact: Yadan Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-28 09:25 UTC by Yadan Pei
Modified: 2019-09-24 08:08 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
In rare cases, a race condition could cause the project list to fail to load after logging into the admin console. The user would need to refresh the page to see the list of projects. The problem has been fixed, and projects now load successfully after login.
Clone Of:
Environment:
Last Closed: 2019-09-24 08:08:08 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
DevConsole Error (44.54 KB, image/png)
2019-04-28 09:26 UTC, Yadan Pei
no flags Details
HAR (2.15 MB, text/plain)
2019-08-09 08:59 UTC, Yadan Pei
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift console pull 2651 0 None closed Bug 1703777: Fix race setting CSRF token after login 2020-12-17 20:59:19 UTC
Red Hat Product Errata RHBA-2019:2816 0 None None None 2019-09-24 08:08:18 UTC

Description Yadan Pei 2019-04-28 09:25:45 UTC
Description of problem:
Sometimes when we open admin console , it keeps loading and return nothing until we click some other menus or reload button.
Usually it happens on the first time user opens admin console and only FF in my trying(not sure if it only reproduces on FF)

Version-Release number of selected component (if applicable):
v3.11.98

How reproducible:
Sometimes

Steps to Reproduce:
1. user open admin console URL
2.
3.

Actual results:
1. it keeps loading and nothing returned after a long time, in developer console we can see
Error: "Forbidden"
    f index.tsx:52
    v index.tsx:52

Expected results:
1. project list page should be returned with projects or empty 

Additional info:

Comment 1 Yadan Pei 2019-04-28 09:26:15 UTC
Created attachment 1559610 [details]
DevConsole Error

Comment 2 Samuel Padgett 2019-04-28 12:56:31 UTC
> Actual results:
> 1. it keeps loading and nothing returned after a long time, in developer
> console we can see
> Error: "Forbidden"
>     f index.tsx:52
>     v index.tsx:52

That error is expected depending on your permissions and should not cause a problem.

Any chance you can record a video of what's happening, ideally with the network tab open? It's not clear to me exactly what you're seeing. You keep seeing the loading indicator on the projects page? Is it possible the API response is simply slow? I would check that the initial request for projects has completed in the network tab.(In reply to Yadan Pei from comment #0)

Comment 3 Yadan Pei 2019-04-29 06:34:04 UTC
Sure, I will add a screenshot next time I meet this issue. Yes, only a loading indicator on projects page, I met this when I logins as a normal user and I have only two projects.

Comment 10 Yadan Pei 2019-05-07 07:08:00 UTC
Sam, thanks for your trying. I didn't reproduce today in 3.11 cluster, I will give necessary files next time I meet this issue.

Comment 13 Yadan Pei 2019-05-31 05:31:18 UTC
I tried to reproduce today but without luck. I will keep trying in following days to see if CLI works when this issue is reproduced.

Comment 14 Samuel Padgett 2019-08-07 20:37:23 UTC
Hi, have you been able to reproduce?

Comment 15 Yadan Pei 2019-08-08 03:24:46 UTC
Hi Sam,

I didn't reproduce recently since we did little testing on 3.11, feel free to close it if you want.

Comment 17 Yadan Pei 2019-08-08 05:25:56 UTC
Using oc login can successfully login

Comment 19 Yadan Pei 2019-08-09 08:58:50 UTC
It happens mostly when switch from Developer Console to Cluster Console, visiting Cluster Console route directly seems can't reproduce the issue

Attached is HAR file

Comment 20 Yadan Pei 2019-08-09 08:59:26 UTC
Created attachment 1602071 [details]
HAR

Comment 23 Samuel Padgett 2019-09-09 19:54:10 UTC
It looks like the request for `openshift-favicon.png` set a new `csrf-token` cookie while other requests were being fired off. There is a race condition where GET requests read the current cookie and add the header to the next request. If another response updates the cookie between the time it's read and when the request is sent, the values don't match. I'm not sure why a new cookie was be set for the favicon request, however.

This should be fixed indirectly by https://github.com/openshift/console/pull/2523, but I'd like to understand why the CSRF token was updated.

Comment 24 Samuel Padgett 2019-09-09 20:04:16 UTC
The favicon path is wrong: /auth/static/assets/openshift-favicon.png

This caused the index handler to run, which reset the CSRF cookie.

https://github.com/openshift/console/blob/master/pkg/server/server.go#L318

Comment 25 Samuel Padgett 2019-09-09 20:07:36 UTC
The paths in tokener.html are wrong:

https://github.com/openshift/console/blob/release-3.11/frontend/public/tokener.html

This is only a problem in 3.11.

Comment 28 Yadan Pei 2019-09-18 08:02:40 UTC
# oc get pods -n openshift-console -o yaml | grep image
      image: registry.reg-aws.openshift.com:443/openshift3/ose-console:v3.11
# docker inspect registry.reg-aws.openshift.com:443/openshift3/ose-console:v3.11
....
"Labels":{
"License":"GPLv2+",
"architecture":"x86_64",
"authoritative-source-url":"registry.access.redhat.com",
"build-date":"2019-09-17T22:08:49.979227",
"com.redhat.build-host":"cpt-1004.osbs.prod.upshift.rdu2.redhat.com",
"com.redhat.component":"openshift-enterprise-console-container",
"com.redhat.license_terms":"https://www.redhat.com/en/about/red-hat-end-user-license-agreements",
"description":"ThisisacomponentofOpenShiftContainerPlatformandprovidesawebconsole.",
"distribution-scope":"public",
"io.k8s.description":"ThisisacomponentofOpenShiftContainerPlatformandprovidesawebconsole.",
"io.k8s.display-name":"OpenShiftConsole",
"io.openshift.build.commit.id":"2abfa57365be06e6e14e751ef4b4a6a0dfc8fdaa",
"io.openshift.build.commit.url":"https://github.com/openshift/console/commit/2abfa57365be06e6e14e751ef4b4a6a0dfc8fdaa",
"io.openshift.build.source-location":"https://github.com/openshift/console",
"io.openshift.tags":"openshift,console",
"maintainer":"SamuelPadgett<spadgett>",
"name":"openshift3/ose-console",
"release":"1",
"summary":"ProvidesthelatestreleaseofRedHatEnterpriseLinux7inafullyfeaturedandsupportedbaseimage.",
"url":"https://access.redhat.com/containers/#/registry.access.redhat.com/openshift3/ose-console/images/v3.11.146-1",
"vcs-ref":"8df6ebf6dfbd0a53196640d0e3b60006af4bcb57",
"vcs-type":"git",
"vendor":"RedHat,Inc.",
"version":"v3.11.146"
....

I don't see this issue happen on 3.11.146

Comment 29 Yadan Pei 2019-09-18 08:03:10 UTC
Thanks very much for the fix and will continue to track this issue in next few builds

Comment 31 errata-xmlrpc 2019-09-24 08:08:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2816


Note You need to log in before you can comment on or make changes to this bug.