Bug 1648110 - user's project list is empty sometimes
Summary: user's project list is empty sometimes
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: openshift-apiserver
Version: 3.9.0
Hardware: All
OS: Linux
medium
high
Target Milestone: ---
: 3.9.z
Assignee: Stefan Schimanski
QA Contact: Xingxing Xia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-08 20:54 UTC by Alex Chvatal
Modified: 2021-12-10 18:12 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-20 18:56:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Alex Chvatal 2018-11-08 20:54:06 UTC
Description of problem:
When loading up the web console, the user's projects aren't displayed until they reload the page.

The user sees the same behavior when using `oc get projects` - sometimes the project list is empty and sometimes it lists projects.

When the customer tries to delete the user from the membership of the projects, the browser shows around 8126 requests, making their browser hang.


Version-Release number of selected component (if applicable):
v3.9.41

How reproducible:
It seems unpredictable but frequent, around 50% of the calls to `oc get projects`

Steps to Reproduce:
1. Run `oc get projects` or load the web console

Actual results:
No projects listed

Expected results:
Projects listed

Comment 8 Alex Chvatal 2018-11-08 21:03:50 UTC
Created attachment 1503467 [details]
multiple `oc projects` output pt 2

Comment 15 Alex Chvatal 2018-11-12 14:54:31 UTC
We've moved the masters' /var/lib/etcd directories onto their own volumes. It looks like that's alleviating this issue, and we're going to confirm with the customer.

Comment 18 Alex Chvatal 2018-11-13 19:50:39 UTC
Scratch my last, the issue has not been resolved by moving etcd onto its own volume.

It sounds like the customer has a cron-db-backup pod that is created every hour, which is using a _lot_ of network traffic. 3 mins past every hour, the cluster is basically unusable for 10/11 mins.  etcd, api - they are saturating their Direct connect link.

Comment 20 Stephen Cuppett 2019-11-20 18:56:10 UTC
OCP 3.6-3.10 is no longer on full support [1]. Marking CLOSED DEFERRED. If you have a customer case with a support exception or have reproduced on 3.11+, please reopen and include those details. When reopening, please set the Target Release to the appropriate version where needed.

[1]: https://access.redhat.com/support/policy/updates/openshift


Note You need to log in before you can comment on or make changes to this bug.