Bug 1913802

Summary: Unable to create 'app' index pattern in OpenShift Logging 4.5 when permissions are granted via group
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: LoggingAssignee: Lukas Vlcek <lvlcek>
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: high Docs Contact:
Priority: high    
Version: 4.5CC: alchan, anli, aos-bugs, bjarolim, ewolinet, jcantril, lvlcek, mrobson, openshift-bugs-escalate, periklis, scott.worthington, sponnaga, sreber, vlaad, wzheng
Target Milestone: ---   
Target Release: 4.5.z   
Hardware: x86_64   
OS: Linux   
Whiteboard: logging-exploration
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-09 13:25:23 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1913801    
Bug Blocks:    

Comment 5 Anping Li 2021-02-03 16:24:06 UTC
The group user can see the app index in kibana on elasticsearch-operator.4.5.0-202102030632.p0.  
No openshift proejct: 151
Active app: 1
ES: resource
  logStore:
    elasticsearch:
      nodeCount: 3
      redundancyPolicy: SingleRedundancy
      resources:
        limits:
          cpu: 1
          memory: 4Gi
        requests:
          cpu: 1
          memory: 4Gi
      storage:
        size: 200G
        storageClassName: gp2


I hit resouce limitaton in testing. the fluend cann't send some logs to ES before I reduce the ES Memory from 8Gi to 4Gi. I will verify this again on a larger cluster.
2021-02-03 15:56:28 +0000 [warn]: [clo_default_output_es] failed to flush the buffer. retry_time=7 next_retry_seconds=2021-02-03 15:57:35 +0000 chunk="5ba709e712fcc043e6d839acf0061e3f" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>\"elasticsearch.openshift-logging.svc.cluster.local\", :port=>9200, :scheme=>\"https\", :user=>\"fluentd\", :password=>\"obfuscated\"}): [500] {\"code\":500,\"message\":\"Internal Error\",\"error\":{}}\n"
  2021-02-03 15:56:28 +0000 [warn]: suppressed same stacktrace

Comment 6 Anping Li 2021-02-04 11:34:11 UTC
The problem was reproduced elasticsearch-operator.4.5.0-202101230744.p0 in cluster below. and verified on using elasticsearch-operator.4.5.0-202102031005.p0 
Cluster Info:
AWS Clusters
Master: 3* M3.xlarge
Worker: 4* M4.X4Large 
ES: nodeCount: 3
Elasticsearch: "cpu": "1","memory": "16Gi"
Proxy: "cpu": "100m","memory": "256Mi"

500 Group and Non-Openshift Projeccts..
5 Applicaitons in 5 Projects

Comment 8 errata-xmlrpc 2021-02-09 13:25:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.5.31 extras update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0315