Bug 1552049

Summary:	Container reports take too much time to generate
Product:	Red Hat CloudForms Management Engine	Reporter:	Niladri Roy <niroy>
Component:	Performance	Assignee:	Nick LaMuro <nlamuro>
Status:	CLOSED CURRENTRELEASE	QA Contact:	juwatts
Severity:	high	Docs Contact:
Priority:	high
Version:	5.8.0	CC:	cpelland, ktenzer, nlamuro, obarenbo
Target Milestone:	GA	Keywords:	TestOnly, ZStream
Target Release:	5.10.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	5.10.0.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1565677 1565678 (view as bug list)		Environment:
Last Closed:	2018-07-30 14:44:27 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	CFME Core	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1565677, 1565678

Description Niladri Roy 2018-03-06 12:07:01 UTC

Description of problem:
Custom reports on OpenShift images take too much time to run, about 25-30 mins

Version-Release number of selected component (if applicable):
5.8.3.4

How reproducible:
everytime at customers environment

Steps to Reproduce:
1. Cloud Intel> Reports > reports accordion> Custom 
2. try to run report named "9 - OpenSCAP: Non-Compliant Containers"
3. try to run other two custom reports

Actual results:
Report gets stuck at generating

Expected results:
report should be generated with minimum delay

Additional info:
Initially Cu was using 5.8.1.5 and there was memory constraint on the appliance and workers
Cu has updated to 5.8.3.4 now and as of the current logs no memory issue can be seen with the workers

Comment 19 Nick LaMuro 2018-04-09 21:34:25 UTC

The fixes for this have been merged into master just now and should be backported to the respective releases soon.

The fixes are:

https://github.com/ManageIQ/manageiq/pull/17141 => This fixes a very real possibility for a "LEFT JOIN BOMB" to happen in RBAC, with the data set that I was fixing this for causing a query that was larger than 40Gigs for the result set.  The fix should attempt to avoid the case where this is possible and will fallback to working when the query to the database that is generated is invalid.  This fix was tested against the original issuer's data set, and kept the query dataset size quiet small as a result.

https://github.com/ManageIQ/manageiq/pull/17195 => While not the original issue I was working on, this also drastically improved the speed at which the report was generated (about 3x faster with both fixes in place).  This simply avoids a bunch of duplicate queries to determine column types for rows in a report, and nothing should change otherwise as a result.

-Nick

Comment 24 Dave Johnson 2018-07-30 14:44:27 UTC

Closing this as its already been verified in two z-streams and has test coverage around it.