Description of problem: Cloud Intel > Reports is never accessible. Version-Release number of selected component (if applicable): 5.10.4.2 How reproducible: Everytime at customer environment Steps to Reproduce: 1. restore Cu db provided in comments / access the reproducer 2. try to navigate to Cloud Intel > reports 3. Actual results: page keeps loading and after sometime popup with below error is seen Status 503 Service Unavailable (screenshot attached) Expected results: Page should load fine Additional info: I can access other pages such as Compute > Infrastructure > Providers Info provided by Cu - A week ago we deleted a infrastructure provider and after few day we attach the same provider, this is the only activity we have done.
Created attachment 1585661 [details] error screenshot as seen in the reproducer
Yuri, your suggestion has helped resolve the issue, thanks for you help. BZ can be closed.
Hey Niladri! Could I please get you to check using the db attached to this ticket if it still gets a 503 on the latest 5.11?
Yuri confirmed that it doesn't hit the 503 on latest master. But it's still slow so I'm still looking into it.
It looks like we're running https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/presenters/tree_builder.rb#L206 for a whole bunch of things that we don't need to, but it's generic code, and so I expect that it's going to be difficult to stop running it so many times for this specific case. log/evm.log:[----] I, [2019-07-16T20:04:03.336835 #4422:3fdac8e38e64] INFO -- : XXXXXHERE1111 with {:id=>"1000000000115", :text=>"VMs with Avg Daily CPU > 85% (past mo.)", :icon=>"fa fa-file-text-o", :tip=>"VMs with Avg Daily CPU > 85% (past mo.)"} => #<MiqReport id: 1000000000115, name: "VMs with Avg Daily CPU > 85% (past mo.)", title: "VMs with Average Daily CPU Usage > 85% over last m...", rpt_group: "Performance by Asset Type - Virtual Machines", rpt_type: "Default", priority: 120, db: "VmPerformance", cols: ["resource_name", "v_date", "cpu_usage_rate_average", "abs_max_cpu_usage_rate_average_value", "max_cpu_usage_rate_average", "derived_cpu_available"], include: {"host"=>{"columns"=>["hostname"]}, "ems_cluster"=>{"columns"=>["name"]}}, col_order: ["resource_name", "host.hostname", "ems_cluster.name", "v_date", "cpu_usage_rate_average", "abs_max_cpu_usage_rate_average_value", "max_cpu_usage_rate_average", "derived_cpu_available"], headers: ["VM Name", "Host Hostname", "Cluster Name", "Activity Sample", "CPU - Usage Rate for Collected Intervals (%)", "CPU - Absolute Max Usage Rate (%)", "CPU - Peak Usage Rate Avg for Collected Intervals (%)", "CPU - Total Available - from VM Analysis (MHz)"], conditions: #<MiqExpression:0x00007fb35f105da0 @exp={"and"=>[{"IS NOT EMPTY"=>{"field"=>"VmPerformance-sys_uptime_absolute_latest", "value"=>""}}, {">"=>{"field"=>"VmPerformance-max_cpu_usage_rate_average", "value"=>85}}]}>, order: "Descending", sortby: ["resource_name", "cpu_usage_rate_average"], group: "c", graph: nil, dims: nil, created_on: "2009-05-15 14:37:01", updated_on: "2019-07-16 15:19:42", filename: "650_Performance by Asset Type - Virtual Machines/1...", file_mtime: "2018-09-19 02:19:19", categories: nil, timeline: nil, template_type: "report", where_clause: nil, db_options: {:start_offset=>2592000, :end_offset=>0, :interval=>"daily"}, generate_cols: nil, generate_rows: nil, col_formats: nil, tz: nil, time_profile_id: nil, display_filter: nil, col_options: nil, rpt_options: nil, miq_group_id: nil, user_id: nil> I think Yuri's point was that for this screen we don't need to be running the specific miq reports, and I'm seeing all of them loading, which, as he said, is why this is so slow. That line of code also gets hit with a whole bunch of things on that screen, we load, among a bunch of other stuff, MiqGroups. I don't know if that's necessary, but I'm not particularly familiar with this area of the code. log/evm.log:[----] I, [2019-07-16T20:03:55.862141 #4422:3fdac8e38e64] INFO -- : XXXXXHERE1111 with #<MiqGroup:0x00007fb5904a4b90> I know Keenan did refactoring on some of this a while ago to make it faster. It'd be cool to get a UI person to look at this and see if any of it could be improved.
Just a bit more: it looks like https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/controllers/mixins/breadcrumbs_mixin.rb#L129 is where we determine what to load, and we get [#<struct ApplicationController::Feature role="miq_report_saved_reports", role_any=true, name=:savedreports, title="Saved Reports">, #<struct ApplicationController::Feature role="miq_report_reports", role_any=true, name=:reports, title="Reports">, #<struct ApplicationController::Feature role="miq_report_schedules", role_any=true, name=:schedules, title="Schedules">, #<struct ApplicationController::Feature role="miq_report_dashboard_editor", role_any=nil, name=:db, title="Dashboards">, #<struct ApplicationController::Feature role="miq_report_widget_editor", role_any=nil, name=:widgets, title="Dashboard Widgets">, #<struct ApplicationController::Feature role="miq_report_menu_editor", role_any=nil, name=:roles, title="Edit Report Menus">, #<struct ApplicationController::Feature role="miq_report_export", role_any=nil, name=:export, title="Import/Export">] which includes ReportReports.
One more: it looks like all the features for reports get set here: https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/controllers/report_controller.rb#L289
But, it's not as simple as just removing elements from that list, because then the tree won't get built correctly.
just got off chat with David, Yaml file of interest is: MiqReportResult-all.yaml: | column | virtual | sql | sort | hidden | cond | |:----------------------|:--------|:-----|:-----|:---------|:-----| | id | | sql | | | | | created_on | | sql | sort | | | | last_run_on | | sql | | | | | name | | sql | | | | | report_source | | sql | | | | | userid | | sql | | | | | miq_group_description | attr | sql | | | | | status | attr | ruby | | | | | miq_group_id | | sql | | sql only | | | miq_report_id | | sql | | sql only | | | miq_task_id | | sql | | sql only | | The virtual attribute of interest is status It depends upon miq_task and report_result. report_result looks up the blob we'll look into other ways of checking whether report_results.blob is empty (to know if there was an error
https://github.com/ManageIQ/manageiq/pull/19082
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/257c561a6714be689147bd698441f2bdffa389dc commit 257c561a6714be689147bd698441f2bdffa389dc Author: Dávid Halász <dhalasz> AuthorDate: Wed Jul 31 16:43:44 2019 -0400 Commit: Dávid Halász <dhalasz> CommitDate: Wed Jul 31 16:43:44 2019 -0400 Don't load BinaryBlobParts when determining if MiqReportResult is blank Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1725142 app/models/miq_report_result.rb | 11 +- spec/models/miq_report_result_spec.rb | 19 + 2 files changed, 29 insertions(+), 1 deletion(-)
New commit detected on ManageIQ/manageiq/ivanchuk: https://github.com/ManageIQ/manageiq/commit/701ab300efb23cdc57ceb0452d3b8035cfa1000f commit 701ab300efb23cdc57ceb0452d3b8035cfa1000f Author: Keenan Brock <keenan> AuthorDate: Thu Aug 1 15:42:28 2019 -0400 Commit: Keenan Brock <keenan> CommitDate: Thu Aug 1 15:42:28 2019 -0400 Merge pull request #19082 from skateman/report-result-blob-blank Don't load BinaryBlobParts when determining if MiqReportResult is blank (cherry picked from commit 421d105b6c8b851f315c72608ca89250a5052b4a) https://bugzilla.redhat.com/show_bug.cgi?id=1725142 app/models/miq_report_result.rb | 11 +- spec/models/miq_report_result_spec.rb | 19 + 2 files changed, 29 insertions(+), 1 deletion(-)
FIXED. Verified on 5.11.0.22.20190827200559_e618ece. Used the steps mentioned here (https://bugzilla.redhat.com/show_bug.cgi?id=1726467#c7) to use 510 DB on 511 appliance and reproduce the environment.