Bug 1725142

Created attachment 1585661 [details]
error screenshot as seen in the reproducer

Yuri,

your suggestion has helped resolve the issue, thanks for you help.
BZ can be closed.

Hey Niladri! Could I please get you to check using the db attached to this ticket if it still gets a 503 on the latest 5.11?

Yuri confirmed that it doesn't hit the 503 on latest master. But it's still slow so I'm still looking into it.

It looks like we're running https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/presenters/tree_builder.rb#L206 for a whole bunch of things that we don't need to, but it's generic code, and so I expect that it's going to be difficult to stop running it so many times for this specific case. 

log/evm.log:[----] I, [2019-07-16T20:04:03.336835 #4422:3fdac8e38e64]  INFO -- : XXXXXHERE1111 with {:id=>"1000000000115", :text=>"VMs with Avg Daily CPU > 85% (past mo.)", :icon=>"fa fa-file-text-o", :tip=>"VMs with Avg Daily CPU > 85% (past mo.)"}

 => #<MiqReport id: 1000000000115, name: "VMs with Avg Daily CPU > 85% (past mo.)", title: "VMs with Average Daily CPU Usage > 85% over last m...", rpt_group: "Performance by Asset Type - Virtual Machines", rpt_type: "Default", priority: 120, db: "VmPerformance", cols: ["resource_name", "v_date", "cpu_usage_rate_average", "abs_max_cpu_usage_rate_average_value", "max_cpu_usage_rate_average", "derived_cpu_available"], include: {"host"=>{"columns"=>["hostname"]}, "ems_cluster"=>{"columns"=>["name"]}}, col_order: ["resource_name", "host.hostname", "ems_cluster.name", "v_date", "cpu_usage_rate_average", "abs_max_cpu_usage_rate_average_value", "max_cpu_usage_rate_average", "derived_cpu_available"], headers: ["VM Name", "Host Hostname", "Cluster Name", "Activity Sample", "CPU - Usage Rate for Collected Intervals (%)", "CPU - Absolute Max Usage Rate (%)", "CPU - Peak Usage Rate Avg for Collected Intervals (%)", "CPU - Total Available - from VM Analysis (MHz)"], conditions: #<MiqExpression:0x00007fb35f105da0 @exp={"and"=>[{"IS NOT EMPTY"=>{"field"=>"VmPerformance-sys_uptime_absolute_latest", "value"=>""}}, {">"=>{"field"=>"VmPerformance-max_cpu_usage_rate_average", "value"=>85}}]}>, order: "Descending", sortby: ["resource_name", "cpu_usage_rate_average"], group: "c", graph: nil, dims: nil, created_on: "2009-05-15 14:37:01", updated_on: "2019-07-16 15:19:42", filename: "650_Performance by Asset Type - Virtual Machines/1...", file_mtime: "2018-09-19 02:19:19", categories: nil, timeline: nil, template_type: "report", where_clause: nil, db_options: {:start_offset=>2592000, :end_offset=>0, :interval=>"daily"}, generate_cols: nil, generate_rows: nil, col_formats: nil, tz: nil, time_profile_id: nil, display_filter: nil, col_options: nil, rpt_options: nil, miq_group_id: nil, user_id: nil> 

I think Yuri's point was that for this screen we don't need to be running the specific miq reports, and I'm seeing all of them loading, which, as he said, is why this is so slow. 

That line of code also gets hit with a whole bunch of things on that screen, we load, among a bunch of other stuff, MiqGroups. I don't know if that's necessary, but I'm not particularly familiar with this area of the code. 
log/evm.log:[----] I, [2019-07-16T20:03:55.862141 #4422:3fdac8e38e64]  INFO -- : XXXXXHERE1111 with #<MiqGroup:0x00007fb5904a4b90>

I know Keenan did refactoring on some of this a while ago to make it faster. It'd be cool to get a UI person to look at this and see if any of it could be improved.

Just a bit more: it looks like https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/controllers/mixins/breadcrumbs_mixin.rb#L129 is where we determine what to load, and we get 

[#<struct ApplicationController::Feature role="miq_report_saved_reports", role_any=true, name=:savedreports, title="Saved Reports">, #<struct ApplicationController::Feature role="miq_report_reports", role_any=true, name=:reports, title="Reports">, #<struct ApplicationController::Feature role="miq_report_schedules", role_any=true, name=:schedules, title="Schedules">, #<struct ApplicationController::Feature role="miq_report_dashboard_editor", role_any=nil, name=:db, title="Dashboards">, #<struct ApplicationController::Feature role="miq_report_widget_editor", role_any=nil, name=:widgets, title="Dashboard Widgets">, #<struct ApplicationController::Feature role="miq_report_menu_editor", role_any=nil, name=:roles, title="Edit Report Menus">, #<struct ApplicationController::Feature role="miq_report_export", role_any=nil, name=:export, title="Import/Export">]

which includes ReportReports.

One more: it looks like all the features for reports get set here: https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/controllers/report_controller.rb#L289

But, it's not as simple as just removing elements from that list, because then the tree won't get built correctly.

just got off chat with David,

Yaml file of interest is: MiqReportResult-all.yaml:

| column                | virtual | sql  | sort | hidden   | cond | 
|:----------------------|:--------|:-----|:-----|:---------|:-----|
| id                    |         | sql  |      |          |      | 
| created_on            |         | sql  | sort |          |      | 
| last_run_on           |         | sql  |      |          |      | 
| name                  |         | sql  |      |          |      | 
| report_source         |         | sql  |      |          |      | 
| userid                |         | sql  |      |          |      | 
| miq_group_description | attr    | sql  |      |          |      | 
| status                | attr    | ruby |      |          |      | 
| miq_group_id          |         | sql  |      | sql only |      | 
| miq_report_id         |         | sql  |      | sql only |      | 
| miq_task_id           |         | sql  |      | sql only |      | 

The virtual attribute of interest is status

It depends upon miq_task and report_result. report_result looks up the blob

we'll look into other ways of checking whether report_results.blob is empty (to know if there was an error

https://github.com/ManageIQ/manageiq/pull/19082

New commit detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/257c561a6714be689147bd698441f2bdffa389dc
commit 257c561a6714be689147bd698441f2bdffa389dc
Author:     Dávid Halász <dhalasz>
AuthorDate: Wed Jul 31 16:43:44 2019 -0400
Commit:     Dávid Halász <dhalasz>
CommitDate: Wed Jul 31 16:43:44 2019 -0400

    Don't load BinaryBlobParts when determining if MiqReportResult is blank

    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1725142

 app/models/miq_report_result.rb | 11 +-
 spec/models/miq_report_result_spec.rb | 19 +
 2 files changed, 29 insertions(+), 1 deletion(-)

New commit detected on ManageIQ/manageiq/ivanchuk:

https://github.com/ManageIQ/manageiq/commit/701ab300efb23cdc57ceb0452d3b8035cfa1000f
commit 701ab300efb23cdc57ceb0452d3b8035cfa1000f
Author:     Keenan Brock <keenan>
AuthorDate: Thu Aug  1 15:42:28 2019 -0400
Commit:     Keenan Brock <keenan>
CommitDate: Thu Aug  1 15:42:28 2019 -0400

    Merge pull request #19082 from skateman/report-result-blob-blank

    Don't load BinaryBlobParts when determining if MiqReportResult is blank

    (cherry picked from commit 421d105b6c8b851f315c72608ca89250a5052b4a)

    https://bugzilla.redhat.com/show_bug.cgi?id=1725142

 app/models/miq_report_result.rb | 11 +-
 spec/models/miq_report_result_spec.rb | 19 +
 2 files changed, 29 insertions(+), 1 deletion(-)

FIXED. Verified on 5.11.0.22.20190827200559_e618ece.
Used the steps mentioned here (https://bugzilla.redhat.com/show_bug.cgi?id=1726467#c7) to use 510 DB on 511 appliance and reproduce the environment.