Bug 1575963

Summary: VM Utilization report skips dates though data is present in db
Product: Red Hat CloudForms Management Engine Reporter: Niladri Roy <niroy>
Component: ReportingAssignee: Gregg Tanzillo <gtanzill>
Status: CLOSED NOTABUG QA Contact: Nandini Chandra <nachandr>
Severity: high Docs Contact:
Priority: high    
Version: 5.7.0CC: cpelland, nachandr, niroy, obarenbo, yrudman
Target Milestone: GA   
Target Release: 5.8.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-09 19:10:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:

Description Niladri Roy 2018-05-08 12:19:26 UTC
Description of problem:
VM utilization report for 2 weeks doesn't show the data for all consecutive dates, data is missing for random dates

Version-Release number of selected component (if applicable):
5.7.3.2

How reproducible:
Everytime at Cu environment

Steps to Reproduce:
1. Navigate to Cloud Intel>Reports>UIDAI- VM Utilization Report
2. Queue the report or check the save reports
3.

Actual results:
From a period of 24 Apr to 8 May (2 weeks), many dates are missing

Expected results:
Report shouldn't skip any date

Additional info:
RHV provider

Comment 3 Gregg Tanzillo 2018-05-08 12:42:43 UTC
Ned to verify -
1. That c&U data exists for the missing periods.
2. If the data is missing, check the queue to see if C&U collection and processing is not backed up

Comment 4 Niladri Roy 2018-05-08 13:03:25 UTC
Checked that data exists

Comment 6 Gregg Tanzillo 2018-05-08 14:02:56 UTC
Querying daily C&U for that VM in that time profile shows that there is C&U data for the missing days-

vm=Vm.where(:name => "UIDAI_DB_Server_02").take
pp vm.metric_rollups.where(:capture_interval_name=>"daily").where(:time_profile => TimeProfile.find_by_description("IST")).where(["timestamp > ?",16.days.ago.utc]).map(&:timestamp).uniq.sort
[Sun, 22 Apr 2018 18:30:00 UTC +00:00,
 Mon, 23 Apr 2018 18:30:00 UTC +00:00,
 Tue, 24 Apr 2018 18:30:00 UTC +00:00,
 Wed, 25 Apr 2018 18:30:00 UTC +00:00,
 Thu, 26 Apr 2018 18:30:00 UTC +00:00,
 Fri, 27 Apr 2018 18:30:00 UTC +00:00,
 Sat, 28 Apr 2018 18:30:00 UTC +00:00,
 Sun, 29 Apr 2018 18:30:00 UTC +00:00,
 Mon, 30 Apr 2018 18:30:00 UTC +00:00,
 Tue, 01 May 2018 18:30:00 UTC +00:00,
 Wed, 02 May 2018 18:30:00 UTC +00:00,
 Thu, 03 May 2018 18:30:00 UTC +00:00,
 Fri, 04 May 2018 18:30:00 UTC +00:00,
 Sat, 05 May 2018 18:30:00 UTC +00:00,
 Sun, 06 May 2018 18:30:00 UTC +00:00]

Further investigation is needed to determine why the report is not picking up the data for those days

Comment 7 Yuri Rudman 2018-05-08 14:08:49 UTC
Niladry, do you know when this report stop working correctly, any recent changes ?

Comment 9 Yuri Rudman 2018-05-08 15:32:42 UTC
"UIDAI - VM Utilization Report" is custom report created at 2018-04-13 06:06:22.159316.
Was it working correctly originally or did not work properly from the beginning ?

Comment 11 Gregg Tanzillo 2018-05-09 19:10:39 UTC
After downloading the DB locally and inspecting the report we discovered the the report includes 3 columns for consolidation -

"Number of CPU Cores Allocated"
"Power State"
"CPU - Usage Average Intervals (%)"

Rows that have the same values in those columns (we determined that there are, in fact matching rows) will be merged together into a single row. This means that the rows for the days that appear to be missing on the report were merged into a single row. With regard to the randomness of the missing dates, that is explained by the fact that there are some days that match values in the above columns and some that don't. The days that don't match will always be visible as a separate row on the report. This is all working as designed.

If the consolidation is removed from the report, all dates in the range are shown.

We saw the same consolidation on several other reports. So it appears that this particular report configuration was intentional.