Bug 1671580

Summary: [Regression] Unable to capture metrics from Azure
Product: Red Hat CloudForms Management Engine Reporter: Nandini Chandra <nachandr>
Component: C&U Capacity and UtilizationAssignee: Adam Grare <agrare>
Status: CLOSED CURRENTRELEASE QA Contact: Nandini Chandra <nachandr>
Severity: high Docs Contact: Red Hat CloudForms Documentation <cloudforms-docs>
Priority: high    
Version: 5.10.0CC: bsorota, dmetzger, jdupuy, jprause, niroy, obarenbo, simaishi, tpapaioa
Target Milestone: GAKeywords: TestOnly, ZStream
Target Release: 5.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.11.0.11 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1722937 (view as bug list) Environment:
Last Closed: 2019-12-13 14:55:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Azure Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1704905, 1722937    

Description Nandini Chandra 2019-01-31 23:42:39 UTC
Description of problem:
-----------------------
Metrics are not being captured from Azure.


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
5.9, 5.10


How reproducible:
----------------
Always


Steps to Reproduce:
------------------
1.
2.
3.


Actual results:
---------------
Unable to capture metrics from Azure


Expected results:
----------------
It should be possible to capture metrics from Azure.


Additional info:
---------------

Comment 2 Nandini Chandra 2019-01-31 23:43:42 UTC
It looks like MS is sending different metric names and we would have to switch over to the new names.

https://github.com/ManageIQ/manageiq-providers-azure/issues/314

Comment 3 Nandini Chandra 2019-02-12 18:00:04 UTC
Alexander,

Please let me know if this is being targeted for 5.10.1.

Thanks,
Nandini

Comment 4 Alexander Zagaynov 2019-02-25 14:22:25 UTC
I guess it's too late for 5.10.1

Comment 5 Nandini Chandra 2019-02-28 16:44:16 UTC
*** Bug 1679428 has been marked as a duplicate of this bug. ***

Comment 10 Alexander Zagaynov 2019-06-20 20:16:31 UTC
Root reason of this BZ is that Microsoft doesn't serve "2014-04-01" api version anymore, and collection code where relying on it hard (https://github.com/ManageIQ/azure-armrest/blob/master/lib/azure/armrest/insights/metrics_service.rb#L129).
Right now I can find on the Microsoft's site documentation for the latest version only: https://docs.microsoft.com/en-us/rest/api/monitor/metrics/list
But it doesn't have data, which is needed to gather "memory" metrics (https://github.com/ManageIQ/manageiq-providers-azure/blob/124ab858422fa249acf2e12c6f47d15eefd682bf/app/models/manageiq/providers/azure/cloud_manager/metrics_capture.rb#L152)
So, I had to use trial-and-error method to find replacement in other available api versions.

Problem is with the "memory" metric only, others are available through modern API easily.
I propose to do release without "memory" right now, because I need more time to research for alternative ways to get it.
Everything else (CPU, Disk, Network) is working.

Comment 14 John Dupuy 2019-07-02 15:40:54 UTC
I am able to verify that cap and util data is collected for Azure in CFME 5.11.0.11 via enabling the Cap and Util roles and looking at the utilization data for a VM ("Monitoring -> Utilization" from a VM details page). However is this all that needs to be done for verification? Is there any (better) way to verify that metrics are captured for azure?

Comment 15 John Dupuy 2019-07-02 16:09:57 UTC
After discussing with nachandr, this appears to be sufficient for verifying the BZ. I am going to mark this as verified.

Comment 17 Alexander Zagaynov 2019-07-08 11:58:58 UTC
To do some load and make metrics observable, you can use following commands (on linux):
- Disk load: `sudo dd if=/dev/urandom of=/mnt/data/test/something bs=1GiB count=2 iflag=fullblock status=progress`
- Memory load: `let size=1024**3*2; export BIG=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w $size - | head -n 1)`
- Network load: `wget https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_30mb.mkv`
- CPU load: same as memory or disk load.