Bug 2218211 - ceilometer_memory_usage fails to report metrics in case where VMs disks are stored at ceph [NEEDINFO]
Summary: ceilometer_memory_usage fails to report metrics in case where VMs disks are s...
Keywords:
Status: ASSIGNED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ceilometer
Version: 17.1 (Wallaby)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z2
: 17.1
Assignee: Yadnesh Kulkarni
QA Contact: Leonid Natapov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-28 13:08 UTC by Leonid Natapov
Modified: 2023-08-11 13:59 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
ifrangs: needinfo? (ykulkarn)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker OSP-26200 0 None None None 2023-06-28 13:12:02 UTC

Description Leonid Natapov 2023-06-28 13:08:44 UTC
ceilometer_memory_usage fails to report metrics in case where VMs disks are stored at ceph

I have an OSPdO running OSP 16.2 in HCI env. 
I have noticed that there is  no data in VM Memory Utilization panel in Grafana.

After checking compute.log in ceilometer directory on compute nodes I see the following message


ceilometer_memory_usage shows no data
6:56
2023-06-28 03:33:36.063 15 WARNING ceilometer.compute.pollsters [-] memory.usage statistic in not available for instance 93366f22-da75-4df1-bb64-ef5710d123a7: ceilometer.compute.pollsters.NoVolumeException

Comment 4 Yadnesh Kulkarni 2023-07-03 14:17:58 UTC
Looks like libvirt itself cannot expose the memory statistics of an instance in an HCI env
~~~
[root@computehci-1 /]# virsh list
 Id   Name                State
-----------------------------------
 1    instance-00000001   running
 2    instance-00000003   running

[root@computehci-1 /]# virsh dommemstat 1                                                                                                                                                                                              
actual 524288
last_update 0
rss 228372

[root@computehci-1 /]# virsh dommemstat 2
actual 524288
last_update 0
rss 209964
~~~

From a compute node of non-HCI env but nova uses ceph as a backend
~~~
[root@compute-0 /]# virsh list
 Id   Name                State
-----------------------------------
 1    instance-00000001   running

[root@compute-0 /]# virsh dommemstat instance-00000001
actual 524288
swap_in 0
swap_out 0
major_fault 0
minor_fault 84757
unused 402840
available 477392
usable 431064
last_update 1688384922
disk_caches 26528
hugetlb_pgalloc 0
hugetlb_pgfail 0
rss 353636
~~~

Ceilometer uses these readings to generate memory usage samples [1] which are missing in HCI env.

We must analyze this behaviour in OSP17 as well.

[1] https://github.com/openstack/ceilometer/blob/stable/train/ceilometer/compute/virt/libvirt/inspector.py#L171-L185


Note You need to log in before you can comment on or make changes to this bug.