Description of problem: Cannot get 'disk_caches' value by memoryStats(). Version-Release number of selected component (if applicable): python3-libvirt-5.0.0-2.module+el8+2780+89be6baf.x86_64 libvirt-5.0.0-3.module+el8+2807+8bbe0a41.x86_64 How reproducible 100% Steps to Reproduce: 1. Using virsh dommemstat command can get 'disk_caches' value. # virsh dommemstat test actual 2097152 swap_in 0 swap_out 0 major_fault 229 minor_fault 112754 unused 1724684 available 1873044 usable 1678040 last_update 1551160364 disk_caches 54976 rss 573904 2. Using memoryStats() API cann't get 'disk_caches' value. # python Python 3.6.8 (default, Jan 11 2019, 02:17:16) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import libvirt >>> conn = libvirt.open() >>> dom = conn.lookupByName('test') >>> dom.memoryStats() {'actual': 2097152, 'swap_in': 0, 'swap_out': 0, 'major_fault': 229, 'minor_fault': 112754, 'unused': 1724684, 'available': 1873044, 'usable': 1678040, 'last_update': 1551160364, 'rss': 602904} >>> exit() Actual results: Step 2, cann't get 'disk_caches' value. Expected results: Step 2, can get 'disk_caches' value. Additional info: Don't find VIR_DOMAIN_MEMORY_STAT_DISK_CACHES flag in libvirt-python-5.0.0/libvirt-override.c. static PyObject * libvirt_virDomainMemoryStats(PyObject *self ATTRIBUTE_UNUSED, PyObject *args) { ... for (i = 0; i < nr_stats; i++) { switch (stats[i].tag) { case VIR_DOMAIN_MEMORY_STAT_SWAP_IN: key = libvirt_constcharPtrWrap("swap_in"); break; case VIR_DOMAIN_MEMORY_STAT_SWAP_OUT: key = libvirt_constcharPtrWrap("swap_out"); break; case VIR_DOMAIN_MEMORY_STAT_MAJOR_FAULT: key = libvirt_constcharPtrWrap("major_fault"); break; case VIR_DOMAIN_MEMORY_STAT_MINOR_FAULT: key = libvirt_constcharPtrWrap("minor_fault"); break; case VIR_DOMAIN_MEMORY_STAT_UNUSED: key = libvirt_constcharPtrWrap("unused"); break; case VIR_DOMAIN_MEMORY_STAT_AVAILABLE: key = libvirt_constcharPtrWrap("available"); break; case VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON: key = libvirt_constcharPtrWrap("actual"); break; case VIR_DOMAIN_MEMORY_STAT_RSS: key = libvirt_constcharPtrWrap("rss"); break; #if LIBVIR_CHECK_VERSION(2, 1, 0) case VIR_DOMAIN_MEMORY_STAT_USABLE: key = libvirt_constcharPtrWrap("usable"); break; case VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE: key = libvirt_constcharPtrWrap("last_update"); break; #endif /* LIBVIR_CHECK_VERSION(2, 1, 0) */ default: continue; }
Hi Pavel, I'm testing new flags in python3-libvirt-5.6.0-1.module+el8.1.0+3891+3b51c436.x86_64. Two new flags [1] cann't get values by memoryStats(). It's similar to bug 1683516. But target release flag of bug 1683516 is 8.0. Should I open a new bug for this issue on RHEL 8.1-AV? [1] VIR_DOMAIN_MEMORY_STAT_HUGETLB_PGALLOC = 11 VIR_DOMAIN_MEMORY_STAT_HUGETLB_PGFAIL = 12 Test steps: 1. Using virsh command can get 'hugetlb_pgalloc' and 'hugetlb_pgfail'. # virsh dommemstat test actual 2097152 swap_in 0 swap_out 0 major_fault 197 minor_fault 135461 unused 1705192 available 1873288 usable 1665996 last_update 1565687219 disk_caches 68528 hugetlb_pgalloc 0 hugetlb_pgfail 0 rss 764436 2. Using memoryStats() cann't get 'hugetlb_pgalloc' and 'hugetlb_pgfail'. >>> dom.memoryStats() {'actual': 2097152, 'swap_in': 0, 'swap_out': 0, 'major_fault': 197, 'minor_fault': 135461, 'unused': 1705192, 'available': 1873288, 'usable': 1665996, 'last_update': 1565687219, 'rss': 764500} >>>
Hi Liping, No need to create a new BUG for this issue. It can be fixed as part of this BZ which I'll move to RHEL-AV. This BZ is not relevant to RHEL as all of the new memory stats where introduced after libvirt 4.5.0.
There are two additional stats missing, hugetlb_pgalloc and hugetlb_pgfail which should fixed as well because they were introduced in libvirt 5.4.0.
Upstream commits: commit d76c27e385ad1f6ed36e17dc7e882767240f69f4 Author: Pavel Hrdina <phrdina> Date: Fri Sep 20 11:16:53 2019 +0200 virDomainMemoryStats: include hugetlb pgalloc and pgfail commit b0a7747ddd6a2066beea3a6e1162c8f959516174 Author: Pavel Hrdina <phrdina> Date: Fri Sep 20 11:14:35 2019 +0200 virDomainMemoryStats: include disk caches
Verify with python3-libvirt-5.6.0-2.module+el8.1.0+4286+bc7b622c.x86_64. 1. Check 'disk_caches', 'hugetlb_pgalloc' and 'hugetlb_pgfail'. # python Python 3.6.8 (default, Aug 29 2019, 22:00:26) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import libvirt >>> conn = libvirt.open() >>> dom = conn.lookupByName('test') >>> dom.memoryStats() {'actual': 2097152, 'swap_in': 0, 'swap_out': 0, 'major_fault': 227, 'minor_fault': 127176, 'unused': 1718236, 'available': 1873288, 'usable': 1673088, 'last_update': 1569402804, 'disk_caches': 56700, 'hugetlb_pgalloc': 0, 'hugetlb_pgfail': 0, 'rss': 1027600} 2. Allocate hugepage to guest kernel. [guest] # echo 1000 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages [guest] # cat /proc/meminfo ... Buffers: 32 kB Cached: 61688 kB SwapCached: 376 kB ... HugePages_Total: 768 HugePages_Free: 768 ... 3. Check 'hugetlb_pgalloc' and 'hugetlb_pgfail' on host. >>> dom.memoryStats() {'actual': 2097152, 'swap_in': 312, 'swap_out': 19860, 'major_fault': 785, 'minor_fault': 874329, 'unused': 80304, 'available': 1873288, 'usable': 47856, 'last_update': 1569471940, 'disk_caches': 62100, 'hugetlb_pgalloc': 768, 'hugetlb_pgfail': 1, 'rss': 1043908} 4. Check 'disk_caches' value. disk_caches = Buffers + Cached + SwapCached
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3723