Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1158715

Summary:	A memory error report when use domstats
Product:	Red Hat Enterprise Linux 7	Reporter:	Luyao Huang <lhuang>
Component:	libvirt	Assignee:	Peter Krempa <pkrempa>
Status:	CLOSED ERRATA	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	7.1	CC:	dyuan, jmiao, mzhan, pkrempa, rbalakri
Target Milestone:	rc
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	libvirt-1.2.8-7.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-03-05 07:46:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Luyao Huang 2014-10-30 00:59:26 UTC

description of problem:
A memory error report when use domstats

Version-Release number of selected component (if applicable):
libvirt-1.2.8-5.el7.x86_64
qemu-kvm-rhev-2.1.2-4.el7.x86_64


How reproducible:
100%

Steps to Reproduce:

1.# valgrind -v --leak-check=full virsh domstats

Domain: 'test3'
  state.state=1
  state.reason=1
  cpu.time=1096447270349
  cpu.user=328510000000
  cpu.system=695060000000
  balloon.current=1048576
  balloon.maximum=1048576
  vcpu.current=1
  vcpu.maximum=2
  vcpu.0.state=1
  vcpu.0.time=19110000000
  net.count=1
  net.0.name=vnet0
  net.0.rx.bytes=196827
  net.0.rx.pkts=3746
  net.0.rx.errs=0
  net.0.rx.drop=0
  net.0.tx.bytes=3795
  net.0.tx.pkts=39
  net.0.tx.errs=0
  net.0.tx.drop=0
  block.count=1
  block.0.name=hda
  block.0.rd.reqs=45898
  block.0.rd.bytes=88999424
  block.0.rd.times=8177028688
  block.0.wr.reqs=779
  block.0.wr.bytes=4816896
  block.0.wr.times=1541997742
  block.0.fl.reqs=272
  block.0.fl.times=3521448382
  block.0.allocation=3221089792
  block.0.capacity=4294967296
  block.0.physical=4294967296

Domain: 'r6'
  state.state=5
  state.reason=2
  balloon.maximum=1048576
  vcpu.current=2
  vcpu.maximum=2
  vcpu.0.state=0
  vcpu.1.state=0


==30515== 
==30515== HEAP SUMMARY:
==30515==     in use at exit: 109,021 bytes in 1,093 blocks
==30515==   total heap usage: 5,680 allocs, 4,587 frees, 1,484,320 bytes allocated
==30515== 
==30515== Searching for pointers to 1,093 not-freed blocks
==30515== Checked 1,487,152 bytes
==30515== 
==30515== 40 bytes in 1 blocks are possibly lost in loss record 118 of 189
==30515==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30515==    by 0xBA21FDB: _PR_Getfd (prfdcach.c:112)
==30515==    by 0xBA3BCB6: pt_SetMethods.isra.11 (ptio.c:3303)
==30515==    by 0xBA3C2A4: PR_OpenFile (ptio.c:3581)
==30515==    by 0xD951989: blapi_SHVerifyFile (shvfy.c:355)
==30515==    by 0xD951CB0: blapi_SHVerify (shvfy.c:289)
==30515==    by 0xD927AE9: freebl_fipsSoftwareIntegrityTest (fipsfreebl.c:1541)
==30515==    by 0xD927AE9: bl_startup_tests (fipsfreebl.c:1732)
==30515==    by 0x400F502: call_init (dl-init.c:82)
==30515==    by 0x400F502: _dl_init (dl-init.c:131)
==30515==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==30515==    by 0x1: ???
==30515==    by 0xFFF0002EE: ???
==30515==    by 0xFFF0002F4: ???
==30515== 
==30515== 48 bytes in 1 blocks are possibly lost in loss record 123 of 189
==30515==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30515==    by 0xBA21FC9: _PR_Getfd (prfdcach.c:109)
==30515==    by 0xBA3BCB6: pt_SetMethods.isra.11 (ptio.c:3303)
==30515==    by 0xBA3C2A4: PR_OpenFile (ptio.c:3581)
==30515==    by 0xD951989: blapi_SHVerifyFile (shvfy.c:355)
==30515==    by 0xD951CB0: blapi_SHVerify (shvfy.c:289)
==30515==    by 0xD927AE9: freebl_fipsSoftwareIntegrityTest (fipsfreebl.c:1541)
==30515==    by 0xD927AE9: bl_startup_tests (fipsfreebl.c:1732)
==30515==    by 0x400F502: call_init (dl-init.c:82)
==30515==    by 0x400F502: _dl_init (dl-init.c:131)
==30515==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==30515==    by 0x1: ???
==30515==    by 0xFFF0002EE: ???
==30515==    by 0xFFF0002F4: ???
==30515== 
==30515== LEAK SUMMARY:
==30515==    definitely lost: 0 bytes in 0 blocks
==30515==    indirectly lost: 0 bytes in 0 blocks
==30515==      possibly lost: 88 bytes in 2 blocks
==30515==    still reachable: 108,933 bytes in 1,091 blocks
==30515==         suppressed: 0 bytes in 0 blocks
==30515== Reachable blocks (those to which a pointer was found) are not shown.
==30515== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==30515== 
==30515== Use --track-origins=yes to see where uninitialised values come from
==30515== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)
==30515== 
==30515== 1 errors in context 1 of 3:
==30515== Conditional jump or move depends on uninitialised value(s)
==30515==    at 0x85E9402: xdr_array (xdr_array.c:88)
==30515==    by 0x4FD8FC9: xdr_remote_connect_get_all_domain_stats_args (remote_protocol.c:6473)
==30515==    by 0x4FE72F2: virNetMessageEncodePayload (virnetmessage.c:350)
==30515==    by 0x4FDD21C: virNetClientProgramCall (virnetclientprogram.c:326)
==30515==    by 0x4FB4D01: callFull.isra.2 (remote_driver.c:6667)
==30515==    by 0x4FCBD45: call (remote_driver.c:6689)
==30515==    by 0x4FCBD45: remoteConnectGetAllDomainStats (remote_driver.c:7793)
==30515==    by 0x4FA0E75: virConnectGetAllDomainStats (libvirt.c:21678)
==30515==    by 0x147FD1: cmdDomstats (virsh-domain-monitor.c:2148)
==30515==    by 0x13006B: vshCommandRun (virsh.c:1915)
==30515==    by 0x12A9E1: main (virsh.c:3699)
==30515== 
--30515-- 
--30515-- used_suppression:      2 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a /usr/lib64/valgrind/default.supp:1296
==30515== 
==30515== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)





Actual results:

there is a memory error when use domstats,seems there is some problem 

Expected results:

no memory error


Additional info:

There is a upstream patch write by Jincheng Miao:

commit 28b7601dc7acf99d06277267afb63fff4167b755
Author: Jincheng Miao <jmiao>
Date:   Fri Oct 24 12:03:13 2014 +0800

    remote: fix jump depends on uninitialised value
    
    Currently remote driver only initializes partial fields of
    remote_connect_get_all_domain_stats_args. But xdr_array()
    will check the uninitialised field 'doms_val'.
    For safty reason, memset all fields of args is better.
    
    Fix the following error from valgrind, like:
    ==30515== 1 errors in context 1 of 3:
    ==30515== Conditional jump or move depends on uninitialised value(s)
    ==30515==    at 0x85E9402: xdr_array (xdr_array.c:88)
    ==30515==    by 0x4FD8FC9: xdr_remote_connect_get_all_domain_stats_args (remote_protocol.c:6473)
    ==30515==    by 0x4FE72F2: virNetMessageEncodePayload (virnetmessage.c:350)
    ==30515==    by 0x4FDD21C: virNetClientProgramCall (virnetclientprogram.c:326)
    ==30515==    by 0x4FB4D01: callFull.isra.2 (remote_driver.c:6667)
    ==30515==    by 0x4FCBD45: call (remote_driver.c:6689)
    ==30515==    by 0x4FCBD45: remoteConnectGetAllDomainStats (remote_driver.c:7793)
    ==30515==    by 0x4FA0E75: virConnectGetAllDomainStats (libvirt.c:21678)
    ==30515==    by 0x147FD1: cmdDomstats (virsh-domain-monitor.c:2148)
    ==30515==    by 0x13006B: vshCommandRun (virsh.c:1915)
    ==30515==    by 0x12A9E1: main (virsh.c:3699)
    
    Signed-off-by: Jincheng Miao <jmiao>

Comment 3 Luyao Huang 2014-11-05 09:40:26 UTC

A memory leak happen when with --domain and maybe forget free args:

# valgrind -v --leak-check=full virsh domstats test3 test4
==8372== 
==8372== HEAP SUMMARY:
==8372==     in use at exit: 109,145 bytes in 1,098 blocks
==8372==   total heap usage: 5,100 allocs, 4,002 frees, 1,575,606 bytes allocated
==8372== 
==8372== Searching for pointers to 1,098 not-freed blocks
==8372== Checked 1,487,240 bytes
==8372== 
==8372== 40 bytes in 1 blocks are possibly lost in loss record 119 of 192
==8372==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==8372==    by 0xBA23FDB: _PR_Getfd (prfdcach.c:112)
==8372==    by 0xBA3DCB6: pt_SetMethods.isra.11 (ptio.c:3303)
==8372==    by 0xBA3E2A4: PR_OpenFile (ptio.c:3581)
==8372==    by 0xD953989: blapi_SHVerifyFile (shvfy.c:355)
==8372==    by 0xD953CB0: blapi_SHVerify (shvfy.c:289)
==8372==    by 0xD929AE9: freebl_fipsSoftwareIntegrityTest (fipsfreebl.c:1541)
==8372==    by 0xD929AE9: bl_startup_tests (fipsfreebl.c:1732)
==8372==    by 0x400F502: call_init (dl-init.c:82)
==8372==    by 0x400F502: _dl_init (dl-init.c:131)
==8372==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==8372==    by 0x3: ???
==8372==    by 0xFFF0002DE: ???
==8372==    by 0xFFF0002E4: ???
==8372== 
==8372== 48 bytes in 1 blocks are possibly lost in loss record 125 of 192
==8372==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==8372==    by 0xBA23FC9: _PR_Getfd (prfdcach.c:109)
==8372==    by 0xBA3DCB6: pt_SetMethods.isra.11 (ptio.c:3303)
==8372==    by 0xBA3E2A4: PR_OpenFile (ptio.c:3581)
==8372==    by 0xD953989: blapi_SHVerifyFile (shvfy.c:355)
==8372==    by 0xD953CB0: blapi_SHVerify (shvfy.c:289)
==8372==    by 0xD929AE9: freebl_fipsSoftwareIntegrityTest (fipsfreebl.c:1541)
==8372==    by 0xD929AE9: bl_startup_tests (fipsfreebl.c:1732)
==8372==    by 0x400F502: call_init (dl-init.c:82)
==8372==    by 0x400F502: _dl_init (dl-init.c:131)
==8372==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==8372==    by 0x3: ???
==8372==    by 0xFFF0002DE: ???
==8372==    by 0xFFF0002E4: ???
==8372== 
==8372== 64 bytes in 1 blocks are definitely lost in loss record 134 of 192
==8372==    at 0x4C2B934: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==8372==    by 0x4E922EC: virAllocN (viralloc.c:191)
==8372==    by 0x4FCD2BB: remoteConnectGetAllDomainStats (remote_driver.c:7781)
==8372==    by 0x4FA26B5: virDomainListGetStats (libvirt.c:21776)
==8372==    by 0x147ED7: cmdDomstats (virsh-domain-monitor.c:2142)
==8372==    by 0x13006B: vshCommandRun (virsh.c:1915)
==8372==    by 0x12A9E1: main (virsh.c:3699)
==8372== 
==8372== LEAK SUMMARY:
==8372==    definitely lost: 64 bytes in 1 blocks
==8372==    indirectly lost: 0 bytes in 0 blocks
==8372==      possibly lost: 88 bytes in 2 blocks
==8372==    still reachable: 108,993 bytes in 1,095 blocks
==8372==         suppressed: 0 bytes in 0 blocks
==8372== Reachable blocks (those to which a pointer was found) are not shown.
==8372== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==8372== 
==8372== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)
--8372-- 
--8372-- used_suppression:      2 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a /usr/lib64/valgrind/default.supp:1296
==8372== 
==8372== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)

Comment 4 Peter Krempa 2014-11-05 15:30:03 UTC

The issue is now fixed upstream:

commit bf1f8e280c330f51833189d40a99dc894205a129
Author: Peter Krempa <pkrempa>
Date:   Wed Nov 5 12:34:51 2014 +0100

    remote: Fix memory leak in remoteConnectGetAllDomainStats
    
    The remote call actually doesn't free the arguments array so we leak
    memory in case a domain list is specified. As the remote domain list
    array consists only of stolen pointers from the actual domain objects
    it's sufficient just to free the array.
    
    Valgrind message:
    ==1081452== 64 bytes in 1 blocks are definitely lost in loss record 632 of 726
    ==1081452==    at 0x4C296D0: calloc (vg_replace_malloc.c:618)
    ==1081452==    by 0x4EA5CB4: virAllocN (viralloc.c:191)
    ==1081452==    by 0x505D21E: remoteConnectGetAllDomainStats (remote_driver.c:7785)
    ==1081452==    by 0x50081AA: virDomainListGetStats (libvirt-domain.c:11080)
    ==1081452==    by 0x155249: cmdDomstats (virsh-domain-monitor.c:2147)
    ==1081452==    by 0x12FB73: vshCommandRun (virsh.c:1935)
    ==1081452==    by 0x133FEB: main (virsh.c:3719)

Comment 6 Luyao Huang 2014-11-18 02:11:15 UTC

Verify this bug with libvirt-1.2.8-7.el7:

# valgrind -v --leak-check=full virsh domstats test3 test4

==3839== 
==3839== LEAK SUMMARY:
==3839==    definitely lost: 0 bytes in 0 blocks
==3839==    indirectly lost: 0 bytes in 0 blocks
==3839==      possibly lost: 88 bytes in 2 blocks
==3839==    still reachable: 108,993 bytes in 1,095 blocks
==3839==         suppressed: 0 bytes in 0 blocks
==3839== Reachable blocks (those to which a pointer was found) are not shown.
==3839== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==3839== 

# valgrind -v --leak-check=full virsh domstats --list-active --block

==4124== 
==4124== LEAK SUMMARY:
==4124==    definitely lost: 0 bytes in 0 blocks
==4124==    indirectly lost: 0 bytes in 0 blocks
==4124==      possibly lost: 88 bytes in 2 blocks
==4124==    still reachable: 108,981 bytes in 1,093 blocks
==4124==         suppressed: 0 bytes in 0 blocks
==4124== Reachable blocks (those to which a pointer was found) are not shown.
==4124== To see them, rerun with: --leak-check=full --show-leak-kinds=all


And do some extend test, no memory leak found when use domstats

Comment 8 errata-xmlrpc 2015-03-05 07:46:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html