Bug 1067338

Summary: Mem leak while start a guest with a character followed
Product: Red Hat Enterprise Linux 7 Reporter: zhenfeng wang <zhwang>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: ajia, dyuan, gsun, jmiao, jtomko, juzhou, mzhan, rbalakri, vivianzhang, ydu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.2.7-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 07:30:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zhenfeng wang 2014-02-20 09:22:47 UTC
Description of problem:
Mem leak while start a guest with a character followed

Version-Release number of selected component (if applicable):
kernel-3.10.0-88.el7.x86_64
qemu-kvm-rhev-1.5.3-47.el7.x86_64
libvirt-1.1.1-23.el7.x86_64
libselinux-2.2.2-6.el7.x86_64
selinux-policy-3.12.1-125.el7.noarch
How reproducible:
100%

Steps
1.Prepare a normal guest
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     rhel75                         shut off

2.Start a guest with a character followed
#valgrind -v --leak-check=full virsh start rhel75 a
--
error: Unable to parse FD number 'a'

error: One or more references were leaked after disconnect from the hypervisor
==24569==
==24569== HEAP SUMMARY:
==24569==     in use at exit: 559,891 bytes in 2,302 blocks
==24569==   total heap usage: 6,279 allocs, 3,977 frees, 1,252,525 bytes allocated
==24569==
==24569== Searching for pointers to 2,302 not-freed blocks
==24569== Checked 1,542,280 bytes
==24569==
==24569== 63 (56 direct, 7 indirect) bytes in 1 blocks are definitely lost in loss record 130 of 234
==24569==    at 0x4C2A1D4: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==24569==    by 0x4E879A4: virAllocVar (viralloc.c:544)
==24569==    by 0x4EBD625: virObjectNew (virobject.c:190)
==24569==    by 0x4F3A18A: virGetDomain (datatypes.c:226)
==24569==    by 0x4F9311F: remoteDomainLookupByName (remote_driver.c:6636)
==24569==    by 0x4F44F20: virDomainLookupByName (libvirt.c:2277)
==24569==    by 0x12F616: vshCommandOptDomainBy (virsh-domain.c:105)
==24569==    by 0x131C79: cmdStart (virsh-domain.c:3330)
==24569==    by 0x12C4AB: vshCommandRun (virsh.c:1752)
==24569==    by 0x127001: main (virsh.c:3218)
==24569==
==24569== LEAK SUMMARY:
==24569==    definitely lost: 56 bytes in 1 blocks
==24569==    indirectly lost: 7 bytes in 1 blocks
==24569==      possibly lost: 0 bytes in 0 blocks
==24569==    still reachable: 559,828 bytes in 2,300 blocks
==24569==         suppressed: 0 bytes in 0 blocks



Actual results:
Mem leak while start a guest with a character followed

Expected results:
Shouldn't have mem leak during start the guest. BTW, From the error info in upper step2, we can find that this mem leak may caused by the new virsh start option "--pass-fds"

Comment 3 Ján Tomko 2014-02-24 12:03:26 UTC
Fixed upstream by:
commit 6c1059ef24f72a5d35cbdb0e01b443a8f55e5a15
Author:     Jincheng Miao <jmiao>
AuthorDate: 2014-02-20 17:29:15 +0800
Commit:     Eric Blake <eblake>
CommitDate: 2014-02-20 05:40:13 -0700

    virsh: fix memleak when starting a guest with invalid fd

git describe: v1.2.1-265-g6c1059e contains: v1.2.2-rc1~27

Looking at the code, it seems there is a similar issue with 'virsh create'.

Comment 4 Ján Tomko 2014-02-24 13:31:38 UTC
I have proposed a patch for the 'virsh create' leak:
https://www.redhat.com/archives/libvir-list/2014-February/msg01444.html

Comment 5 Ján Tomko 2014-02-25 09:26:36 UTC
Now fixed upstream by:
commit fe1b6e72d2d2e350e49d0f5d5b0fecbbed854a1b
Author:     Ján Tomko <jtomko>
CommitDate: 2014-02-24 18:46:34 +0100

    virsh: Don't leak buffer if GetFDs fails in cmdCreate
    
    Change the logic of the function to return false by default
    and move the freeing of the buffer to the cleanup section.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1067338

git describe: v1.2.2-rc1-11-gfe1b6e7

Comment 7 vivian zhang 2014-11-26 06:45:07 UTC
I can produce this bug on build libvirt-1.1.1-23.el7.x86_64

verify this bug on build
libvirt-1.2.8-9.el7.x86_64
qemu-kvm-rhev-2.1.2-12.el7.x86_64
kernel-3.10.0-208.el7.x86_64

verify steps:
1. prepare a guest xml and create it with invalid fd number, check mem leak with valgrind is 0 bytes

# valgrind -v --leak-check=full virsh create /tmp/rhel6new.xml --pass-fds a
...
error: Unable to parse FD number 'a'

==25601== 
==25601== HEAP SUMMARY:
==25601==     in use at exit: 109,089 bytes in 1,097 blocks
==25601==   total heap usage: 4,645 allocs, 3,548 frees, 1,289,014 bytes allocated
==25601== 
==25601== Searching for pointers to 1,097 not-freed blocks
==25601== Checked 1,554,488 bytes
==25601== 
==25601== 40 bytes in 1 blocks are possibly lost in loss record 120 of 192
==25601==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25601==    by 0xBA28FDB: ??? (in /usr/lib64/libnspr4.so)
==25601==    by 0xBA42CB6: ??? (in /usr/lib64/libnspr4.so)
==25601==    by 0xBA432A4: PR_OpenFile (in /usr/lib64/libnspr4.so)
==25601==    by 0xD957AC9: ??? (in /usr/lib64/libfreebl3.so)
==25601==    by 0xD957DF0: ??? (in /usr/lib64/libfreebl3.so)
==25601==    by 0xD92DAE9: ??? (in /usr/lib64/libfreebl3.so)
==25601==    by 0x400F502: _dl_init (in /usr/lib64/ld-2.17.so)
==25601==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==25601==    by 0x4: ???
==25601==    by 0xFFF00067A: ???
==25601==    by 0xFFF000680: ???
==25601== 
==25601== 48 bytes in 1 blocks are possibly lost in loss record 126 of 192
==25601==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25601==    by 0xBA28FC9: ??? (in /usr/lib64/libnspr4.so)
==25601==    by 0xBA42CB6: ??? (in /usr/lib64/libnspr4.so)
==25601==    by 0xBA432A4: PR_OpenFile (in /usr/lib64/libnspr4.so)
==25601==    by 0xD957AC9: ??? (in /usr/lib64/libfreebl3.so)
==25601==    by 0xD957DF0: ??? (in /usr/lib64/libfreebl3.so)
==25601==    by 0xD92DAE9: ??? (in /usr/lib64/libfreebl3.so)
==25601==    by 0x400F502: _dl_init (in /usr/lib64/ld-2.17.so)
==25601==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==25601==    by 0x4: ???
==25601==    by 0xFFF00067A: ???
==25601==    by 0xFFF000680: ???
==25601== 
==25601== LEAK SUMMARY:
==25601==    definitely lost: 0 bytes in 0 blocks
==25601==    indirectly lost: 0 bytes in 0 blocks
==25601==      possibly lost: 88 bytes in 2 blocks
==25601==    still reachable: 109,001 bytes in 1,095 blocks
==25601==         suppressed: 0 bytes in 0 blocks
==25601== Reachable blocks (those to which a pointer was found) are not shown.
==25601== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==25601== 
==25601== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2)
--25601-- 
--25601-- used_suppression:      2 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a /usr/lib64/valgrind/default.supp:1296
==25601== 
==25601== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2)

2. prepare a normal guest in shutoff
# virsh list --all
 Id    Name                           State
----------------------------------------------------
-     rhel7new                       shut off

3. start guest with invalide fd number, also check mem leak with valgrind, leak 0 bytes
# valgrind -v --leak-check=full virsh start rhel7new --pass-fds a
error: Unable to parse FD number 'a'

==25922== 
==25922== HEAP SUMMARY:
==25922==     in use at exit: 109,080 bytes in 1,097 blocks
==25922==   total heap usage: 4,662 allocs, 3,565 frees, 1,346,758 bytes allocated
==25922== 
==25922== Searching for pointers to 1,097 not-freed blocks
==25922== Checked 1,554,912 bytes
==25922== 
==25922== 40 bytes in 1 blocks are possibly lost in loss record 120 of 192
==25922==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25922==    by 0xBA28FDB: ??? (in /usr/lib64/libnspr4.so)
==25922==    by 0xBA42CB6: ??? (in /usr/lib64/libnspr4.so)
==25922==    by 0xBA432A4: PR_OpenFile (in /usr/lib64/libnspr4.so)
==25922==    by 0xD957AC9: ??? (in /usr/lib64/libfreebl3.so)
==25922==    by 0xD957DF0: ??? (in /usr/lib64/libfreebl3.so)
==25922==    by 0xD92DAE9: ??? (in /usr/lib64/libfreebl3.so)
==25922==    by 0x400F502: _dl_init (in /usr/lib64/ld-2.17.so)
==25922==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==25922==    by 0x4: ???
==25922==    by 0xFFF000686: ???
==25922==    by 0xFFF00068C: ???
==25922== 
==25922== 48 bytes in 1 blocks are possibly lost in loss record 126 of 192
==25922==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25922==    by 0xBA28FC9: ??? (in /usr/lib64/libnspr4.so)
==25922==    by 0xBA42CB6: ??? (in /usr/lib64/libnspr4.so)
==25922==    by 0xBA432A4: PR_OpenFile (in /usr/lib64/libnspr4.so)
==25922==    by 0xD957AC9: ??? (in /usr/lib64/libfreebl3.so)
==25922==    by 0xD957DF0: ??? (in /usr/lib64/libfreebl3.so)
==25922==    by 0xD92DAE9: ??? (in /usr/lib64/libfreebl3.so)
==25922==    by 0x400F502: _dl_init (in /usr/lib64/ld-2.17.so)
==25922==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==25922==    by 0x4: ???
==25922==    by 0xFFF000686: ???
==25922==    by 0xFFF00068C: ???
==25922== 
==25922== LEAK SUMMARY:
==25922==    definitely lost: 0 bytes in 0 blocks
==25922==    indirectly lost: 0 bytes in 0 blocks
==25922==      possibly lost: 88 bytes in 2 blocks
==25922==    still reachable: 108,992 bytes in 1,095 blocks
==25922==         suppressed: 0 bytes in 0 blocks
==25922== Reachable blocks (those to which a pointer was found) are not shown.
==25922== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==25922== 
==25922== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2)
--25922-- 
--25922-- used_suppression:      2 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a /usr/lib64/valgrind/default.supp:1296
==25922== 
==25922== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2)


move to verified

Comment 8 vivian zhang 2014-12-04 06:16:29 UTC
Due to some debuginfo missing leads to a lot of ??? shown in result, retest the verify steps

I can produce this bug on build libvirt-1.1.1-23.el7.x86_64

verify this bug on build
libvirt-1.2.8-9.el7.x86_64
qemu-kvm-rhev-2.1.2-12.el7.x86_64
kernel-3.10.0-208.el7.x86_64

verify steps:
1. prepare a guest start it with invalid fd number, check mem leak 
# valgrind -v --leak-check=full virsh start rhel7new --pass-fds a
error: Unable to parse FD number 'a'

--9362-- REDIR: 0x8615020 (libc.so.6:__memmove_ssse3_back) redirected to 0x4c2ded0 (memcpy.5)
==9362== 
==9362== HEAP SUMMARY:
==9362==     in use at exit: 108,216 bytes in 1,075 blocks
==9362==   total heap usage: 2,300 allocs, 1,225 frees, 902,263 bytes allocated
==9362== 
==9362== Searching for pointers to 1,075 not-freed blocks
==9362== Checked 1,485,504 bytes
==9362== 
==9362== 40 bytes in 1 blocks are possibly lost in loss record 100 of 170
==9362==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==9362==    by 0xBA28FDB: _PR_Getfd (prfdcach.c:112)
==9362==    by 0xBA42CB6: pt_SetMethods.isra.11 (ptio.c:3303)
==9362==    by 0xBA432A4: PR_OpenFile (ptio.c:3581)
==9362==    by 0xD957AC9: blapi_SHVerifyFile (shvfy.c:355)
==9362==    by 0xD957DF0: blapi_SHVerify (shvfy.c:289)
==9362==    by 0xD92DAE9: freebl_fipsSoftwareIntegrityTest (fipsfreebl.c:1541)
==9362==    by 0xD92DAE9: bl_startup_tests (fipsfreebl.c:1732)
==9362==    by 0x400F4E2: call_init (dl-init.c:82)
==9362==    by 0x400F4E2: _dl_init (dl-init.c:131)
==9362==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==9362==    by 0x4: ???
==9362==    by 0xFFF000612: ???
==9362==    by 0xFFF000618: ???
==9362== 
==9362== 48 bytes in 1 blocks are possibly lost in loss record 106 of 170
==9362==    at 0x4C29BBD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==9362==    by 0xBA28FC9: _PR_Getfd (prfdcach.c:109)
==9362==    by 0xBA42CB6: pt_SetMethods.isra.11 (ptio.c:3303)
==9362==    by 0xBA432A4: PR_OpenFile (ptio.c:3581)
==9362==    by 0xD957AC9: blapi_SHVerifyFile (shvfy.c:355)
==9362==    by 0xD957DF0: blapi_SHVerify (shvfy.c:289)
==9362==    by 0xD92DAE9: freebl_fipsSoftwareIntegrityTest (fipsfreebl.c:1541)
==9362==    by 0xD92DAE9: bl_startup_tests (fipsfreebl.c:1732)
==9362==    by 0x400F4E2: call_init (dl-init.c:82)
==9362==    by 0x400F4E2: _dl_init (dl-init.c:131)
==9362==    by 0x4001459: ??? (in /usr/lib64/ld-2.17.so)
==9362==    by 0x4: ???
==9362==    by 0xFFF000612: ???
==9362==    by 0xFFF000618: ???
==9362== 
==9362== LEAK SUMMARY:
==9362==    definitely lost: 0 bytes in 0 blocks
==9362==    indirectly lost: 0 bytes in 0 blocks
==9362==      possibly lost: 88 bytes in 2 blocks
==9362==    still reachable: 108,128 bytes in 1,073 blocks
==9362==         suppressed: 0 bytes in 0 blocks
==9362== Reachable blocks (those to which a pointer was found) are not shown.
==9362== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==9362== 
==9362== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2)
--9362-- 
--9362-- used_suppression:      2 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a /usr/lib64/valgrind/default.supp:1296
==9362== 
==9362== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2)


move to verified

Comment 10 errata-xmlrpc 2015-03-05 07:30:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html