Bug 1809571

Summary: lvm2 hints file usage leaks memory
Product: Red Hat Enterprise Linux 8 Reporter: Zdenek Kabelac <zkabelac>
Component: lvm2Assignee: David Teigland <teigland>
lvm2 sub component: Command-line tools QA Contact: cluster-qe <cluster-qe>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: agk, heinzm, jbrassow, mcsontos, msnitzer, pasik, prajnoha, rbednar, rhandlin, tbzatek, teigland, thornber, zkabelac
Version: 8.2Flags: pm-rhel: mirror+
Target Milestone: rc   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: lvm2-2.03.09-2.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 02:00:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1835450    

Description Zdenek Kabelac 2020-03-03 12:37:19 UTC
Description of problem:

These are observable memory leaks per command execution:

21,480 bytes in 5 blocks are indirectly lost in loss record 1 of 2
   at 0x4839809: malloc (vg_replace_malloc.c:309)
   by 0x1E437C: _read_hint_file (hints.c:776)
   by 0x1E531C: get_hints (hints.c:1307)
   by 0x1E15E8: label_scan (label.c:1099)
   by 0x1ABA6A: lvmcache_label_scan (lvmcache.c:1013)
   by 0x189C75: process_each_vg (toollib.c:2260)
   by 0x15F8E2: lvcreate (lvcreate.c:1782)
   by 0x168A47: lvm_run_command (lvmcmdline.c:3130)
   by 0x169E18: lvm2_main (lvmcmdline.c:3660)
   by 0x1A0867: main (lvm.c:22)

25,776 (4,296 direct, 21,480 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 2
   at 0x4839809: malloc (vg_replace_malloc.c:309)
   by 0x1E437C: _read_hint_file (hints.c:776)
   by 0x1E531C: get_hints (hints.c:1307)
   by 0x1E15E8: label_scan (label.c:1099)
   by 0x1ABA6A: lvmcache_label_scan (lvmcache.c:1013)
   by 0x189C75: process_each_vg (toollib.c:2260)
   by 0x15F8E2: lvcreate (lvcreate.c:1782)
   by 0x168A47: lvm_run_command (lvmcmdline.c:3130)
   by 0x169E18: lvm2_main (lvmcmdline.c:3660)
   by 0x1A0867: main (lvm.c:22)

Not a big issue for 'standalong' command execution, however accumulates leaked memory for dmeventd lvm2 command execution.

Version-Release number of selected component (if applicable):
upstream as of today - 2.02.09

How reproducible:
create a thin volume in thin-pool: lvcreate -V10 vg/pool

Comment 1 David Teigland 2020-03-03 18:30:25 UTC
fix pushed to master
https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=f50e7ce76c0c6b2745d0ed2a766f46cea07f310c

(It looks like both backtraces refer to the same issue, and I'm not seeing another issue in this location apart from the one fixed.)

Comment 2 Jonathan Earl Brassow 2020-03-10 21:38:21 UTC
setting assignee to patch author
setting target release
setting devel_ack

Comment 3 Roman Bednář 2020-04-09 09:51:32 UTC
Adding QA ack for 8.3. Still reproducible with lvm2-2.03.08-3.el8.x86_64

# valgrind --leak-check=full lvcreate -V1G -n thinlv vg/POOL
...
==57594==
==57594== 488 bytes in 1 blocks are definitely lost in loss record 4 of 9
==57594==    at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==57594==    by 0x67EBD8E: fdopen@@GLIBC_2.2.5 (in /usr/lib64/libc-2.28.so)
==57594==    by 0x1C7B23: reopen_standard_stream (in /usr/sbin/lvm)
==57594==    by 0x19F2EA: create_toolcontext (in /usr/sbin/lvm)
==57594==    by 0x160983: init_lvm (in /usr/sbin/lvm)
==57594==    by 0x161078: lvm2_main (in /usr/sbin/lvm)
==57594==    by 0x679E6A2: (below main) (in /usr/lib64/libc-2.28.so)
==57594==
==57594== 8,592 (4,296 direct, 4,296 indirect) bytes in 1 blocks are definitely lost in loss record 9 of 9
==57594==    at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==57594==    by 0x1C45D9: ??? (in /usr/sbin/lvm)
==57594==    by 0x1C5826: get_hints (in /usr/sbin/lvm)
==57594==    by 0x1C1EDE: label_scan (in /usr/sbin/lvm)
==57594==    by 0x197297: lvmcache_label_scan (in /usr/sbin/lvm)
==57594==    by 0x17C849: process_each_vg (in /usr/sbin/lvm)
==57594==    by 0x157392: lvcreate (in /usr/sbin/lvm)
==57594==    by 0x15FE9C: lvm_run_command (in /usr/sbin/lvm)
==57594==    by 0x1611C2: lvm2_main (in /usr/sbin/lvm)
==57594==    by 0x679E6A2: (below main) (in /usr/lib64/libc-2.28.so)
...

Comment 5 David Teigland 2020-05-13 22:32:20 UTC
fix in master
https://sourceware.org/git/?p=lvm2.git;a=commit;h=5c095400de2f84511726cc8178800d19657b814d


[root@null-02 ~]# valgrind --leak-check=full lvcreate -l1 foo
==58269== Memcheck, a memory error detector
==58269== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==58269== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==58269== Command: lvcreate -l1 foo
==58269== 
  Logical volume "lvol2" created.
==58269== 
==58269== HEAP SUMMARY:
==58269==     in use at exit: 2,280 bytes in 5 blocks
==58269==   total heap usage: 25,554 allocs, 25,549 frees, 12,894,156 bytes allocated
==58269== 
==58269== 8 bytes in 1 blocks are definitely lost in loss record 1 of 5
==58269==    at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==58269==    by 0x5C910C9: strdup (in /usr/lib64/libc-2.17.so)
==58269==    by 0x1ADF4F: _lvmcache_update_vgstatus (lvmcache.c:1463)
==58269==    by 0x1AE7DD: lvmcache_update_vgname_and_id (lvmcache.c:1667)
==58269==    by 0x1DF71C: _text_read (text_label.c:506)
==58269==    by 0x1E0C03: _process_block (label.c:434)
==58269==    by 0x1E1849: _scan_list (label.c:721)
==58269==    by 0x1E248D: label_scan (label.c:1110)
==58269==    by 0x1ACBD6: lvmcache_label_scan (lvmcache.c:1050)
==58269==    by 0x18ABF5: process_each_vg (toollib.c:2029)
==58269==    by 0x160770: lvcreate (lvcreate.c:1795)
==58269==    by 0x169797: lvm_run_command (lvmcmdline.c:3133)
==58269== 
==58269== 568 bytes in 1 blocks are definitely lost in loss record 4 of 5
==58269==    at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==58269==    by 0x5C732B4: fdopen@@GLIBC_2.2.5 (in /usr/lib64/libc-2.17.so)
==58269==    by 0x1E7421: reopen_standard_stream (log.c:199)
==58269==    by 0x1B5342: create_toolcontext (toolcontext.c:1627)
==58269==    by 0x16A57B: init_lvm (lvmcmdline.c:3444)
==58269==    by 0x16ADCF: lvm2_main (lvmcmdline.c:3598)
==58269==    by 0x1A1868: main (lvm.c:22)
==58269== 
==58269== 568 bytes in 1 blocks are definitely lost in loss record 5 of 5
==58269==    at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==58269==    by 0x5C732B4: fdopen@@GLIBC_2.2.5 (in /usr/lib64/libc-2.17.so)
==58269==    by 0x1E7421: reopen_standard_stream (log.c:199)
==58269==    by 0x1B5480: create_toolcontext (toolcontext.c:1638)
==58269==    by 0x16A57B: init_lvm (lvmcmdline.c:3444)
==58269==    by 0x16ADCF: lvm2_main (lvmcmdline.c:3598)
==58269==    by 0x1A1868: main (lvm.c:22)
==58269== 
==58269== LEAK SUMMARY:
==58269==    definitely lost: 1,144 bytes in 3 blocks
==58269==    indirectly lost: 0 bytes in 0 blocks
==58269==      possibly lost: 0 bytes in 0 blocks
==58269==    still reachable: 1,136 bytes in 2 blocks
==58269==         suppressed: 0 bytes in 0 blocks
==58269== Reachable blocks (those to which a pointer was found) are not shown.
==58269== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==58269== 
==58269== For counts of detected and suppressed errors, rerun with: -v
==58269== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)

Comment 8 Roman Bednář 2020-06-16 08:39:34 UTC
Valgrind still detecting memleak with latest build - lvm2-2.03.09-2.el8.x86_64.


# valgrind --leak-check=full lvcreate -l1 foo
==7079== Memcheck, a memory error detector
==7079== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==7079== Using Valgrind-3.16.0 and LibVEX; rerun with -h for copyright info
==7079== Command: lvcreate -l1 foo
==7079==
==7079== Warning: invalid file descriptor 1024 in syscall close()
==7079== Warning: invalid file descriptor 1025 in syscall close()
==7079== Warning: invalid file descriptor 1026 in syscall close()
==7079== Warning: invalid file descriptor 1027 in syscall close()
==7079==    Use --log-fd=<number> to select an alternative log fd.
==7079== Warning: invalid file descriptor 1028 in syscall close()
==7079== Warning: invalid file descriptor 1029 in syscall close()
==7079== Warning: invalid file descriptor 1030 in syscall close()
  Volume group "foo" not found
  Cannot process volume group foo
==7079==
==7079== HEAP SUMMARY:
==7079==     in use at exit: 10,144 bytes in 6 blocks
==7079==   total heap usage: 24,445 allocs, 24,439 frees, 15,484,589 bytes allocated
==7079==
==7079== 488 bytes in 1 blocks are definitely lost in loss record 3 of 6
==7079==    at 0x4C30F0B: malloc (vg_replace_malloc.c:307)
==7079==    by 0x67EBE7E: fdopen@@GLIBC_2.2.5 (in /usr/lib64/libc-2.28.so)
==7079==    by 0x1C8C43: reopen_standard_stream (in /usr/sbin/lvm)
==7079==    by 0x1A0175: create_toolcontext (in /usr/sbin/lvm)
==7079==    by 0x161843: init_lvm (in /usr/sbin/lvm)
==7079==    by 0x161F38: lvm2_main (in /usr/sbin/lvm)
==7079==    by 0x679E7B2: (below main) (in /usr/lib64/libc-2.28.so)
==7079==
==7079== 488 bytes in 1 blocks are definitely lost in loss record 4 of 6
==7079==    at 0x4C30F0B: malloc (vg_replace_malloc.c:307)
==7079==    by 0x67EBE7E: fdopen@@GLIBC_2.2.5 (in /usr/lib64/libc-2.28.so)
==7079==    by 0x1C8C43: reopen_standard_stream (in /usr/sbin/lvm)
==7079==    by 0x1A01FA: create_toolcontext (in /usr/sbin/lvm)
==7079==    by 0x161843: init_lvm (in /usr/sbin/lvm)
==7079==    by 0x161F38: lvm2_main (in /usr/sbin/lvm)
==7079==    by 0x679E7B2: (below main) (in /usr/lib64/libc-2.28.so)
==7079==
==7079== LEAK SUMMARY:
==7079==    definitely lost: 976 bytes in 2 blocks
==7079==    indirectly lost: 0 bytes in 0 blocks
==7079==      possibly lost: 0 bytes in 0 blocks
==7079==    still reachable: 9,168 bytes in 4 blocks
==7079==         suppressed: 0 bytes in 0 blocks
==7079== Reachable blocks (those to which a pointer was found) are not shown.
==7079== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==7079==
==7079== For lists of detected and suppressed errors, rerun with: -s
==7079== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

Comment 9 David Teigland 2020-06-16 14:49:31 UTC
Different issue, will need a different bug.

Comment 10 Zdenek Kabelac 2020-06-16 14:53:53 UTC
This 'reopen' is actually not a 'real' problem - it's our current solution to avoid glibc messing with its memory allocations inside critical section.
It's unclear what's the best solution for this issue - but it's unrelated to hints.

Ideally create 'valgrind' suppression pattern for these 2 reports.

Comment 13 errata-xmlrpc 2020-11-04 02:00:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4546