Bug 557104

Summary: dmeventd crash if both snapshot and mirror monitoring DSO are used
Product: Red Hat Enterprise Linux 5 Reporter: Milan Broz <mbroz>
Component: lvm2Assignee: Petr Rockai <prockai>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: high    
Version: 5.5CC: agk, cmarthal, dwysocha, edamato, heinzm, jbrassow, mbroz, prockai, pvrabec, zkabelac
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-30 09:01:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Milan Broz 2010-01-20 12:46:44 UTC
Description of problem:

Unsafe lvm2lib initialisation between two plugins cause memory corruption and dmeventd crash.

Version-Release number of selected component (if applicable):

lvm2-2.02.56-5.el5
device-mapper-1.02.39-1.el5

How reproducible:

1) run dmeventd in foreground mode "dmeventd -d" 

2)create VG with two mirrors and two snapshots, e.g.

# lvs vg_bar
  LV     VG     Attr   LSize Origin Snap%  Move Log        Copy%  Convert
  lv1    vg_bar owi-a- 4.00M
  lv1_s1 vg_bar swi-a- 4.00M lv1      0.20
  lv1_s2 vg_bar swi-a- 4.00M lv1      0.20
  lv2m   vg_bar mwi-a- 4.00M                    lv2m_mlog  100.00
  lv2m2  vg_bar mwi-a- 4.00M                    lv2m2_mlog 100.00

3) Activate and deactivate all volumes:
while :; do vgchange -a n vg_bar ; vgchange -a y vg_bar ; done

4) Enjoy the crash
# dmeventd -d     
You have a memory leak (not released memory pool):
 [0x9a5d458]                                      
 [0x9a5d478]                                      
 [0x9a5df48]                                      
 [0x9a5df68]                                      
 [0x9a83380]                                      
 [0x9a835a8]                                      
 [0x9a839d0]                                      
 [0x9a844d0]                                      
 [0x9a6c4c8]                                      
device-mapper: waitevent ioctl failed: Interrupted system call
You have a memory leak (not released memory pool):            
 [0x9a83380]                                                  
*** glibc detected *** dmeventd: realloc(): invalid next size: 0x09a73710 ***
======= Backtrace: =========                                                 
/lib/libc.so.6[0x7ff851]                                                     
/lib/libc.so.6(realloc+0xe6)[0x800276]                                       
/usr/lib/liblvm2cmd.so.2.02[0x1769be]                                        
/usr/lib/liblvm2cmd.so.2.02[0x176ad8]                                        
/usr/lib/liblvm2cmd.so.2.02[0x177845]                                        
/usr/lib/liblvm2cmd.so.2.02[0x1912f7]                                        
/usr/lib/liblvm2cmd.so.2.02(lvm2_init+0x1e)[0x19133e]                        
/lib/libdevmapper-event-lvm2mirror.so(register_device+0x9b)[0x6b3bbb]        
dmeventd[0x804aea9]
dmeventd(main+0xc93)[0x804be23]
/lib/libc.so.6(__libc_start_main+0xdc)[0x7a9e9c]

Expected results:

lvm2_init and lvm2_exit (and internal constructors/destructors) must use some kind od reference counting or globally allocated structs must be moved into command context.

Comment 2 Milan Broz 2010-01-27 13:40:49 UTC
Fix in upstream cvs -> POST.

Comment 3 Milan Broz 2010-01-27 15:01:53 UTC
Patch in lvm2-2_02_56-6_el5.

Comment 6 Corey Marthaler 2010-02-03 22:11:08 UTC
Is this message still expected in the latest rpms?

device-mapper: waitevent ioctl failed: Interrupted system call

Comment 7 Corey Marthaler 2010-02-03 22:30:10 UTC
The segfault in this bug is verified fixed in lvm2-2.02.56-6.el5. I also verified that multiple VGs containing multiple snaps/mirrors also works.

Waiting to put this into the VERIFIED state until the question in comment #6 is answered.

Comment 8 Milan Broz 2010-02-04 09:53:00 UTC
(In reply to comment #6)
> device-mapper: waitevent ioctl failed: Interrupted system call    

yes, it is expected.
dmeventd has one thread per monitored device, every thread waits for event (in dm-ioctl syscall). If the device is removed waiting is interrupted.
This warning should be probably more quite (like debug level only) but it is just cosmetic change.

Comment 10 errata-xmlrpc 2010-03-30 09:01:34 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0298.html