Bug 1000894

Summary: [abrt] lvm2-2.02.98-12.fc19: dm_malloc_aux: Process /usr/sbin/lvmetad was killed by signal 11 (SIGSEGV)
Product: [Fedora] Fedora Reporter: Ian Pilcher <ipilcher>
Component: lvm2Assignee: Petr Rockai <prockai>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: agk, bmarzins, bmr, cristian.ciupitu, deanhunter, dwysocha, heinzm, ipilcher, jonathan, lvm-team, msnitzer, prajnoha, prockai, zkabelac
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:2f87b682c1dd27cd1ca4c61e2c6a4e95ea978aa3
Fixed In Version: lvm2-2.02.98-13.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-03 04:31:58 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
File: backtrace
none
File: cgroup
none
File: core_backtrace
none
File: dso_list
none
File: environ
none
File: exploitable
none
File: limits
none
File: maps
none
File: open_fds
none
File: proc_pid_status
none
File: var_log_messages
none
Output of lvmdump -l none

Description Ian Pilcher 2013-08-26 04:19:56 UTC
Description of problem:
Log in to iSCSI target.

Version-Release number of selected component:
lvm2-2.02.98-12.fc19

Additional info:
reporter:       libreport-2.1.6
backtrace_rating: 4
cmdline:        /usr/sbin/lvmetad
crash_function: dm_malloc_aux
executable:     /usr/sbin/lvmetad
kernel:         3.10.9-200.fc19.x86_64
runlevel:       N 5
uid:            0

Truncated backtrace:
Thread no. 1 (9 frames)
 #3 dm_malloc_aux at mm/dbg_malloc.c:269
 #4 _new_chunk at mm/pool-fast.c:281
 #5 dm_pool_alloc_aligned at mm/pool-fast.c:101
 #6 dm_pool_alloc at mm/pool-fast.c:85
 #7 dm_pool_zalloc at mm/pool.c:72
 #8 dm_config_create at libdm-config.c:102
 #9 dm_config_from_string at libdm-config.c:181
 #10 client_thread at daemon-server.c:382
 #12 clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Comment 1 Ian Pilcher 2013-08-26 04:20:03 UTC
Created attachment 790262 [details]
File: backtrace

Comment 2 Ian Pilcher 2013-08-26 04:20:07 UTC
Created attachment 790263 [details]
File: cgroup

Comment 3 Ian Pilcher 2013-08-26 04:20:11 UTC
Created attachment 790264 [details]
File: core_backtrace

Comment 4 Ian Pilcher 2013-08-26 04:20:15 UTC
Created attachment 790265 [details]
File: dso_list

Comment 5 Ian Pilcher 2013-08-26 04:20:19 UTC
Created attachment 790266 [details]
File: environ

Comment 6 Ian Pilcher 2013-08-26 04:20:23 UTC
Created attachment 790267 [details]
File: exploitable

Comment 7 Ian Pilcher 2013-08-26 04:20:27 UTC
Created attachment 790268 [details]
File: limits

Comment 8 Ian Pilcher 2013-08-26 04:20:31 UTC
Created attachment 790269 [details]
File: maps

Comment 9 Ian Pilcher 2013-08-26 04:20:35 UTC
Created attachment 790270 [details]
File: open_fds

Comment 10 Ian Pilcher 2013-08-26 04:20:40 UTC
Created attachment 790271 [details]
File: proc_pid_status

Comment 11 Ian Pilcher 2013-08-26 04:20:44 UTC
Created attachment 790272 [details]
File: var_log_messages

Comment 12 Ian Pilcher 2013-08-28 20:14:02 UTC
Log in to iSCSI target.  LUNs on target contain VM disks, which use LVM.

reporter:       libreport-2.1.6
backtrace_rating: 4
cmdline:        /usr/sbin/lvmetad
crash_function: dm_malloc_aux
executable:     /usr/sbin/lvmetad
kernel:         3.10.9-200.fc19.x86_64
package:        lvm2-2.02.98-12.fc19
reason:         Process /usr/sbin/lvmetad was killed by signal 11 (SIGSEGV)
runlevel:       N 5
uid:            0

Comment 13 Peter Rajnoha 2013-08-29 07:49:01 UTC
(In reply to Ian Pilcher from comment #12)
> Log in to iSCSI target.  LUNs on target contain VM disks, which use LVM.
> 

Please, post the output of lvmdump -l (this will include lvmetad dump so we'll see what's cached in lvmetad daemon).

If I understand correctly, your setup is:
   on host side: iSCSI connected LUN -> LVM (then the LVs used as VM disk)
   on guest side: LVM

If that is true, please, consider using devices/global_filter to completely filter out the guest side LVM volumes so they don't interfere with host side LVM.
Anyway, even without filtering, it should not segfault, of course. That "lvmdump -l" should help us a bit - please, attach it to this report. Thanks.

Comment 14 Ian Pilcher 2013-09-13 14:21:28 UTC
Created attachment 797372 [details]
Output of lvmdump -l

(In reply to Peter Rajnoha from comment #13)
> Please, post the output of lvmdump -l (this will include lvmetad dump so
> we'll see what's cached in lvmetad daemon).

Done.

> If I understand correctly, your setup is:
>    on host side: iSCSI connected LUN -> LVM (then the LVs used as VM disk)
>    on guest side: LVM

That is correct.

> If that is true, please, consider using devices/global_filter to completely
> filter out the guest side LVM volumes so they don't interfere with host side
> LVM.

That was actually my first thought.  Unfortunately, I can't come up with a filter with will ignore /dev/sdc (for example) when it's an iSCSI LUN but still use it when it's a USB drive.  (Yes, I've been known to use LVM on external drives.)

Comment 15 Peter Rajnoha 2013-09-16 13:01:01 UTC
(In reply to Ian Pilcher from comment #14)
> Created attachment 797372 [details]
> Output of lvmdump -l
> 
> (In reply to Peter Rajnoha from comment #13)
> > Please, post the output of lvmdump -l (this will include lvmetad dump so
> > we'll see what's cached in lvmetad daemon).
> 
> Done.

Thanks, I'll have a look...

> 
> > If that is true, please, consider using devices/global_filter to completely
> > filter out the guest side LVM volumes so they don't interfere with host side
> > LVM.
> 
> That was actually my first thought.  Unfortunately, I can't come up with a
> filter with will ignore /dev/sdc (for example) when it's an iSCSI LUN but
> still use it when it's a USB drive.  (Yes, I've been known to use LVM on
> external drives.)

The best is to accept only known PVs and reject any other - maybe try to use a match against /dev/disk/by-* entries - these include some constant name schemes or IDs that do not change upon reboot/connection (e.g. vendor, model names or WWN in case of iSCSI).

If you're using lvmetad (global/use_lvmetad=1 in lvm.conf), be sure to use devices/global_filter instead of devices/filter setting.

Comment 16 Petr Rockai 2013-10-09 22:42:58 UTC
Upstream fix in b5aad86710be57833879ac0e8609021828949682.

Comment 17 Zdenek Kabelac 2013-10-22 08:50:55 UTC
*** Bug 1003278 has been marked as a duplicate of this bug. ***

Comment 18 Ian Pilcher 2013-10-22 15:20:35 UTC
(In reply to Petr Rockai from comment #16)
> Upstream fix in b5aad86710be57833879ac0e8609021828949682.

Any chance of a build with this fix included?

Comment 19 Peter Rajnoha 2013-10-23 07:33:20 UTC
(In reply to Ian Pilcher from comment #18)
> (In reply to Petr Rockai from comment #16)
> > Upstream fix in b5aad86710be57833879ac0e8609021828949682.
> 
> Any chance of a build with this fix included?

There are more patches queued for F19. I'd need to find some time to backport them, but I hope this Friday or during next week definitely.

Comment 20 Peter Rajnoha 2013-10-29 08:13:12 UTC
*** Bug 997656 has been marked as a duplicate of this bug. ***

Comment 21 Fedora Update System 2013-10-31 08:32:02 UTC
lvm2-2.02.98-13.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/lvm2-2.02.98-13.fc19

Comment 22 Fedora Update System 2013-11-01 03:57:25 UTC
Package lvm2-2.02.98-13.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing lvm2-2.02.98-13.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-20436/lvm2-2.02.98-13.fc19
then log in and leave karma (feedback).

Comment 23 Fedora Update System 2013-11-03 04:31:58 UTC
lvm2-2.02.98-13.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.