Bug 1445987
| Summary: | RAID TAKEOVER: lvconvert segfault in _raid_add_target_line | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Corey Marthaler <cmarthal> | ||||
| Component: | lvm2 | Assignee: | Heinz Mauelshagen <heinzm> | ||||
| lvm2 sub component: | Mirroring and RAID | QA Contact: | cluster-qe <cluster-qe> | ||||
| Status: | CLOSED CURRENTRELEASE | Docs Contact: | |||||
| Severity: | medium | ||||||
| Priority: | unspecified | CC: | agk, cmarthal, heinzm, jbrassow, msnitzer, prajnoha, prockai, teigland, zkabelac | ||||
| Version: | 7.4 | ||||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-05-26 14:30:46 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Corey Marthaler
2017-04-26 22:48:45 UTC
Created attachment 1274454 [details]
verbose lvconvert attempt
Corey, this works for me, so if you can reproduce it save the core file (or better I can log in and run gdb on the machine.) Saved this core if more info is needed.
Core was generated by `lvconvert --yes -R 2048.00k --type raid4 centipede2/takeover'.
Program terminated with signal 11, Segmentation fault.
#0 0x000055fee48ab8cb in _add_dev_to_dtree (dm=dm@entry=0x55fee6ab2360,
dtree=dtree@entry=0x55fee6a6daa0, lv=lv@entry=0x55fee6a78170, layer=layer@entry=0x0)
at activate/dev_manager.c:1723
1723 if (!(name = dm_build_dm_name(dm->mem, lv->vg->name, lv->name, layer)))
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.168-5.el7.x86_64 elfutils-libs-0.168-5.el7.x86_64 glibc-2.17-194.el7.x86_64 libattr-2.4.46-12.el7.x86_64 libblkid-2.23.2-39.el7.x86_64 libcap-2.22-9.el7.x86_64 libgcc-4.8.5-14.el7.x86_64 libselinux-2.5-11.el7.x86_64 libsepol-2.5-6.el7.x86_64 libuuid-2.23.2-39.el7.x86_64 ncurses-libs-5.9-13.20130511.el7.x86_64 pcre-8.32-17.el7.x86_64 readline-6.2-10.el7.x86_64 systemd-libs-219-38.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64
(gdb) bt
#0 0x000055fee48ab8cb in _add_dev_to_dtree (dm=dm@entry=0x55fee6ab2360,
dtree=dtree@entry=0x55fee6a6daa0, lv=lv@entry=0x55fee6a78170, layer=layer@entry=0x0)
at activate/dev_manager.c:1723
#1 0x000055fee48abdd0 in _add_lv_to_dtree (dm=dm@entry=0x55fee6ab2360,
dtree=dtree@entry=0x55fee6a6daa0, lv=0x55fee6a78170, origin_only=origin_only@entry=0)
at activate/dev_manager.c:2030
#2 0x000055fee48ac22b in _add_lv_to_dtree (dm=dm@entry=0x55fee6ab2360,
dtree=dtree@entry=0x55fee6a6daa0, lv=lv@entry=0x55fee6a87150, origin_only=0)
at activate/dev_manager.c:2185
#3 0x000055fee48afdd4 in _create_partial_dtree (origin_only=0, lv=0x55fee6a87150,
dm=0x55fee6ab2360) at activate/dev_manager.c:2210
#4 _tree_action (dm=dm@entry=0x55fee6ab2360, lv=lv@entry=0x55fee6a87150,
laopts=laopts@entry=0x7fff1dbbfd60, action=action@entry=PRELOAD)
at activate/dev_manager.c:3102
#5 0x000055fee48b2796 in dev_manager_preload (dm=dm@entry=0x55fee6ab2360,
lv=lv@entry=0x55fee6a87150, laopts=laopts@entry=0x7fff1dbbfd60,
flush_required=flush_required@entry=0x7fff1dbbfd0c) at activate/dev_manager.c:3210
#6 0x000055fee48090a4 in _lv_preload (lv=lv@entry=0x55fee6a87150,
laopts=laopts@entry=0x7fff1dbbfd60,
flush_required=flush_required@entry=0x7fff1dbbfd0c) at activate/activate.c:1414
#7 0x000055fee480dc1d in _lv_suspend (error_if_not_suspended=0, lv_pre=0x55fee6a87150,
lv=0x55fee6a9b170, laopts=0x7fff1dbbfd60,
lvid_s=0x7fff1dbc0e80 "sNM4hOHdQ39rEw82Qp9KH6X9khLJtAcjY4fHuctxCxB8R0rMcGM0H3aDKXyx1ed---Type <return> to continue, or q <return> to quit---
w", cmd=0x55fee69c9020) at activate/activate.c:2152
#8 lv_suspend_if_active (cmd=cmd@entry=0x55fee69c9020,
lvid_s=lvid_s@entry=0x7fff1dbc0e80 "sNM4hOHdQ39rEw82Qp9KH6X9khLJtAcjY4fHuctxCxB8R0rMcGM0H3aDKXyx1edw", origin_only=origin_only@entry=0, exclusive=exclusive@entry=0,
lv=<optimized out>, lv_pre=<optimized out>) at activate/activate.c:2265
#9 0x000055fee48c3e9d in _file_lock_resource (cmd=0x55fee69c9020,
resource=0x7fff1dbc0e80 "sNM4hOHdQ39rEw82Qp9KH6X9khLJtAcjY4fHuctxCxB8R0rMcGM0H3aDKXyx1edw", flags=60, lv=<optimized out>) at locking/file_locking.c:114
#10 0x000055fee483ee38 in _lock_vol (cmd=cmd@entry=0x55fee69c9020,
resource=<optimized out>,
resource@entry=0x7fff1dbc0e80 "sNM4hOHdQ39rEw82Qp9KH6X9khLJtAcjY4fHuctxCxB8R0rMcGM0H3aDKXyx1edw", flags=flags@entry=60, lv_op=lv_op@entry=LV_SUSPEND,
lv=lv@entry=0x55fee6a87150) at locking/locking.c:275
#11 0x000055fee483f783 in lock_vol (cmd=0x55fee69c9020, vol=<optimized out>,
vol@entry=0x55fee6a87150 "sNM4hOHdQ39rEw82Qp9KH6X9khLJtAcjY4fHuctxCxB8R0rMcGM0H3aDKXyx1edw", flags=60, lv=lv@entry=0x55fee6a87150) at locking/locking.c:355
#12 0x000055fee484972c in _lv_update_and_reload (lv=lv@entry=0x55fee6a87150,
origin_only=origin_only@entry=0) at metadata/lv_manip.c:6371
#13 0x000055fee4851d07 in lv_update_and_reload (lv=lv@entry=0x55fee6a87150)
at metadata/lv_manip.c:6397
#14 0x000055fee487efe6 in _lv_update_reload_fns_reset_eliminate_lvs (
lv=lv@entry=0x55fee6a87150, origin_only=0, origin_only=0)
at metadata/raid_manip.c:580
#15 0x000055fee4883716 in _takeover_upconvert_wrapper (lv=0x55fee6a87150,
new_segtype=0x55fee6a07590, new_image_count=<optimized out>, new_data_copies=2,
new_stripe_size=<optimized out>, new_region_size=<optimized out>,
allocate_pvs=0x55fee6a863e8, force=<optimized out>, yes=<optimized out>,
new_stripes=0) at metadata/raid_manip.c:5274
#16 0x000055fee48869d3 in lv_raid_convert (lv=lv@entry=0x55fee6a87150,
new_segtype=0x55fee6a07590, yes=1, force=0, new_stripes=0,
new_stripe_size_supplied=<optimized out>, new_stripe_size=128, new_region_size=4096,
allocate_pvs=0x55fee6a863e8) at metadata/raid_manip.c:6143
#17 0x000055fee47d121d in _lvconvert_raid (lv=lv@entry=0x55fee6a87150,
lp=lp@entry=0x7fff1dbc1820) at lvconvert.c:1408
#18 0x000055fee47d307c in _convert_striped_raid (cmd=<optimized out>, lp=0x7fff1dbc1820,
lv=0x55fee6a87150) at lvconvert.c:1617
#19 _convert_striped (lp=<optimized out>, lv=<optimized out>, cmd=<optimized out>)
at lvconvert.c:1684
#20 _lvconvert_raid_types (cmd=cmd@entry=0x55fee69c9020, lv=lv@entry=0x55fee6a87150,
lp=lp@entry=0x7fff1dbc1820) at lvconvert.c:1757
#21 0x000055fee47d329a in _lvconvert_raid_types_single (cmd=cmd@entry=0x55fee69c9020,
lv=0x55fee6a87150, handle=handle@entry=0x55fee6a14ff8) at lvconvert.c:4249
#22 0x000055fee47f81e8 in process_each_lv_in_vg (cmd=cmd@entry=0x55fee69c9020,
vg=vg@entry=0x55fee6a86310, arg_lvnames=arg_lvnames@entry=0x7fff1dbc1700,
tags_in=tags_in@entry=0x7fff1dbc16b0, stop_on_error=stop_on_error@entry=0,
handle=handle@entry=0x55fee6a14ff8,
---Type <return> to continue, or q <return> to quit---
check_single_lv=check_single_lv@entry=0x55fee47cd2f0 <_lvconvert_raid_types_check>,
process_single_lv=process_single_lv@entry=0x55fee47d3220 <_lvconvert_raid_types_single>) at toollib.c:3136
#23 0x000055fee47f9634 in _process_lv_vgnameid_list (
process_single_lv=0x55fee47d3220 <_lvconvert_raid_types_single>,
check_single_lv=0x55fee47cd2f0 <_lvconvert_raid_types_check>, handle=0x55fee6a14ff8,
arg_tags=0x7fff1dbc16b0, arg_lvnames=0x7fff1dbc16d0, arg_vgnames=0x7fff1dbc16c0,
vgnameids_to_process=0x7fff1dbc16f0, read_flags=1048576, cmd=0x55fee69c9020)
at toollib.c:3631
#24 process_each_lv (cmd=cmd@entry=0x55fee69c9020, argc=argc@entry=1,
argv=<optimized out>, one_vgname=one_vgname@entry=0x0,
one_lvname=one_lvname@entry=0x0, read_flags=read_flags@entry=1048576,
handle=handle@entry=0x55fee6a14ff8,
check_single_lv=check_single_lv@entry=0x55fee47cd2f0 <_lvconvert_raid_types_check>,
process_single_lv=process_single_lv@entry=0x55fee47d3220 <_lvconvert_raid_types_single>) at toollib.c:3783
#25 0x000055fee47d54f8 in lvconvert_raid_types_cmd (cmd=0x55fee69c9020,
argc=<optimized out>, argv=<optimized out>) at lvconvert.c:4336
#26 0x000055fee47e1198 in lvm_run_command (cmd=cmd@entry=0x55fee69c9020, argc=1,
argc@entry=7, argv=0x7fff1dbc1ce8, argv@entry=0x7fff1dbc1cb8) at lvmcmdline.c:2951
#27 0x000055fee47e21f3 in lvm2_main (argc=7, argv=0x7fff1dbc1cb8) at lvmcmdline.c:3485
#28 0x00007f7ebab6ac05 in __libc_start_main () from /lib64/libc.so.6
#29 0x000055fee47c0ece in _start ()
(gdb) frame
#2 0x000055fee48ac22b in _add_lv_to_dtree (dm=dm@entry=0x55fee6ab2360,
dtree=dtree@entry=0x55fee6a6daa0, lv=lv@entry=0x55fee6a87150, origin_only=0)
at activate/dev_manager.c:2185
2185 !_add_lv_to_dtree(dm, dtree, seg_metalv(seg, s), 0))
Looks like the meta_areas[0] value is bad:
(gdb) p *((struct logical_volume *)(seg->meta_areas[0].u.lv.lv))
$14 = {lvid = {id = {{
uuid = "\002\000\000\000\000\000\000\000\221\b\000\000\000\000\000\000\310\\\360\272~\177\000\000\310\\\360\272~\177\000"}, {
uuid = "p\201\247\346\376U\000\000p\201\247\346\376U\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000"}},
s = "\002\000\000\000\000\000\000\000\221\b\000\000\000\000\000\000\310\\\360\272~\177\000\000\310\\\360\272~\177\000\000p\201\247\346\376U\000\000p\201\247\346\376U\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\330{\250\346\376U\000"},
name = 0x0, vg = 0x2, status = 94553279856408, alloc = ALLOC_INVALID, profile = 0x0,
read_ahead = 1634560370, major = 6251879, minor = 0, size = 0, le_count = 2065,
origin_count = 0, external_count = 3136314536, snapshot_segs = {
n = 0x7f7ebaf05ca8 <main_arena+1352>, p = 0x55fee6a781f0},
snapshot = 0x55fee6a781f0, rdevice = 0x55fee6a758e0, rsites = {n = 0x55fee6a179c0,
p = 0x55fee6a81d70}, segments = {n = 0x55fee6a78220, p = 0x0}, tags = {n = 0x0,
p = 0x0}, segs_using_this_lv = {n = 0x0, p = 0x0}, indirect_glvs = {n = 0x0,
p = 0x0}, this_glv = 0x0, timestamp = 94553279792488, new_lock_args = 0,
hostname = 0x55fee6a78710 "\220\202\247\346\376U",
lock_args = 0x55fee6a78710 "\220\202\247\346\376U"}
Let's retest this with the latest code (which fixed an in-memory metadata corruption problem). If it still fails, grab fresh -vvvv traces and gdb backtraces. This appears to be fixed in the latest rpms. Will reopen if seen again. For testing details, see: https://bugzilla.redhat.com/show_bug.cgi?id=1441334#c14 3.10.0-651.el7.x86_64 lvm2-2.02.171-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 lvm2-libs-2.02.171-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 lvm2-cluster-2.02.171-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 device-mapper-1.02.140-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 device-mapper-libs-1.02.140-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 device-mapper-event-1.02.140-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 device-mapper-event-libs-1.02.140-2.el7 BUILT: Wed May 24 09:02:34 CDT 2017 device-mapper-persistent-data-0.7.0-0.1.rc6.el7 BUILT: Mon Mar 27 10:15:46 CDT 2017 |