Bug 1872903
| Summary: | remove pvmove restriction on naming writecache LV | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | David Teigland <teigland> |
| Component: | lvm2 | Assignee: | David Teigland <teigland> |
| lvm2 sub component: | Cache Logical Volumes | QA Contact: | cluster-qe <cluster-qe> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | unspecified | ||
| Priority: | high | CC: | agk, cmarthal, heinzm, jbrassow, mcsontos, msnitzer, prajnoha, zkabelac |
| Version: | 8.3 | Keywords: | Reopened, Triaged |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
| Target Release: | 8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | lvm2-2.03.12-3.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-11-09 19:45:20 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
David Teigland
2020-08-26 21:09:02 UTC
pushed to master https://sourceware.org/git/?p=lvm2.git;a=commit;h=d8bb85d9638e1b0cad96c654a1e4a61b6d33aa97 Reopening... I was auditing our tests to ensure that we do now test pvmoves for write cache volumes and found a bug when attempting just the named "main" volume. Also to be clear, the write cache pool (fast vol) continues to *not* be supported? kernel-4.18.0-305.6.el8 BUILT: Thu May 13 14:06:29 CDT 2021 lvm2-2.03.12-2.el8 BUILT: Tue Jun 1 06:55:37 CDT 2021 lvm2-libs-2.03.12-2.el8 BUILT: Tue Jun 1 06:55:37 CDT 2021 [root@hayes-03 ~]# lvs -a -o +devices,segtype | grep cw cworigin pv_shuffle_B Cwi-aoC--- 752.00m [cwpool_cvol] [cworigin_wcorig] 3.75 cworigin_wcorig(0) writecache [cworigin_wcorig] pv_shuffle_B owi-aoC--- 752.00m /dev/sdl2(0) linear [cwpool_cvol] pv_shuffle_B Cwi-aoC--- 500.00m /dev/sdl2(3947) linear # Scenario 1: (the named _wcorig volume) [root@hayes-03 ~]# pvmove --atomic -n pv_shuffle_B/cworigin_wcorig -v /dev/sdd2 /dev/sdl2 Archiving volume group "pv_shuffle_B" metadata (seqno 98). Creating logical volume pvmove0 activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/cworigin. Moving 188 extents of logical volume pv_shuffle_B/cworigin_wcorig. Creating logical volume pvmove0_mimage_0 Creating logical volume pvmove0_mimage_1 activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/cworigin. Loading table for pv_shuffle_B-cwpool_cvol (253:76). Suppressed pv_shuffle_B-cwpool_cvol (253:76) identical table reload. Creating pv_shuffle_B-pvmove0_mimage_0 Loading table for pv_shuffle_B-pvmove0_mimage_0 (253:80). Creating pv_shuffle_B-pvmove0_mimage_1 Loading table for pv_shuffle_B-pvmove0_mimage_1 (253:81). Creating pv_shuffle_B-pvmove0 Loading table for pv_shuffle_B-pvmove0 (253:82). Loading table for pv_shuffle_B-cworigin_wcorig (253:77). Loading table for pv_shuffle_B-cworigin (253:75). Suppressed pv_shuffle_B-cworigin (253:75) identical table reload. Suspending pv_shuffle_B-cworigin (253:75) with device flush Suspending pv_shuffle_B-cwpool_cvol (253:76) with device flush Suspending pv_shuffle_B-cworigin_wcorig (253:77) with device flush Loading table for pv_shuffle_B-cwpool_cvol (253:76). Suppressed pv_shuffle_B-cwpool_cvol (253:76) identical table reload. Loading table for pv_shuffle_B-cworigin (253:75). Suppressed pv_shuffle_B-cworigin (253:75) identical table reload. Resuming pv_shuffle_B-pvmove0_mimage_0 (253:80). Resuming pv_shuffle_B-pvmove0_mimage_1 (253:81). Resuming pv_shuffle_B-pvmove0 (253:82). Resuming pv_shuffle_B-cwpool_cvol (253:76). Resuming pv_shuffle_B-cworigin_wcorig (253:77). Resuming pv_shuffle_B-cworigin (253:75). Creating volume group backup "/etc/lvm/backup/pv_shuffle_B" (seqno 99). activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/pvmove0. Checking progress before waiting every 15 seconds. /dev/sdd2: Moved: 4.26% /dev/sdd2: Moved: 100.00% Polling finished successfully. # Scenario 2: (the named _cvol volume) [root@hayes-03 ~]# pvmove --atomic -n pv_shuffle_B/cwpool_cvol -v /dev/sdl2 /dev/sdd2 Archiving volume group "pv_shuffle_B" metadata (seqno 98). Creating logical volume pvmove0 Unable to pvmove device used for writecache. # Scenario 3: (the named top level main volume) [root@hayes-03 ~]# pvmove -n pv_shuffle_B/cworigin -v /dev/sdl2 /dev/sdd2 Archiving volume group "pv_shuffle_B" metadata (seqno 104). Creating logical volume pvmove0 activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/cworigin. activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/cworigin. Moving 125 extents of logical volume pv_shuffle_B/cwpool_cvol. activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/cworigin. Moving 188 extents of logical volume pv_shuffle_B/cworigin_wcorig. activation/volume_list configuration setting not defined: Checking only host tags for pv_shuffle_B/cworigin. Segmentation fault (core dumped) Core was generated by `pvmove -n pv_shuffle_B/cworigin -v /dev/sdd2 /dev/sdl2'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000564bc76a3aad in lv_on_pmem (lv=0x564bc9ae2900) at metadata/metadata.c:4407
4407 if (dev_is_pmem(lv->vg->cmd->dev_types, pv->dev)) {
Missing separate debuginfos, use: yum debuginfo-install pcre2-10.32-2.el8.x86_64
(gdb) bt
#0 0x0000564bc76a3aad in lv_on_pmem (lv=0x564bc9ae2900) at metadata/metadata.c:4407
#1 0x0000564bc763c142 in _writecache_add_target_line (dm=<optimized out>, mem=0x564bc9aab1f0, cmd=<optimized out>, target_state=<optimized out>, seg=0x564bc9ae6c98, laopts=<optimized out>, node=<optimized out>, len=<optimized out>,
pvmove_mirror_count=<optimized out>) at writecache/writecache.c:320
#2 0x0000564bc76dce2f in _add_target_to_dtree (dm=<optimized out>, dnode=<optimized out>, seg=<optimized out>, laopts=<optimized out>) at activate/dev_manager.c:250
#3 0x0000564bc76dfd63 in _add_segment_to_dtree (layer=0x0, laopts=0x7ffc5bee0860, seg=<optimized out>, dnode=0x564bc916d310, dtree=0x564bc916d130, dm=0x564bc9b0cb40) at activate/dev_manager.c:3217
#4 _add_new_lv_to_dtree (dm=dm@entry=0x564bc9b0cb40, dtree=dtree@entry=0x564bc916d130, lv=lv@entry=0x564bc9ae2660, laopts=laopts@entry=0x7ffc5bee0860, layer=0x0) at activate/dev_manager.c:3523
#5 0x0000564bc76e28f7 in _tree_action (dm=dm@entry=0x564bc9b0cb40, lv=lv@entry=0x564bc9ae2660, laopts=laopts@entry=0x7ffc5bee0860, action=action@entry=PRELOAD) at activate/dev_manager.c:3803
#6 0x0000564bc76e575a in dev_manager_preload (dm=dm@entry=0x564bc9b0cb40, lv=lv@entry=0x564bc9ae2660, laopts=laopts@entry=0x7ffc5bee0860, flush_required=flush_required@entry=0x7ffc5bee07fc) at activate/dev_manager.c:3867
#7 0x0000564bc7630391 in _lv_preload (lv=lv@entry=0x564bc9ae2660, laopts=laopts@entry=0x7ffc5bee0860, flush_required=flush_required@entry=0x7ffc5bee07fc) at activate/activate.c:1402
#8 0x0000564bc76350f0 in _lv_suspend (error_if_not_suspended=0, lvid_s=0x0, lv_pre=0x564bc9ae2660, lv=0x564bc9aee690, laopts=0x7ffc5bee0860, cmd=0x564bc901ea10) at activate/activate.c:2090
#9 lv_suspend_if_active (cmd=0x564bc901ea10, lvid_s=lvid_s@entry=0x0, origin_only=origin_only@entry=0, exclusive=exclusive@entry=0, lv=0x564bc9aee690, lv_pre=lv_pre@entry=0x564bc9ae2660) at activate/activate.c:2228
#10 0x0000564bc763667e in suspend_lv (cmd=<optimized out>, lv=lv@entry=0x564bc9ae2660) at activate/activate.c:2887
#11 0x0000564bc7681495 in _lv_update_and_reload (lv=0x564bc9ae2660, origin_only=<optimized out>, origin_only@entry=0) at metadata/lv_manip.c:7034
#12 0x0000564bc768ad4b in lv_update_and_reload (lv=<optimized out>) at metadata/lv_manip.c:7064
#13 0x0000564bc760c16e in _update_metadata (exclusive=<optimized out>, lvs_changed=<optimized out>, lv_mirr=0x564bc9aec018) at pvmove.c:562
#14 _pvmove_setup_single (cmd=cmd@entry=0x564bc901ea10, vg=vg@entry=0x564bc9ae0ac0, pv=pv@entry=0x564bc9ae0c80, handle=handle@entry=0x564bc9119128) at pvmove.c:724
#15 0x0000564bc76201c2 in _process_pvs_in_vg (process_all_devices=0, process_single_pv=0x564bc760bdb0 <_pvmove_setup_single>, handle=0x564bc9119128, error_flags=0, skip=0, process_all_pvs=0, arg_tags=0x7ffc5bee0b50, arg_devices=0x7ffc5bee0b70,
all_devices=0x7ffc5bee0b90, vg=0x564bc9ae0ac0, cmd=0x564bc901ea10) at toollib.c:4215
#16 _process_pvs_in_vgs (process_all_devices=0, process_single_pv=0x564bc760bdb0 <_pvmove_setup_single>, handle=0x564bc9119128, process_all_pvs=0, arg_tags=0x7ffc5bee0b50, arg_devices=0x7ffc5bee0b70, all_devices=0x7ffc5bee0b90,
all_vgnameids=0x7ffc5bee0b80, read_flags=1310720, cmd=0x564bc901ea10) at toollib.c:4337
#17 process_each_pv (cmd=cmd@entry=0x564bc901ea10, argc=argc@entry=1, argv=argv@entry=0x7ffc5bee1020, only_this_vgname=only_this_vgname@entry=0x0, all_is_set=all_is_set@entry=0, read_flags=1310720, read_flags@entry=1048576, handle=<optimized out>,
process_single_pv=<optimized out>) at toollib.c:4470
#18 0x0000564bc760d6dd in pvmove (cmd=0x564bc901ea10, argc=<optimized out>, argv=0x7ffc5bee13e0) at pvmove.c:898
#19 0x0000564bc75fd628 in lvm_run_command (cmd=cmd@entry=0x564bc901ea10, argc=<optimized out>, argc@entry=6, argv=<optimized out>, argv@entry=0x7ffc5bee13b8) at lvmcmdline.c:3239
#20 0x0000564bc75fe704 in lvm2_main (argc=6, argv=0x7ffc5bee13b8) at lvmcmdline.c:3771
#21 0x00007fdc30cf5493 in __libc_start_main (main=0x564bc75d7980 <main>, argc=6, argv=0x7ffc5bee13b8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc5bee13a8) at ../csu/libc-start.c:314
#22 0x0000564bc75d7a4e in _start () at lvm.c:22
This was caused by the unusual setup in which the writecache cachevol exists on the same underlying device as the main LV. Fixed in: https://sourceware.org/git/?p=lvm2.git;a=commit;h=247f69f9aafe731ef85268da1e6ce817295b265d https://sourceware.org/git/?p=lvm2.git;a=commit;h=e7f107c24666c8577f30e530b74f1ce0347e459b # lvs -a test -o+devices LV VG Attr LSize Pool Origin Data% Devices [fast_cvol] test Cwi-aoC--- 128.00m /dev/sde(64) main test Cwi-a-C--- 256.00m [fast_cvol] [main_wcorig] 0.00 main_wcorig(0) [main_wcorig] test owi-aoC--- 256.00m /dev/sde(0) # pvmove -n test/main /dev/sde /dev/sdd Unable to move device used for writecache cachevol test/fast_cvol. > Also to be clear, the write cache pool (fast vol) continues to *not* be supported? Right, we don't want to move the cachevol LV, it should remain on the same underlying fast device, while we allow moving the origin LV between slow devices. If someone wants to use a different fast device for their cache, they can remove the cache and add a new cache from the new device. Actually, this doesn't require the cachevol to exist on the same underlying device as the main LV, just that that is the device called to move along with the main LV.
# different devices
[root@hayes-01 ~]# lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
[POOL_cvol] writecache_sanity Cwi-aoC--- 2.00g /dev/sdc1(0)
main_pvmove writecache_sanity Cwi-a-C--- 4.00g [POOL_cvol] [main_pvmove_wcorig] 0.00 main_pvmove_wcorig(0)
[main_pvmove_wcorig] writecache_sanity owi-aoC--- 4.00g /dev/sdd1(0)
# use wrong device here when naming the main origin
[root@hayes-01 ~]# pvmove -n writecache_sanity/main_pvmove -v /dev/sdc1 /dev/sdb1
Archiving volume group "writecache_sanity" metadata (seqno 196).
Creating logical volume pvmove0
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove.
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove.
Moving 512 extents of logical volume writecache_sanity/POOL_cvol.
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove.
Segmentation fault (core dumped)
# use correct device
[root@hayes-01 ~]# pvmove -n writecache_sanity/main_pvmove -v /dev/sdd1 /dev/sdb1
Archiving volume group "writecache_sanity" metadata (seqno 196).
Creating logical volume pvmove0
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove.
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove.
Moving 1024 extents of logical volume writecache_sanity/main_pvmove_wcorig.
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove.
Loading table for writecache_sanity-POOL_cvol (253:1).
Suppressed writecache_sanity-POOL_cvol (253:1) identical table reload.
Creating writecache_sanity-pvmove0
Loading table for writecache_sanity-pvmove0 (253:3).
Loading table for writecache_sanity-main_pvmove_wcorig (253:2).
Loading table for writecache_sanity-main_pvmove (253:0).
Suppressed writecache_sanity-main_pvmove (253:0) identical table reload.
Suspending writecache_sanity-main_pvmove (253:0) with device flush
Suspending writecache_sanity-POOL_cvol (253:1) with device flush
Suspending writecache_sanity-main_pvmove_wcorig (253:2) with device flush
Loading table for writecache_sanity-POOL_cvol (253:1).
Suppressed writecache_sanity-POOL_cvol (253:1) identical table reload.
Loading table for writecache_sanity-main_pvmove (253:0).
Suppressed writecache_sanity-main_pvmove (253:0) identical table reload.
Resuming writecache_sanity-pvmove0 (253:3).
Resuming writecache_sanity-POOL_cvol (253:1).
Resuming writecache_sanity-main_pvmove_wcorig (253:2).
Resuming writecache_sanity-main_pvmove (253:0).
Creating volume group backup "/etc/lvm/backup/writecache_sanity" (seqno 197).
activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/pvmove0.
Checking progress before waiting every 15 seconds.
/dev/sdd1: Moved: 4.30%
/dev/sdd1: Moved: 35.16%
/dev/sdd1: Moved: 66.02%
/dev/sdd1: Moved: 96.97%
/dev/sdd1: Moved: 100.00%
Polling finished successfully.
Marking Verified:Tested in the latest rpms. kernel-4.18.0-310.el8 BUILT: Thu May 27 14:24:00 CDT 2021 lvm2-2.03.12-3.el8 BUILT: Tue Jun 15 11:37:17 CDT 2021 lvm2-libs-2.03.12-3.el8 BUILT: Tue Jun 15 11:37:17 CDT 2021 SCENARIO - [pvmove_main_origin_but_call_fast_pool_device] Create an origin volume and attempt to pvmove by calling the fast pool device (Bug 1187290) *** Writecache info for this scenario *** * origin (slow): /dev/sde1 * pool (fast): /dev/sdh1 /dev/sdd1 ************************************ Adding "slow" and "fast" tags to corresponding pvs Create origin (slow) volume lvcreate --yes --wipesignatures y -L 4G -n main_pvmove writecache_sanity @slow Create writecache cvol (fast) volume using the same slow device lvcreate --yes -L 2G -n POOL writecache_sanity /dev/sdh1 Deactivate *ONLY* fast pool before conversion to write cache (SEE bug 1185347) Create writecached volume by combining the cache pool (fast) and origin (slow) volumes lvconvert --yes --type writecache --cachevol writecache_sanity/POOL writecache_sanity/main_pvmove Attempt to pvmove main origin volume but specify the fast POOL volume (/dev/sdh1) pvmove -n writecache_sanity/main_pvmove -v /dev/sdh1 /dev/sdb1 Archiving volume group "writecache_sanity" metadata (seqno 106). Creating logical volume pvmove0 Unable to move device used for writecache cachevol writecache_sanity/POOL_cvol. Pvmove main origin volume but specify the fast POOL volume (/dev/sdh1) as the device to move to pvmove -n writecache_sanity/main_pvmove -v /dev/sde1 /dev/sdh1 Archiving volume group "writecache_sanity" metadata (seqno 106). Creating logical volume pvmove0 activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove. activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove. Moving 1024 extents of logical volume writecache_sanity/main_pvmove_wcorig. activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/main_pvmove. Loading table for writecache_sanity-POOL_cvol (253:1). Suppressed writecache_sanity-POOL_cvol (253:1) identical table reload. Creating writecache_sanity-pvmove0 Loading table for writecache_sanity-pvmove0 (253:3). Loading table for writecache_sanity-main_pvmove_wcorig (253:2). Loading table for writecache_sanity-main_pvmove (253:0). Suppressed writecache_sanity-main_pvmove (253:0) identical table reload. Suspending writecache_sanity-main_pvmove (253:0) with device flush Suspending writecache_sanity-POOL_cvol (253:1) with device flush Suspending writecache_sanity-main_pvmove_wcorig (253:2) with device flush Loading table for writecache_sanity-POOL_cvol (253:1). Suppressed writecache_sanity-POOL_cvol (253:1) identical table reload. Loading table for writecache_sanity-main_pvmove (253:0). Suppressed writecache_sanity-main_pvmove (253:0) identical table reload. Resuming writecache_sanity-pvmove0 (253:3). Resuming writecache_sanity-POOL_cvol (253:1). Resuming writecache_sanity-main_pvmove_wcorig (253:2). Resuming writecache_sanity-main_pvmove (253:0). Creating volume group backup "/etc/lvm/backup/writecache_sanity" (seqno 107). activation/volume_list configuration setting not defined: Checking only host tags for writecache_sanity/pvmove0. Checking progress before waiting every 15 seconds. Polling finished successfully. Separating cache pool (lvconvert --yes --splitcache writecache_sanity/main_pvmove) from cache origin Removing cache pool writecache_sanity/POOL Removing cache origin volume writecache_sanity/main_pvmove lvremove -f /dev/writecache_sanity/main_pvmove Marking verified in the latest rpms as well. kernel-4.18.0-314.el8 BUILT: Tue Jun 15 11:04:32 CDT 2021 lvm2-2.03.12-4.el8 BUILT: Tue Jun 22 03:35:27 CDT 2021 lvm2-libs-2.03.12-4.el8 BUILT: Tue Jun 22 03:35:27 CDT 2021 [root@hayes-03 ~]# lvs -a -o +devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices [POOL_cvol] writecache_sanity Cwi-aoC--- 2.00g /dev/sdm1(0) main_pvmove writecache_sanity Cwi-a-C--- 4.00g [POOL_cvol] [main_pvmove_wcorig] 0.00 main_pvmove_wcorig(0) [main_pvmove_wcorig] writecache_sanity owi-aoC--- 4.00g /dev/sdn1(0) [root@hayes-03 ~]# pvmove -n writecache_sanity/main_pvmove -v /dev/sdm1 /dev/sdc1 Archiving volume group "writecache_sanity" metadata (seqno 48). Creating logical volume pvmove0 Unable to move device used for writecache cachevol writecache_sanity/POOL_cvol. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:4431 |