Bug 1888817
| Summary: | attempting to partial activate a writecache pool and origin with a missing pool device segfaults | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Corey Marthaler <cmarthal> |
| Component: | lvm2 | Assignee: | David Teigland <teigland> |
| lvm2 sub component: | Activating existing Logical Volumes | QA Contact: | cluster-qe <cluster-qe> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | agk, heinzm, jbrassow, mcsontos, msnitzer, prajnoha, zkabelac |
| Version: | 8.3 | ||
| Target Milestone: | rc | ||
| Target Release: | 8.0 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | lvm2-2.03.11-0.2.20201103git8801a86.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-05-18 15:01:53 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Corey Marthaler
2020-10-15 19:17:21 UTC
pushed to master: https://sourceware.org/git/?p=lvm2.git;a=commit;h=020d1edaa06e5e05b58e8d39530408f86a22d1b8 Missing a device in the fast LV: [root@null-04 ~]# lvs -a foo -o+devices WARNING: Couldn't find device with uuid IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr. WARNING: VG foo is missing PV IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr (last written to /dev/loop2). LV VG Attr LSize Pool Origin Devices [fast_cvol] foo Cwi---C-p- 100.00m [unknown](0) [fast_cvol] foo Cwi---C-p- 100.00m /dev/loop3(0) main foo Cwi---C--- 100.00m [fast_cvol] [main_wcorig] main_wcorig(0) [main_wcorig] foo owi---C--- 100.00m /dev/loop0(0) [main_wcorig] foo owi---C--- 100.00m /dev/loop1(0) [root@null-04 ~]# lvchange -ay foo/main WARNING: Couldn't find device with uuid IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr. WARNING: VG foo is missing PV IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr (last written to /dev/loop2). Cannot use partial or degraded activation with writecache. [root@null-04 ~]# lvchange -ay --partial foo/main PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr. WARNING: VG foo is missing PV IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr (last written to /dev/loop2). Cannot use partial or degraded activation with writecache. [root@null-04 ~]# lvchange -ay --activationmode degraded foo/main WARNING: Couldn't find device with uuid IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr. WARNING: VG foo is missing PV IA18ZC-kprI-tHKj-jatL-wyeX-ES2e-jVepAr (last written to /dev/loop2). Cannot use partial or degraded activation with writecache. Missing a device in the main LV: [root@null-04 ~]# lvs -a foo -o+devices WARNING: Couldn't find device with uuid tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd. WARNING: VG foo is missing PV tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd (last written to /dev/loop1). LV VG Attr LSize Pool Origin Devices [fast_cvol] foo Cwi---C--- 100.00m /dev/loop2(0) [fast_cvol] foo Cwi---C--- 100.00m /dev/loop3(0) main foo Cwi---C-p- 100.00m [fast_cvol] [main_wcorig] main_wcorig(0) [main_wcorig] foo owi---C-p- 100.00m /dev/loop0(0) [main_wcorig] foo owi---C-p- 100.00m [unknown](0) [root@null-04 ~]# lvchange -ay foo/main WARNING: Couldn't find device with uuid tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd. WARNING: VG foo is missing PV tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd (last written to /dev/loop1). Refusing activation of partial LV foo/main. Use '--activationmode partial' to override. [root@null-04 ~]# lvchange -ay --partial foo/main PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd. WARNING: VG foo is missing PV tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd (last written to /dev/loop1). Cannot use partial or degraded activation with writecache. [root@null-04 ~]# lvchange -ay --activationmode degraded foo/main WARNING: Couldn't find device with uuid tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd. WARNING: VG foo is missing PV tRv3rn-2bs9-Znhq-zvD7-RaqS-Z3f9-xcS8wd (last written to /dev/loop1). Refusing activation of partial LV foo/main. Use '--activationmode partial' to override. Fix verified in the latest 8.4 .d build. kernel-4.18.0-246.el8.dt2 BUILT: Mon Nov 9 07:22:41 CST 2020 lvm2-2.03.11-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 lvm2-libs-2.03.11-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 lvm2-lockd-2.03.11-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 device-mapper-1.02.175-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 device-mapper-libs-1.02.175-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 device-mapper-event-1.02.175-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 device-mapper-event-libs-1.02.175-0.2.20201103git8801a86.el8 BUILT: Wed Nov 4 07:04:46 CST 2020 SCENARIO - [recover_corrupt_writecache_pool_pv_w_pvck_dump] Create a writecache origin and pool, corrupt metadata on pool PV, and then restore the volume using pvck dump *** Writecache info for this scenario *** * origin (slow): /dev/sdh1 * pool (fast): /dev/sdf1 ************************************ Adding "slow" and "fast" tags to corresponding pvs Create origin (slow) volume lvcreate --yes --wipesignatures y -L 4G -n cworigin writecache_sanity @slow Create writecache cvol (fast) volumes lvcreate --yes -L 1G -n corrupt_pool_meta writecache_sanity @fast Deactivate *ONLY* fast pool before conversion to write cache (SEE bug 1185347) Create writecached volume by combining the cache pool (fast) and origin (slow) volumes lvconvert --yes --type writecache --cachesettings 'low_watermark=33 high_watermark=97 writeback_jobs=2860 autocommit_blocks=1250 autocommit_time=2668' --cachevol writecache_sanity/corrupt_pool_meta writecache_sanity/cworigin Corrupting PV /dev/sdf1 (used in this cache pool) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00101888 s, 1.0 GB/s Running vgck (bug 894136) WARNING: Couldn't find device with uuid EWIPRJ-R3eh-Q45L-IPow-1B7H-h52q-wuAvEk. WARNING: VG writecache_sanity is missing PV EWIPRJ-R3eh-Q45L-IPow-1B7H-h52q-wuAvEk (last written to /dev/sdf1). WARNING: Couldn't find all devices for LV writecache_sanity/corrupt_pool_meta_cvol while checking used and assumed devices. The volume group is missing 1 physical volumes. lvchange -an writecache_sanity WARNING: Couldn't find device with uuid EWIPRJ-R3eh-Q45L-IPow-1B7H-h52q-wuAvEk. WARNING: VG writecache_sanity is missing PV EWIPRJ-R3eh-Q45L-IPow-1B7H-h52q-wuAvEk (last written to /dev/sdf1). WARNING: Couldn't find all devices for LV writecache_sanity/corrupt_pool_meta_cvol while checking used and assumed devices. vgchange -ay --partial writecache_sanity PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid EWIPRJ-R3eh-Q45L-IPow-1B7H-h52q-wuAvEk. WARNING: VG writecache_sanity is missing PV EWIPRJ-R3eh-Q45L-IPow-1B7H-h52q-wuAvEk (last written to /dev/sdf1). Cannot use partial or degraded activation with writecache. Checking syslog to see if vgchange segfaulted Fix verified in the latest rpms. kernel-4.18.0-271.el8 BUILT: Fri Jan 8 03:32:43 CST 2021 lvm2-2.03.11-0.4.20201222gitb84a992.el8 BUILT: Tue Dec 22 06:33:49 CST 2020 lvm2-libs-2.03.11-0.4.20201222gitb84a992.el8 BUILT: Tue Dec 22 06:33:49 CST 2020 ============================================================ Iteration 4 of 4 started at Mon Jan 11 12:40:20 CST 2021 ============================================================ SCENARIO - [recover_corrupt_writecache_pool_pv_w_pvck_dump] Create a writecache origin and pool, corrupt metadata on pool PV, and then restore the volume using pvck dump *** Writecache info for this scenario *** * origin (slow): /dev/sde1 * pool (fast): /dev/sdb1 ************************************ ls: cannot access '/dev/writecache_sanity': No such file or directory Adding "slow" and "fast" tags to corresponding pvs Create origin (slow) volume lvcreate --yes --wipesignatures y -L 4G -n cworigin writecache_sanity @slow Create writecache cvol (fast) volumes lvcreate --yes -L 1G -n corrupt_pool_meta writecache_sanity @fast Deactivate *ONLY* fast pool before conversion to write cache (SEE bug 1185347) Create writecached volume by combining the cache pool (fast) and origin (slow) volumes lvconvert --yes --type writecache --cachesettings 'low_watermark=47 high_watermark=86 writeback_jobs=2274 autocommit_blocks=1919 autocommit_time=2974' --cachevol writecache_sanity/corrupt_pool_meta writecache_sanity/cworigin Corrupting PV /dev/sdb1 (used in this cache pool) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0030276 s, 346 MB/s Running vgck (bug 894136) WARNING: Couldn't find device with uuid WnbYAo-uv6z-Cz7P-FY9Y-MM0s-3gqI-4PEoer. WARNING: VG writecache_sanity is missing PV WnbYAo-uv6z-Cz7P-FY9Y-MM0s-3gqI-4PEoer (last written to /dev/sdb1). WARNING: Couldn't find all devices for LV writecache_sanity/corrupt_pool_meta_cvol while checking used and assumed devices. The volume group is missing 1 physical volumes. lvchange -an writecache_sanity WARNING: Couldn't find device with uuid WnbYAo-uv6z-Cz7P-FY9Y-MM0s-3gqI-4PEoer. WARNING: VG writecache_sanity is missing PV WnbYAo-uv6z-Cz7P-FY9Y-MM0s-3gqI-4PEoer (last written to /dev/sdb1). WARNING: Couldn't find all devices for LV writecache_sanity/corrupt_pool_meta_cvol while checking used and assumed devices. vgchange -ay --partial writecache_sanity PARTIAL MODE. Incomplete logical volumes will be processed. WARNING: Couldn't find device with uuid WnbYAo-uv6z-Cz7P-FY9Y-MM0s-3gqI-4PEoer. WARNING: VG writecache_sanity is missing PV WnbYAo-uv6z-Cz7P-FY9Y-MM0s-3gqI-4PEoer (last written to /dev/sdb1). Cannot use partial or degraded activation with writecache. Checking syslog to see if vgchange segfaulted Dumping headers and metadata from /dev/sde1 pvck --dump headers /dev/sde1 pvck --dump metadata /dev/sde1 -f /tmp/pvck_meta.215691 pvck --yes --repair -f /tmp/pvck_meta.215691 /dev/sdb1 WARNING: No LVM label found on /dev/sdb1. It may not be an LVM device. WARNING: no previous metadata areas found on device. vgreduce --removemissing writecache_sanity vgck --updatemetadata writecache_sanity Running vgck (bug 894136) Separating cache pool (lvconvert --yes --splitcache writecache_sanity/cworigin) from cache origin Removing cache pool writecache_sanity/corrupt_pool_meta Removing cache origin volume writecache_sanity/cworigin lvremove -f /dev/writecache_sanity/cworigin Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1659 |