Bug 1712983
| Summary: | "Internal error: LV segments corrupted" during syncing disk -> mirrored log conversion with supplied -R regionsize [rhel-7.9.z] | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Corey Marthaler <cmarthal> | ||||
| Component: | lvm2 | Assignee: | Heinz Mauelshagen <heinzm> | ||||
| lvm2 sub component: | Mirroring and RAID | QA Contact: | cluster-qe <cluster-qe> | ||||
| Status: | CLOSED WONTFIX | Docs Contact: | |||||
| Severity: | unspecified | ||||||
| Priority: | urgent | CC: | agk, heinzm, jbrassow, jreznik, mcsontos, msnitzer, prajnoha, zkabelac | ||||
| Version: | 7.7 | Keywords: | Reopened, ZStream | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | lvm2-2.02.187-6.el7_9.1.src.rpm | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-11-04 16:09:11 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Corey Marthaler
2019-05-22 15:53:17 UTC
Created attachment 1572095 [details]
verbose lvconvert attempt
This is *not* a regression and apparently existed in RHEL7.6 as well. [root@hayes-01 ~]# lvcreate --type mirror -m 1 -n mirror_log_convert --mirrorlog core -l 524465 -R 16 mirror_sanity /dev/sdk1 /dev/sdj1 /dev/sde1 /dev/sdd1 Logical volume "mirror_log_convert" created. [root@hayes-01 ~]# lvconvert --type mirror -m 1 --mirrorlog disk /dev/mirror_sanity/mirror_log_convert Logical volume mirror_sanity/mirror_log_convert converted. [root@hayes-01 ~]# lvs -a -o +devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices mirror_log_convert mirror_sanity mwi-a-m--- 2.00t [mirror_log_convert_mlog] 0.02 mirror_log_convert_mimage_0(0),mirror_log_convert_mimage_1(0) [mirror_log_convert_mimage_0] mirror_sanity Iwi-aom--- 2.00t /dev/sdj1(0) [mirror_log_convert_mimage_0] mirror_sanity Iwi-aom--- 2.00t /dev/sdk1(0) [mirror_log_convert_mimage_1] mirror_sanity Iwi-aom--- 2.00t /dev/sde1(0) [mirror_log_convert_mimage_1] mirror_sanity Iwi-aom--- 2.00t /dev/sdd1(0) [mirror_log_convert_mlog] mirror_sanity lwi-aom--- 4.00m /dev/sdh1(0) [root@hayes-01 ~]# lvconvert --type mirror -m 1 --mirrorlog mirrored /dev/mirror_sanity/mirror_log_convert WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead. WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead. LV mirror_log_convert_mlog, segment 1 invalid: region size is bigger then LV itself for mirror segment. Internal error: LV segments corrupted in mirror_log_convert_mlog. 3.10.0-957.el7.x86_64 lvm2-2.02.180-8.el7 BUILT: Mon Sep 10 04:45:22 CDT 2018 lvm2-libs-2.02.180-8.el7 BUILT: Mon Sep 10 04:45:22 CDT 2018 package lvm2-cluster is not installed device-mapper-1.02.149-8.el7 BUILT: Mon Sep 10 04:45:22 CDT 2018 device-mapper-libs-1.02.149-8.el7 BUILT: Mon Sep 10 04:45:22 CDT 2018 device-mapper-event-1.02.149-8.el7 BUILT: Mon Sep 10 04:45:22 CDT 2018 device-mapper-event-libs-1.02.149-8.el7 BUILT: Mon Sep 10 04:45:22 CDT 2018 device-mapper-persistent-data-0.7.3-3.el7 BUILT: Tue Nov 14 05:07:18 CST 2017 It appears the the -R regionsize in the original create is key to reproducing this issue. Corey,
works here with the following el7 versions.
Please reopen in case of reproducer.
# lvm version
LVM version: 2.02.185(2)-RHEL7 (2019-05-13)
Library version: 1.02.158-RHEL7 (2019-05-13)
Driver version: 4.37.1
Configuration: ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=
/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/l
ib --mandir=/usr/share/man --infodir=/usr/share/info --with-default-dm-run-dir=/run --with-default-run-dir=/run/lvm --with-default-pid-dir=/run --with-default-locking-dir=/run/lock/lvm
--with-usrlibdir=/usr/lib64 --enable-lvm1_fallback --enable-fsadm --with-pool=internal --enable-write_install --with-user= --with-group= --with-device-uid=0 --with-device-gid=6 --with
-device-mode=0660 --enable-pkgconfig --enable-applib --enable-cmdlib --enable-dmeventd --enable-blkid_wiping --enable-python2-bindings --with-cluster=internal --with-clvmd=corosync --e
nable-cmirrord --with-udevdir=/usr/lib/udev/rules.d --enable-udev_sync --with-thin=internal --enable-lvmetad --with-cache=internal --enable-lvmpolld --enable-lvmlockd-dlm --enable-lvml
ockd-sanlock --enable-dmfilemapd
# uname -r
3.10.0-1062.el7.x86_64
# vgcreate t /dev/sd*
Volume group "t" successfully created
# vgs t
VG #PV #LV #SN Attr VSize VFree
t 64 0 0 wz--n- 63.75g 63.75g
# lvcreate --ty mirror -m1 -nm --mirrorlog core -L2g -R 16 t
Logical volume "m" created.
# lvs t
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
m t mwi-a-m--- 2.00g 100.00
# lvconvert -m1 --mirrorlog mirrored t/m
WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead.
WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead.
Logical volume t/m converted.
# lvs -ao+devices t
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
m t mwi-a-m--- 2.00g [m_mlog] 100.00 m_mimage_0(0),m_mimage_1(0)
[m_mimage_0] t iwi-aom--- 2.00g /dev/sda(0)
[m_mimage_0] t iwi-aom--- 2.00g /dev/sdab(0)
[m_mimage_0] t iwi-aom--- 2.00g /dev/sdad(0)
[m_mimage_1] t iwi-aom--- 2.00g /dev/sdaa(0)
[m_mimage_1] t iwi-aom--- 2.00g /dev/sdac(0)
[m_mimage_1] t iwi-aom--- 2.00g /dev/sdae(0)
[m_mlog] t mwi-aom--- 4.00m 100.00 m_mlog_mimage_0(0),m_mlog_mimage_1(0)
[m_mlog_mimage_0] t iwi-aom--- 4.00m /dev/sdy(0)
[m_mlog_mimage_1] t iwi-aom--- 4.00m /dev/sdz(0)
I'm still able to hit this with the latest 7.9 rpms. 3.10.0-1153.el7.x86_64 lvm2-2.02.187-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 lvm2-libs-2.02.187-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 lvm2-cluster-2.02.187-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 lvm2-lockd-2.02.187-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 lvm2-python-boom-1.2-2.el7 BUILT: Mon Jun 29 07:03:49 CDT 2020 cmirror-2.02.187-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 device-mapper-1.02.170-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 device-mapper-libs-1.02.170-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 device-mapper-event-1.02.170-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 device-mapper-event-libs-1.02.170-6.el7 BUILT: Mon Jun 29 07:02:09 CDT 2020 [root@harding-02 ~]# pvscan PV /dev/mapper/mpatha1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] PV /dev/mapper/mpathb1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] PV /dev/mapper/mpathc1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] PV /dev/mapper/mpathd1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] PV /dev/mapper/mpathe1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] PV /dev/mapper/mpathf1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] PV /dev/mapper/mpathg1 VG mirror_sanity lvm2 [249.96 GiB / 249.96 GiB free] [root@harding-02 ~]# lvcreate --type mirror -m 1 --mirrorlog core -R 16 -n convert -L 600M mirror_sanity Logical volume "convert" created. [root@harding-02 ~]# lvconvert --type mirror -m 1 --mirrorlog disk mirror_sanity/convert Logical volume mirror_sanity/convert converted. [root@harding-02 ~]# lvconvert --type mirror -m 1 --mirrorlog mirrored mirror_sanity/convert WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead. WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead. LV convert_mlog, segment 1 invalid: region size is bigger then LV itself for mirror segment. Internal error: LV segments corrupted in convert_mlog. Corey, missed the conversion to disk upfront the one to mirrorlog which makes the difference causing the error. Also, when using regionsize 8m, conversion to mirrorlog's not even being performed from disk at all... lvm commit ids: master -> 286a793c12aac1fde17cb9768bacad660e1dfeb3 stable-2.02 -> 7ebe1bc901c3ed9404f92c7415c7c491593aed47 Doesn't appear to be fixed in the latest 7.9.z rpms. Future fixes should include devel test results. 3.10.0-1160.el7.x86_64 lvm2-2.02.187-6.el7_9.2 BUILT: Mon Oct 26 15:21:13 CDT 2020 lvm2-libs-2.02.187-6.el7_9.2 BUILT: Mon Oct 26 15:21:13 CDT 2020 lvm2-cluster-2.02.187-6.el7_9.2 BUILT: Mon Oct 26 15:21:13 CDT 2020 [root@mckinley-01 ~]# vgs VG #PV #LV #SN Attr VSize VFree mirror_sanity 7 0 0 wz--n- <1.71t <1.71t rhel_mckinley-01 1 3 0 wz--n- <556.75g 0 [root@mckinley-01 ~]# lvcreate --type mirror -m 1 --mirrorlog core -R 16 -n convert -L 600M mirror_sanity Logical volume "convert" created. [root@mckinley-01 ~]# lvconvert --type mirror -m 1 --mirrorlog disk mirror_sanity/convert Logical volume mirror_sanity/convert converted. [root@mckinley-01 ~]# lvconvert --type mirror -m 1 --mirrorlog mirrored mirror_sanity/convert WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead. WARNING: Log type "mirrored" is DEPRECATED and will be removed in the future. Use RAID1 LV or disk log instead. device-mapper: reload ioctl on (253:22) failed: Invalid argument Failed to lock logical volume mirror_sanity/convert. [root@mckinley-01 ~]# echo $? 5 Oct 29 13:27:29 mckinley-01 kernel: device-mapper: dirty region log: invalid region size 32768 Oct 29 13:27:29 mckinley-01 kernel: device-mapper: table: 253:22: mirror: Error creating mirror dirty log Oct 29 13:27:29 mckinley-01 kernel: device-mapper: ioctl: error adding target to table |