Bug 1518121
Summary: | multisegment RAID1, allocator uses one disk for both legs | |||
---|---|---|---|---|
Product: | [Community] LVM and device-mapper | Reporter: | Marian Csontos <mcsontos> | |
Component: | lvm2 | Assignee: | Heinz Mauelshagen <heinzm> | |
lvm2 sub component: | Mirroring and RAID | QA Contact: | cluster-qe <cluster-qe> | |
Status: | POST --- | Docs Contact: | ||
Severity: | high | |||
Priority: | high | CC: | agk, cmarthal, heinzm, jbrassow, prajnoha, steved424, zkabelac | |
Version: | 2.02.176 | Keywords: | Triaged | |
Target Milestone: | --- | Flags: | rule-engine:
lvm-technical-solution?
rule-engine: lvm-test-coverage? |
|
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | If docs needed, set a value | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 2204467 2204480 (view as bug list) | Environment: | ||
Last Closed: | Type: | Bug | ||
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 2204467, 2204480 |
Description
Marian Csontos
2017-11-28 09:11:10 UTC
In case of failure such device can not be repaired: # lvconvert --repair vg/r1 WARNING: Disabling lvmetad cache for repair command. WARNING: Not using lvmetad because of repair. /dev/vg/r1: read failed after 0 of 4096 at 6442385408: Input/output error /dev/vg/r1: read failed after 0 of 4096 at 6442442752: Input/output error Couldn't find device with uuid UFK7K0-nGPE-76Rq-F5WC-xGig-UzXP-MnFuDz. Attempt to replace failed RAID images (requires full device resync)? [y/n]: y Unable to replace all PVs from vg/r1 at once. Failed to replace faulty devices in vg/r1. TODO: Test with devices in sync. Workaround: `lvcreate -n r1 -m 1 -L 6G vg /dev/sd[cab]` Waited for sync. 100% in sync unable to repair. Also it is possible to read only first 4 GBs, reading from the second segment fails. Read from first segment: # dd if=/dev/vg/r1 of=/dev/null skip=4000 count=1 bs=1M 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00195199 s, 537 MB/s Read from second segment fails: # dd if=/dev/vg/r1 of=/dev/null skip=5000 count=1 bs=1M dd: error reading ‘/dev/vg/r1’: Input/output error 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.00201181 s, 0.0 kB/s Surprisingly the RAID1 device sanity reports DA: # dmsetup status vg-r1_rmeta_1: 0 8192 linear vg-r1_rimage_1: 0 8372224 linear vg-r1_rimage_1: 8372224 4210688 linear vg-t2: 0 8388608 linear vg-r1_rmeta_0: 0 8192 linear vg-r1_rimage_0: 0 8372224 linear vg-r1_rimage_0: 8372224 4210688 linear vg-t1: 0 8388608 linear vg_stacker_OTzj-root: 0 12582912 linear vg-r1: 0 12582912 raid raid1 2 DA 12582912/12582912 idle 0 0 - And for reference only the segments: # lvs --segments -aolv_name,pe_ranges,le_ranges WARNING: Not using lvmetad because a repair command was run. /dev/vg/r1: read failed after 0 of 4096 at 6442385408: Input/output error /dev/vg/r1: read failed after 0 of 4096 at 6442442752: Input/output error Couldn't find device with uuid jDRXQI-jGSW-BAOG-LB3h-aLhd-8fPb-kHbPZf. LV PE Ranges LE Ranges r1 r1_rimage_0:0-1535 r1_rimage_1:0-1535 [r1_rimage_0]:0-1535,[r1_rimage_1]:0-1535 [r1_rimage_0] [unknown]:1-1022 [unknown]:1-1022 [r1_rimage_0] /dev/sdb:1024-1537 /dev/sdb:1024-1537 [r1_rimage_1] /dev/sda:1025-2046 /dev/sda:1025-2046 [r1_rimage_1] [unknown]:1023-1536 [unknown]:1023-1536 [r1_rmeta_0] [unknown]:0-0 [unknown]:0-0 [r1_rmeta_1] /dev/sda:1024-1024 /dev/sda:1024-1024 I've just been bitten by this for second time, though I swore I specified PVs manually. 2.02.176 (-4.1ubuntu3) on Ubuntu 18.04. I also hit a whole load of scrub errors last night - the data stored in the fs seems fine, it looks like when I extended the RAID1 LV in question, the extensions didn't get synced. Still investigating that one - may have been triggered by me trying to work around this bug. Any thoughts / progress? This behaves like allocation policy 'anywhere' was applied. Reworking the allocator gains importance! Creating with 'cling' allocation policy avoids the problem with existing t[12] LVs allocated on sd[ab] and free sdc: # lvcreate -y -nr1 -l251 -m1 vg Logical volume "r1" created. # lvs --noh -aoname,segperanges vg r1 r1_rimage_0:0-250 r1_rimage_1:0-250 [r1_rimage_0] /dev/sdc:1-126 <--- [r1_rimage_0] /dev/sdb:128-252 [r1_rimage_1] /dev/sda:129-254 [r1_rimage_1] /dev/sdc:127-251 <--- bogus collocation with rimage_0 [r1_rmeta_0] /dev/sdc:0-0 [r1_rmeta_1] /dev/sda:128-128 t1 /dev/sda:0-127 t2 /dev/sdb:0-127 # lvremove -y vg/r1 Logical volume "r1" successfully removed # lvcreate -y --alloc cling -nr1 -l100%FREE -m1 vg # could alternatively give the extent count, same result Logical volume "r1" created. # lvs --noh -aoname,segperanges vg r1 r1_rimage_0:0-125 r1_rimage_1:0-125 [r1_rimage_0] /dev/sdc:1-126 [r1_rimage_1] /dev/sda:129-254 [r1_rmeta_0] /dev/sdc:0-0 [r1_rmeta_1] /dev/sda:128-128 t1 /dev/sda:0-127 t2 /dev/sdb:0-127 # lvextend -l+1 vg/r1 Extending 2 mirror images. Insufficient suitable allocatable extents for logical volume r1: 2 more required Now alloc_anywhere is needed to make use of the existing space and listing sdc, sdb in that sequence (sdb, sdc does not work) to avoid collocation: # lvextend -l+1 vg/r1 Extending 2 mirror images. Insufficient suitable allocatable extents for logical volume r1: 2 more required # lvextend -y --alloc anywhere -l+1 vg/r1 Extending 2 mirror images. Size of logical volume vg/r1 changed from 504.00 MiB (126 extents) to 508.00 MiB (127 extents). Logical volume vg/r1 successfully resized. # lvs --noh -aoname,segperanges vg r1 r1_rimage_0:0-126 r1_rimage_1:0-126 [r1_rimage_0] /dev/sdc:1-126 <--- [r1_rimage_0] /dev/sdb:128-128 [r1_rimage_1] /dev/sda:129-254 [r1_rimage_1] /dev/sdc:127-127 <--- Bogus collocation again [r1_rmeta_0] /dev/sdc:0-0 [r1_rmeta_1] /dev/sda:128-128 t1 /dev/sda:0-127 In this case, reducing the raid1 in size or pvmove'ing the collocated extents off to another unrelated PV are options. # lvreduce -fy -l-1 vg/r1 WARNING: Reducing active logical volume to 504.00 MiB. THIS MAY DESTROY YOUR DATA (filesystem etc.) Size of logical volume vg/r1 changed from 508.00 MiB (127 extents) to 504.00 MiB (126 extents). Logical volume vg/r1 successfully resized. # lvextend -y -l+1 --alloc anywhere vg/r1 /dev/sdc /dev/sdb Extending 2 mirror images. Size of logical volume vg/r1 changed from 504.00 MiB (126 extents) to 508.00 MiB (127 extents). Logical volume vg/r1 successfully resized. # lvs --noh -aoname,segperanges vg r1 r1_rimage_0:0-126 r1_rimage_1:0-126 [r1_rimage_0] /dev/sdc:1-127 [r1_rimage_1] /dev/sda:129-254 [r1_rimage_1] /dev/sdb:128-128 [r1_rmeta_0] /dev/sdc:0-0 [r1_rmeta_1] /dev/sda:128-128 t1 /dev/sda:0-127 t2 /dev/sdb:0-127 Fixed in commit 05c2b10c5d0a99993430ffbcef684a099ba810ad |