Description of problem: This happens when the original LV has logical block size 512 and physical block size 4096 (because it's created on devs with LBS 512 and PBS 4096), and then integrity is added to the active LV (doesn't happen if integrity is added while the LV is inactive). LVM selects an integrity block size of 4096 to match the xfs sector size and physical block size. This causes subsequent mount to fail with kernel errors: [14442.753210] XFS: xfs_readsb sector_size 512 BTOBB 1 [14442.758096] device-mapper: integrity: Bio not aligned on 8 sectors: 0x0, 0x1 [14442.765135] md/raid1:mdX: dm-4: rescheduling sector 0 [14442.770201] device-mapper: integrity: Bio not aligned on 8 sectors: 0x0, 0x1 [14442.777245] device-mapper: integrity: Bio not aligned on 8 sectors: 0x0, 0x1 [14442.784286] md/raid1:mdX: Disk failure on dm-4, disabling device. md/raid1:mdX: Operation continuing on 1 devices. [14442.796008] md/raid1:mdX: redirecting sector 0 to other mirror: dm-6 [14442.802381] device-mapper: integrity: Bio not aligned on 8 sectors: 0x0, 0x1 [14442.809457] XFS (dm-7): SB validate failed with error -5 sector_size 512. mount won't succeed until the LBS reported by the device is updated to 4096. lvchange --refresh doesn't change that, but deactivating the activating the LV does. Manually selecting an integrity block size of 512 (overriding lvm's select of 4096) avoids the problem. I will probably change LVM's default block size selection to work around this issue until it can be resolved. [root@null-04 ~]# lvcreate --type raid1 -n lv1 -L1G test Logical volume "lv1" created. [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# mkfs.xfs -f /dev/test/lv1 meta-data=/dev/test/lv1 isize=512 agcount=4, agsize=65536 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=262144, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@null-04 ~]# mount /dev/test/lv1 /mnt [root@null-04 ~]# echo "hi" > /mnt/hi [root@null-04 ~]# umount /mnt [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# lvconvert --raidintegrity y test/lv1 Creating integrity metadata LV lv1_rimage_0_imeta with size 12.00 MiB. Logical volume "lv1_rimage_0_imeta" created. Creating integrity metadata LV lv1_rimage_1_imeta with size 12.00 MiB. Logical volume "lv1_rimage_1_imeta" created. Using integrity block size 4096 for file system block size 4096. Logical volume test/lv1 has added integrity. [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# mount /dev/test/lv1 /mnt mount: /mnt: can't read superblock on /dev/mapper/test-lv1. [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# lvchange -an test/lv1 [root@null-04 ~]# lvchange -ay test/lv1 [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 4096 4096 [root@null-04 ~]# mount /dev/test/lv1 /mnt [root@null-04 ~]# cat /mnt/hi hi [root@null-04 ~]# umount /mnt Using --raidintegrityblocksize 512 avoids the problem: [root@null-04 ~]# lvcreate --type raid1 -n lv1 -L1G test Logical volume "lv1" created. [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# mkfs.xfs -f /dev/test/lv1 meta-data=/dev/test/lv1 isize=512 agcount=4, agsize=65536 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=262144, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@null-04 ~]# mount /dev/test/lv1 /mnt [root@null-04 ~]# echo "hi" > /mnt/hi [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# umount /mnt [root@null-04 ~]# lvconvert --raidintegrity y --raidintegrityblocksize 512 test/lv1 Creating integrity metadata LV lv1_rimage_0_imeta with size 12.00 MiB. Logical volume "lv1_rimage_0_imeta" created. Creating integrity metadata LV lv1_rimage_1_imeta with size 12.00 MiB. Logical volume "lv1_rimage_1_imeta" created. Using integrity block size 512 for file system block size 4096. Logical volume test/lv1 has added integrity. [root@null-04 ~]# blockdev --getss --getpbsz /dev/test/lv1 512 4096 [root@null-04 ~]# mount /dev/test/lv1 /mnt Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
I test this by creating the VG on ramdisk devs which have LBS 512 and PBS 4096: # blockdev --getss --getpbsz /dev/ram0 512 4096
It seems to me that increasing the logical block size under an active LV could be problematic in general, and what we see from xfs may be indicative of other applications. They expect a certain io size or alignment to be usable while the device remains active. If the block device allows 512 byte size|alignment at one moment, and then requires 4096 byte size|alignment the next, it could be challenging to adapt to that in the application. So, my approach to this when it comes to adding integrity is if integrity is added to an active LV, avoid choosing an integrity block size that would result in the logical block size of the device increasing. In practice the impact would be: if integrity is added to an active LV that has LBS 512 and a file system block size larger than 512, lvm would choose integrity block size of 512 rather than a larger size to match the file system block size. The --raidintegrityblocksize option would allow overriding this restriction.
I reproduced this bug and I've found out that if you run this dmsetup table /dev/test/lv1 | dmsetup load /dev/test/lv1 dmsetup suspend /dev/test/lv1; dmsetup resume /dev/test/lv1 then the sector size will be set to 4k. So, you can reload the raid target like this from lvm. I wouldn't recommend using 512-byte sector size on integrity and writecache, because it makes them slow.
Avoiding the problem is the workaround for now in master: https://sourceware.org/git/?p=lvm2.git;a=commit;h=48872b0369af5ee048c947586d7da82d9769f5c3 I have it printing a message to give the user an idea that they are being subjected to this limitation: # lvconvert --raidintegrity y test/lv1 Creating integrity metadata LV lv1_rimage_0_imeta with size 12.00 MiB. Logical volume "lv1_rimage_0_imeta" created. Creating integrity metadata LV lv1_rimage_1_imeta with size 12.00 MiB. Logical volume "lv1_rimage_1_imeta" created. Limiting integrity block size to 512 because the LV is active. Using integrity block size 512 for file system block size 4096. Logical volume test/lv1 has added integrity. (I would like to be able to rework the raid activation code to allow this, but that libdm activation code can be very difficult to figure out and get working correctly. From my limited experience, it can be an exercise in figuring out all the existing special case workarounds, and then adding another special case workaround to work around the others. We can perhaps use another bz that will let others assess the problem and prioritize the work to fix it.)
I have monitored such exercises which include working with statistics on demographics and other things about population and society. I have noticed that my biggest problem is working with correct data entry. I usually analyze the concepts like national integrity and so on through various mathematical and statistical experiments. Such assignments include math integrity, and I'm having trouble with this chapter too. Fortunately, I found a source that is https://graduateway.com/essay-examples/integrity/ to obtain and input reliable data about social integrity into my exercises. I'm not saying it's facile, but it's much easier to read essay examples with explanations about the experiment I have to analyze and then go through my technical steps. A student must have, at least, some support services during his work.