Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
When raid LV is created and then loses a PV during the sync process, the tagging with lvchange does not work, nor does it display a message.
It exits with code 5
Version-Release number of selected component (if applicable):
lvm2-2.02.98-6.el6.x86_64
lvm2-libs-2.02.98-6.el6.x86_64
device-mapper-1.02.77-6.el6.x86_64
device-mapper-libs-1.02.77-6.el6.x86_64
kernel-2.6.32-347.el6.x86_64
How reproducible:
Create raid LV and while it is still syncing, kill one device. try tagging the LV.
Steps to Reproduce:
vgcreate vg /dev/sd{a..d}1
lvcreate vg -n raid5 -L16G --type raid5 -i4
echo "offline" > /sys/block/sdb/device/state
lvchange --addtag tagged vg/raid5
Actual results:
/dev/sdb1: read failed after 0 of 1024 at 10733879296: Input/output error
/dev/sdb1: read failed after 0 of 1024 at 10733948928: Input/output error
/dev/sdb1: read failed after 0 of 1024 at 0: Input/output error
/dev/sdb1: read failed after 0 of 1024 at 4096: Input/output error
(05:18:43) [root@r6-node02:~]$ echo $?
5
Expected results:
To successfully add a tag since this works without lvmetad.
This is output of the same sequence with lvm2-lvmetad off:
(06:03:28) [root@r6-node02:~]$ vgcreate vg /dev/sd{a..e}1 && lvcreate vg -n raid5 -L16G --type raid5 -i4 && lvs -a -o +devices && echo "offline" > /sys/block/sdb/device/state && lvchange vg/raid5 --addtag tagged
Volume group "vg" successfully created
Using default stripesize 64.00 KiB
Logical volume "raid5" created
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert Devices
lv_root VolGroup -wi-ao--- 7.54g /dev/vda2(0)
lv_swap VolGroup -wi-ao--- 1.97g /dev/vda2(1930)
raid5 vg rwi-a-r-- 16.00g 0.00 raid5_rimage_0(0),raid5_rimage_1(0),raid5_rimage_2(0),raid5_rimage_3(0),raid5_rimage_4(0)
[raid5_rimage_0] vg Iwi-aor-- 4.00g /dev/sda1(1)
[raid5_rimage_1] vg Iwi-aor-- 4.00g /dev/sdb1(1)
[raid5_rimage_2] vg Iwi-aor-- 4.00g /dev/sdc1(1)
[raid5_rimage_3] vg Iwi-aor-- 4.00g /dev/sdd1(1)
[raid5_rimage_4] vg Iwi-aor-- 4.00g /dev/sde1(1)
[raid5_rmeta_0] vg ewi-aor-- 4.00m /dev/sda1(0)
[raid5_rmeta_1] vg ewi-aor-- 4.00m /dev/sdb1(0)
[raid5_rmeta_2] vg ewi-aor-- 4.00m /dev/sdc1(0)
[raid5_rmeta_3] vg ewi-aor-- 4.00m /dev/sdd1(0)
[raid5_rmeta_4] vg ewi-aor-- 4.00m /dev/sde1(0)
/dev/sdb1: read failed after 0 of 1024 at 10733879296: Input/output error
/dev/sdb1: read failed after 0 of 1024 at 10733948928: Input/output error
/dev/sdb1: read failed after 0 of 1024 at 0: Input/output error
/dev/sdb1: read failed after 0 of 1024 at 4096: Input/output error
/dev/sdb1: read failed after 0 of 2048 at 0: Input/output error
Couldn't find device with uuid rDfpaz-hr8Y-a7D5-HP6E-U3TU-Mq32-XrE2vu.
Logical volume "raid5" changed
Additional info:
Not sure if this is of any consequence, but as soon as a device is removed from raid, the syncing jumps to 100% as if finished.
Don't know if this is actually a feature in case a device dies during the sync process.
This happens regardless of lvmetad.
Comment 2RHEL Program Management
2012-12-25 06:48:29 UTC
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Created attachment 676382[details]
lvchange -vvvv --addtag tagged vg/raid5
Added the verbose output of lvchange -vvvv --addtag tagged vg/raid5 to attachments.
Comment 5Jonathan Earl Brassow
2013-01-10 22:55:56 UTC
Here's more info on the sync jump:
[root@bp-01 ~]# devices vg
LV Cpy%Sync Devices
lv 0.00 lv_rimage_0(0),lv_rimage_1(0),lv_rimage_2(0)
[lv_rimage_0] /dev/sda1(1)
[lv_rimage_1] /dev/sdb1(1)
[lv_rimage_2] /dev/sdc1(1)
[lv_rmeta_0] /dev/sda1(0)
[lv_rmeta_1] /dev/sdb1(0)
[lv_rmeta_2] /dev/sdc1(0)
[root@bp-01 ~]# dmsetup status vg-lv; off.sh sdb; sleep 5; dmsetup status vg-lv
0 104857600 raid raid5_ls 3 aaa 6327704/52428800
Turning off sdb
0 104857600 raid raid5_ls 3 ADA 52428800/52428800
[root@bp-01 ~]# on.sh sdb
Turning on sdb
[root@bp-01 ~]# lvchange -an vg/lv; lvchange -ay vg/lv; dmsetup status vg-lv
0 104857600 raid raid5_ls 3 AaA 6907904/52428800
[root@bp-01 ~]# devices vg
LV Cpy%Sync Devices
lv 18.52 lv_rimage_0(0),lv_rimage_1(0),lv_rimage_2(0)
[lv_rimage_0] /dev/sda1(1)
[lv_rimage_1] /dev/sdb1(1)
[lv_rimage_2] /dev/sdc1(1)
[lv_rmeta_0] /dev/sda1(0)
[lv_rmeta_1] /dev/sdb1(0)
[lv_rmeta_2] /dev/sdc1(0)
So, the correct non-100% sync status is properly stored in the RAID superblock. I'm not sure how harmless this is... I'm not even sure if it makes sense to talk about "sync" when a device is missing in RAID5 if it hasn't already sync'ed. MD may be saying, "Hey, I can't perform sync anymore; therefore 100%". When the device is restored, it continues the sync.
So the deactivation and activation of LV is needed to get the sychronization to continue?
Otherwise, even though the device is back the status will remain ADAAA
--repair will probably not work in this case even if the disk missing shows back up on the same spot, since it will try to replace the device by another if I understand correctly?
Metadata for VG can't be change if VG is inconsistent.
The only support transition is to 'fix' VG.
(vgreduce --removemissing)
lvm2 can't be sure what adding tag may cause - so first the admin has to decide whet he want to do - either put PV back, or fix VG.
lvm2 doesn't support metadata updates for 'transiently' missing device (and it's quite hard problem I'd say - so if there would be such request to support this kind of thing - it would need very good justification IMHO).
So far it works as designed - you can't update any metadata of inconsistent VG - with inconistent VG you can only use limited set of lvm2 commands.
Description of problem: When raid LV is created and then loses a PV during the sync process, the tagging with lvchange does not work, nor does it display a message. It exits with code 5 Version-Release number of selected component (if applicable): lvm2-2.02.98-6.el6.x86_64 lvm2-libs-2.02.98-6.el6.x86_64 device-mapper-1.02.77-6.el6.x86_64 device-mapper-libs-1.02.77-6.el6.x86_64 kernel-2.6.32-347.el6.x86_64 How reproducible: Create raid LV and while it is still syncing, kill one device. try tagging the LV. Steps to Reproduce: vgcreate vg /dev/sd{a..d}1 lvcreate vg -n raid5 -L16G --type raid5 -i4 echo "offline" > /sys/block/sdb/device/state lvchange --addtag tagged vg/raid5 Actual results: /dev/sdb1: read failed after 0 of 1024 at 10733879296: Input/output error /dev/sdb1: read failed after 0 of 1024 at 10733948928: Input/output error /dev/sdb1: read failed after 0 of 1024 at 0: Input/output error /dev/sdb1: read failed after 0 of 1024 at 4096: Input/output error (05:18:43) [root@r6-node02:~]$ echo $? 5 Expected results: To successfully add a tag since this works without lvmetad. This is output of the same sequence with lvm2-lvmetad off: (06:03:28) [root@r6-node02:~]$ vgcreate vg /dev/sd{a..e}1 && lvcreate vg -n raid5 -L16G --type raid5 -i4 && lvs -a -o +devices && echo "offline" > /sys/block/sdb/device/state && lvchange vg/raid5 --addtag tagged Volume group "vg" successfully created Using default stripesize 64.00 KiB Logical volume "raid5" created LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert Devices lv_root VolGroup -wi-ao--- 7.54g /dev/vda2(0) lv_swap VolGroup -wi-ao--- 1.97g /dev/vda2(1930) raid5 vg rwi-a-r-- 16.00g 0.00 raid5_rimage_0(0),raid5_rimage_1(0),raid5_rimage_2(0),raid5_rimage_3(0),raid5_rimage_4(0) [raid5_rimage_0] vg Iwi-aor-- 4.00g /dev/sda1(1) [raid5_rimage_1] vg Iwi-aor-- 4.00g /dev/sdb1(1) [raid5_rimage_2] vg Iwi-aor-- 4.00g /dev/sdc1(1) [raid5_rimage_3] vg Iwi-aor-- 4.00g /dev/sdd1(1) [raid5_rimage_4] vg Iwi-aor-- 4.00g /dev/sde1(1) [raid5_rmeta_0] vg ewi-aor-- 4.00m /dev/sda1(0) [raid5_rmeta_1] vg ewi-aor-- 4.00m /dev/sdb1(0) [raid5_rmeta_2] vg ewi-aor-- 4.00m /dev/sdc1(0) [raid5_rmeta_3] vg ewi-aor-- 4.00m /dev/sdd1(0) [raid5_rmeta_4] vg ewi-aor-- 4.00m /dev/sde1(0) /dev/sdb1: read failed after 0 of 1024 at 10733879296: Input/output error /dev/sdb1: read failed after 0 of 1024 at 10733948928: Input/output error /dev/sdb1: read failed after 0 of 1024 at 0: Input/output error /dev/sdb1: read failed after 0 of 1024 at 4096: Input/output error /dev/sdb1: read failed after 0 of 2048 at 0: Input/output error Couldn't find device with uuid rDfpaz-hr8Y-a7D5-HP6E-U3TU-Mq32-XrE2vu. Logical volume "raid5" changed Additional info: Not sure if this is of any consequence, but as soon as a device is removed from raid, the syncing jumps to 100% as if finished. Don't know if this is actually a feature in case a device dies during the sync process. This happens regardless of lvmetad.