RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 917102 - request for clarification: thinpool stacked on raid volume experiences meta device failure
Summary: request for clarification: thinpool stacked on raid volume experiences meta ...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: lvm2
Version: 6.4
Hardware: x86_64
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Zdenek Kabelac
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On: 1015096
Blocks: 960054
TreeView+ depends on / blocked
 
Reported: 2013-03-01 17:35 UTC by Corey Marthaler
Modified: 2015-06-23 15:35 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-03 20:26:15 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Corey Marthaler 2013-03-01 17:35:53 UTC 
Description of problem:
Now that I've added thin pool stacking to the raid volume device failure tests, I'm not sure which cases are expected to fail. 

If I have any raid volume that is converted to a thin pool (with a non raid meta device), if that meta device fails, any file systems on top of any virt volumes are expected to then be corrupt, right?

So in the following case if /dev/sdb1 is failed...

RAID Structure(s):
  LV                                     Attr      LSize   Cpy%Sync Devices
  raid10            rwi-a-r-- 504.00m     0.00
  [raid10_rimage_0] Iwi-aor-- 168.00m          /dev/sdb1(1)
  [raid10_rimage_1] Iwi-aor-- 168.00m          /dev/sdd1(1)
  [raid10_rimage_2] Iwi-aor-- 168.00m          /dev/sdg1(1)
  [raid10_rimage_3] Iwi-aor-- 168.00m          /dev/sdf1(1)
  [raid10_rimage_4] Iwi-aor-- 168.00m          /dev/sdh1(1)
  [raid10_rimage_5] Iwi-aor-- 168.00m          /dev/sdc1(1)
  [raid10_rmeta_0]  ewi-aor--   4.00m          /dev/sdb1(0)
  [raid10_rmeta_1]  ewi-aor--   4.00m          /dev/sdd1(0)
  [raid10_rmeta_2]  ewi-aor--   4.00m          /dev/sdg1(0)
  [raid10_rmeta_3]  ewi-aor--   4.00m          /dev/sdf1(0)
  [raid10_rmeta_4]  ewi-aor--   4.00m          /dev/sdh1(0)
  [raid10_rmeta_5]  ewi-aor--   4.00m          /dev/sdc1(0)

Convert raid volume(s) to Thinpool volume(s) on taft-02...
lvcreate -n meta_raid10 -L 200M black_bird
lvconvert --thinpool black_bird/raid10 --poolmetadata meta_raid10
lvcreate --virtualsize 200M --thinpool black_bird/raid10 -n virt_raid10

RAID (converted to thin pool) structure(s):
  LV                                           Attr      LSize   Cpy%Sync Devices
  raid10                  twi-a-tz- 504.00m          raid10_tdata(0)
  [raid10_tdata]          rwi-aot-- 504.00m   100.00 
  [raid10_tdata_rimage_0] iwi-aor-- 168.00m          /dev/sdb1(1)
  [raid10_tdata_rimage_1] iwi-aor-- 168.00m          /dev/sdd1(1)
  [raid10_tdata_rimage_2] iwi-aor-- 168.00m          /dev/sdg1(1)
  [raid10_tdata_rimage_3] iwi-aor-- 168.00m          /dev/sdf1(1)
  [raid10_tdata_rimage_4] iwi-aor-- 168.00m          /dev/sdh1(1)
  [raid10_tdata_rimage_5] iwi-aor-- 168.00m          /dev/sdc1(1)
  [raid10_tdata_rmeta_0]  ewi-aor--   4.00m          /dev/sdb1(0)
  [raid10_tdata_rmeta_1]  ewi-aor--   4.00m          /dev/sdd1(0)
  [raid10_tdata_rmeta_2]  ewi-aor--   4.00m          /dev/sdg1(0)
  [raid10_tdata_rmeta_3]  ewi-aor--   4.00m          /dev/sdf1(0)
  [raid10_tdata_rmeta_4]  ewi-aor--   4.00m          /dev/sdh1(0)
  [raid10_tdata_rmeta_5]  ewi-aor--   4.00m          /dev/sdc1(0)
**[raid10_tmeta]          ewi-aot-- 200.00m          /dev/sdb1(43) **


# device /dev/sdb1 was then failed

kernel: EXT3-fs (dm-18): error: ext3_journal_start_sb: Detected aborted journal
kernel: EXT3-fs (dm-18): error: remounting filesystem read-only
xinetd[1859]: EXIT: qarsh status=0 pid=29329 duration=0(sec)
kernel: __ratelimit: 970 callbacks suppressed
kernel: Buffer I/O error on device dm-18, logical block 259
kernel: lost page write due to I/O error on dm-18
[...]
kernel: __ratelimit: 246 callbacks suppressed
kernel: Buffer I/O error on device dm-18, logical block 32283
kernel: lost page write due to I/O error on dm-18
lvm[1255]: Failed to parse status.
lvm[1255]: Unmounting thin volume black_bird-raid10-tpool from /mnt/raid10.


Is this just a case where the test should not be attempting this?


Version-Release number of selected component (if applicable):
2.6.32-354.el6.x86_64

lvm2-2.02.98-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
lvm2-libs-2.02.98-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
lvm2-cluster-2.02.98-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
udev-147-2.43.el6    BUILT: Thu Oct 11 05:59:38 CDT 2012
device-mapper-1.02.77-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
device-mapper-libs-1.02.77-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
device-mapper-event-1.02.77-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
device-mapper-event-libs-1.02.77-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
cmirror-2.02.98-9.el6    BUILT: Wed Jan 23 10:06:55 CST 2013
Comment 1 Zdenek Kabelac 2013-03-05 08:58:59 UTC 
We need to clearly document, how to deal with errors on stacked devices.
Comment 5 Zdenek Kabelac 2013-10-07 12:26:47 UTC 
We will generate KB articles dealing with partial problems.

More generic bugzilla dealing with thin pool recover issue is now BZ #1015096.
Comment 6 Alasdair Kergon 2013-10-14 23:22:11 UTC 
This needs to be covered by the documentation being produced for bug 1015096.  There will be no change to the lvm2 package for this at this stage.
Comment 9 Jonathan Earl Brassow 2014-08-19 00:40:36 UTC 
To answer the question, it is expected that I/O to any of the thin-pool's thinLVs would fail in this case.

I cannot tell you for sure if corruption would ensue.  I suspect that if the metadata device could be brought back, the thin-pool would be usable again and there would be no corruption.  You would need to check with the kernel maintainer for sure though.

If the metadata device does not come back, then you have lost all your pointers to where the data is with no good way of recoverying them - a complete loss.

This is why the documentation in 'lvmthin.7' is so emphatic about using RAID (at least) for the metadata device.  See the section entitled "Tolerate device failures using raid".
Comment 11 Jonathan Earl Brassow 2015-03-03 20:26:15 UTC 
(In reply to Jonathan Earl Brassow from comment #9)
> To answer the question, it is expected that I/O to any of the thin-pool's
> thinLVs would fail in this case.
> 
> I cannot tell you for sure if corruption would ensue.  I suspect that if the
> metadata device could be brought back, the thin-pool would be usable again
> and there would be no corruption.  You would need to check with the kernel
> maintainer for sure though.
> 
> If the metadata device does not come back, then you have lost all your
> pointers to where the data is with no good way of recoverying them - a
> complete loss.
> 
> This is why the documentation in 'lvmthin.7' is so emphatic about using RAID
> (at least) for the metadata device.  See the section entitled "Tolerate
> device failures using raid".

Given that the question has been answered, I am closing this as NOTABUG.  If more information is needed, you can reopen.
Note You need to log in before you can comment on or make changes to this bug.