Description of problem: The code in PR https://github.com/rear/rear/pull/1806 (BZ #1450667) added support for restoring Thin volumes into ReaR 2.4. Unfortunately it appears it doesn't work as planned. 1. When restoring the exact same system (which already had the VG containing the thin volumes configured at the time of the restore), the code used to delete the Thin pool after using vgcfgrestore is failing due to line 111 not returning an error, making line 112 return an error now usr/share/rear/layout/prepare/GNU/Linux/110_include_lvm_code.sh: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- 105 lvm lvs --noheadings -o lv_name,vg_name,lv_layout | while read lv vg layout ; do 106 # Consider LVs for our VG only 107 [ \$vg == "$vg" ] || continue 108 # Consider Thin Pools only 109 [[ ,\$layout, == *,thin,* ]] && [[ ,\$layout, == *,pool,* ]] || continue 110 # Remove twice, the first time it fails because of thin volumes in the pool 111 lvm lvremove -q -f -y \$vg/\$lv || true <-- HERE 112 lvm lvremove -q -f -y \$vg/\$lv 113 done -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- At the time PR1806 it was not the case, so there must have been some changed in LVM ... 2. When restoring a new system with same disk capacity (hence with clean new disk, this can be emulated by using "pvremove --force --force /dev/sdaX"), "vgcfgrestore" run but then "lvm lvremove" (line 111) fails with an unrecoverable error -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- +++ lvm lvremove -q -f -y rhel/pool00 Thin pool rhel-pool00-tpool (253:2) transaction_id is 10, while expected 6. Failed to update pool rhel/pool00. -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- Version-Release number of selected component (if applicable): rear-2.4 including Upstream ReaR How reproducible: Always Steps to Reproduce: 1. Install a system with LVM Thin Provisioning and multiple volumes, keep the swap as a standard LVM volume /: thin /var: thin /home: thin /tmp: thin swap: linear 2. Create a rescue ISO 3. Recover the system Actual results: See above Expected results: Additional info: It looks like to me "vgcfgrestore" cannot be used at all as soon as there is a Thin volume. I'm pretty sure it was working at the time PR1806 was submitted (June 2018), I need to test this again. For the demo, I've added a linear swap device because additionally, in scenario 2 ("When restoring a new system"), the swap volume gets restored by vgcfgrestore but doesn't get activated, causing an error when formatting the swap (/dev/vg/swap is not present ...), this is yet an additional issue...
The swap issue seems to happen in scenario 1 as well, due to vgcfgrestore not having enabled the VG it created. +++ mkswap -U 487d9f6f-93cc-4c6d-be48-a65d21ab1af8 /dev/mapper/rhel-lvswap /dev/mapper/rhel-lvswap: No such file or directory
For swap issue, a vgchange is missing on line 107: Upstream Code /usr/share/rear/layout/prepare/GNU/Linux/110_include_lvm_code.sh: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- 95 if lvm vgcfgrestore -f "$VAR_DIR/layout/lvm/${vg}.cfg" $vg >&2 ; then 96 lvm vgchange --available y $vg >&2 97 98 LogPrint "Sleeping 3 seconds to let udev or systemd-udevd create their devices..." 99 sleep 3 >&2 100 create_volume_group=0 101 create_logical_volumes=0 102 103 # 104 # It failed ... restore layout using 'vgcfgrestore --force', but then remove Thin volumes, they are b roken 105 # 106 elif lvm vgcfgrestore --force -f "$VAR_DIR/layout/lvm/${vg}.cfg" $vg >&2 ; then 107 108 lvm lvs --noheadings -o lv_name,vg_name,lv_layout | while read lv vg layout ; do ... -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- Indeed, even though vgcfgrestore failed, some volumes have been created and can be safely activated. Apparently RHEL7.5 GA also failed: +++ lvm lvremove -q -f -y rhel/pool00 Thin pool rhel-pool00-tpool (252:3) transaction_id is 6, while expected 4. Failed to update pool rhel/pool00. # lvm version LVM version: 2.02.177(2)-RHEL7 (2018-01-22) Library version: 1.02.146-RHEL7 (2018-01-22) Driver version: 4.37.0 So it looks like it never worked properly ...
Lvm2 (vgcfgrestore) can't be used this way to restore thin-volumes. Main purpose of vgcfgrestore (with mandatory --force option) is to let users fix existing thin-pool. The only proposal for recreating thinLVs on a newly created thin-pool is to call individual lvcreate commands so kernel thin-pool metadata can have the information about volumes as well. So if this is meant to be used for 'data recovery' - the tool recovering LVs need to create them with proper sizes.
Would it be feasible to avoid using vgcfgrestore and do the same thing as in migration mode always (or at least always with thin pools)? Properties of segments could be preserved by using lvextend, as in bz1732328#c24.
zkabelac, I think it is a bug (in ReaR).
Thanks Zdenek for looking into this. I will be pushing your feedback to ReaR developers so that we come with a proper solution.
By the way, I think the idea was not to restore thin-volumes this way (using vgcfgrestore), but to restore all other volumes (than thin ones), delete the broken thin ones, and recreate them using lvcreate. This is apparently beacuse vgcfgrestore does not have a way to restore the non-thin volumes only.
I'm not familiar with usage of 'rear' tool - if it's only 'backup' tool for device content - it might be out-of-scope for this tool to be able recreate same sort of storage layout - i.e. imagine thinLV on thin-pool this has cached raid6 thin-pool data & raid1 thin-pool metadata - who would be supposed to restore such device stack ? Anyway it might be possible RFE on 'rear' - but it probably needs different title.
Reopening since component is ReaR, not LVM2, and ReaR needs a fix. Opened Issue https://github.com/rear/rear/issues/2222 for that Upstream.
(In reply to Zdenek Kabelac from comment #8) > I'm not familiar with usage of 'rear' tool - if it's only 'backup' tool for > device content - it might be out-of-scope for this tool to be able recreate > same sort of storage layout - > > i.e. imagine thinLV on thin-pool this has cached raid6 thin-pool data & > raid1 thin-pool metadata - who would be supposed to restore such device > stack ? It is a recovery tool, so its main responsibility is to backup and recreate the layout, not the content.
Deletion of thin-pool works as designed with 'double --force' even for damaged thin-pools. The problem is 'restoring' same thin-pool with matching thinID & transactionID. We we likely need to make some 'new lvm2' command to somehow support recreation of all LVs from scratch. (as vgcfgbackup/restore relies on the proper disk content)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (rear bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:4344