Bug 1240549 - Display ancestors and descendants for thin snapshot/origin LVs even after dependency chain is broken after removing some LVs
Summary: Display ancestors and descendants for thin snapshot/origin LVs even after dep...
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: lvm2   
(Show other bugs)
Version: 7.2
Hardware: All
OS: Linux
medium
low
Target Milestone: rc
: ---
Assignee: Peter Rajnoha
QA Contact: cluster-qe@redhat.com
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: 1313485 1163895 1295577
TreeView+ depends on / blocked
 
Reported: 2015-07-07 08:30 UTC by Peter Rajnoha
Modified: 2017-03-29 07:05 UTC (History)
11 users (show)

Fixed In Version: lvm2-2.02.152-1.el7
Doc Type: Enhancement
Doc Text:
LVM can now track and display thin snapshot logical volumes that have been removed You can now configure your system to track thin snapshot logical volumes that have been removed by enabling the `record_lvs_history` metadata option in the `lvm.conf` configuration file. This allows you to display a full thin snapshot dependency chain that includes logical volumes that have been removed from the original dependency chain and have become historical logical volumes. The full dependency chain, including historical LVs, can be displayed with new lv_full_ancestors and lv_full_descendants reporting fields. For information on configuring and displaying historical logical volumes, see https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Logical_Volume_Manager_Administration/LV.html#historical_volumes
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-04 04:09:58 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1445 normal SHIPPED_LIVE lvm2 bug fix and enhancement update 2016-11-03 13:46:41 UTC

Description Peter Rajnoha 2015-07-07 08:30:20 UTC
This is part of solution for original bug #1163895 so all the parts can be properly tested.

Just like we have bug #1240518 to display existing dependency chain for thin snapshot/origin LVs, make it possible to display the chain even after removal of some LVs.

Add new fields lv_ancestors_full and lv_descendants_full to display this dependency chain where removed LVs are marked appropriately (for example with "-" prefix).

This requires an extension to metadata to allow for storing removed LVs and to determine their inclusion in existing dependency chain when this dependency is reported using the new lv_ancestors_full and lv_descendants_full fields.

Also, make it possible to configure maximum number of removed LVs to remember to limit this so the metadata size does not grow indefinitely.

Comment 1 Peter Rajnoha 2015-09-02 11:35:34 UTC
The code is in this devel branch:
https://git.fedorahosted.org/cgit/lvm2.git/log/?h=dev-prajnoha-track-removed-lvs

Comment 2 Peter Rajnoha 2015-09-02 13:31:28 UTC
It's quite late in the process now - the patchset grew more than I expected - it's adding proper internal interface to handle removed entries (which allows for them to be processed in a very similar way as live entries, making the code consistent).

Also, this patchset introduces changes to metadata (new section to keep records about removed LVs). The change may be risky and it's better to have this tested more in Fedora/rawhide first before adding this into RHEL.

Though the code in devel branch is finished, it needs thorough review and testing. I'm moving this to 7.3.

Comment 4 Roman Bednář 2016-02-10 10:28:23 UTC
Adding QA ACK for 7.3. 

Test as shown in 1240518.

Comment 5 Peter Rajnoha 2016-03-14 13:20:14 UTC
The patchset is now upstream since lvm2 v2.02.145.

Summary of changes (taken from WHATS_NEW):

  - Make it possible to use lvremove and lvrename on historical LVs. 
  - For historical LVs, report 'none' for lv_layout and 'history' for lv_role.
  - Add full_{ancestors,descendants} fields to report LV ancestry with history.
  - Report (h)istorical state within 5th bit (State) of the lv_attr field.
  - Add lv_historical reporting field to report if LV is historical or not. 
  - Add lv_time_removed reporting field to display removal time for hist. LVs. 
  - Report lv_name, lv_uuid, vg_name, lv_time for historical LVs. 
  - Add --nohistory switch to lvremove to disable history recording on demand.
  - Add -H|--history switch to lvs and lvdisplay to include historical LVs. 
  - Create historical LVs out of removed thin snapshot LVs and record in history.
  - Add metadata/lvs_history_retention_time for automatic removal of hist. LVs. 
  - Add metadata/record_lvs_history config for switching LV history recording.
  - Add support and infrastructure for tracking historical LVs.

For example:

# vgcreate vg /dev/sda
  Volume group "vg" successfully created

[1] fedora/~ # lvcreate -l100%FREE -T vg/pool
  Logical volume "pool" created.

[1] fedora/~ # lvcreate -V1 -T vg/pool
  Rounding up size to full physical extent 4.00 MiB
  Logical volume "lvol1" created.

[1] fedora/~ # lvcreate -s vg/lvol1
  Logical volume "lvol2" created.

[1] fedora/~ # lvcreate -s vg/lvol2
  Logical volume "lvol3" created.

[1] fedora/~ # lvcreate -s vg/lvol3
  Logical volume "lvol4" created.

[1] fedora/~ # lvcreate -s vg/lvol3
  Logical volume "lvol5" created.

(reporint thin snapshot chain - full_ancestors == ancestors and full_descendants == descendants still)

[1] fedora/~ # lvs -o name,full_ancestors,full_descendants vg
  LV    FAncestors        FDescendants           
  lvol1                   lvol2,lvol3,lvol4,lvol5
  lvol2 lvol1             lvol3,lvol4,lvol5      
  lvol3 lvol2,lvol1       lvol4,lvol5            
  lvol4 lvol3,lvol2,lvol1                        
  lvol5 lvol3,lvol2,lvol1                        
  pool   

[1] fedora/~ # lvs -o name,ancestors,descendants vg
  LV    Ancestors         Descendants            
  lvol1                   lvol2,lvol3,lvol4,lvol5
  lvol2 lvol1             lvol3,lvol4,lvol5      
  lvol3 lvol2,lvol1       lvol4,lvol5            
  lvol4 lvol3,lvol2,lvol1                        
  lvol5 lvol3,lvol2,lvol1                        
  pool              

(removing vg/lvol3 will cause lvol3 to become "historical LV")

[1] fedora/~ # lvremove -ff vg/lvol3
  Logical volume "lvol3" successfully removed

(reporting LVs, not including historical LVs, but the original chain is still preserved - that's difference from the older lvs -o ancestors,descendants)

[1] fedora/~ # lvs -o name,ancestors,descendants vg
  LV    Ancestors Descendants
  lvol1           lvol2      
  lvol2 lvol1                
  lvol4                      
  lvol5                      
  pool  

[1] fedora/~ # lvs -o name,full_ancestors,full_descendants vg
  LV    FAncestors  FDescendants     
  lvol1             lvol2,lvol4,lvol5
  lvol2 lvol1       lvol4,lvol5      
  lvol4 lvol2,lvol1                  
  lvol5 lvol2,lvol1                  
  pool      

(using -H to include historical LVs in report)

[1] fedora/~ # lvs -H -o name,full_ancestors,full_descendants vg
  LV     FAncestors         FDescendants            
  lvol1                     lvol2,-lvol3,lvol4,lvol5
  lvol2  lvol1              -lvol3,lvol4,lvol5      
  -lvol3 lvol2,lvol1        lvol4,lvol5             
  lvol4  -lvol3,lvol2,lvol1                         
  lvol5  -lvol3,lvol2,lvol1                         
  pool    

(if there's no live LV following in the chain, historical LV is removed automatically from history)

[1] fedora/~ # lvremove -ff vg/lvol4
  Automatically removing historical logical volume vg/-lvol4.
  Logical volume "lvol4" successfully removed

(historical LVs can be referenced by vgname/-lvname, there's also 'h' in 5th bit of the lv_attr field for historical LVs)

[1] fedora/~ # lvs -H vg/-lvol3
  LV     VG   Attr       LSize 
  -lvol3 vg   ----h-----    0  

(historical LVs can be removed directly by lvremove)

[1] fedora/~ # lvremove -ff vg/-lvol3
  Historical logical volume "lvol3" successfully removed

(and if the removed historical LVs is the very last historical LV that preserved the chain, the chain is now broken because we don't have history for the chain anymore)

[1] fedora/~ # lvs -o name,full_ancestors,full_descendants vg
  LV    FAncestors FDescendants
  lvol1            lvol2       
  lvol2 lvol1                  
  lvol4                        
  lvol5                        
  pool  

(let's create snapshot for vg/lvol5 now)

[1] fedora/~ # lvcreate -s vg/lvol5
  Logical volume "lvol6" created.

(we can report removal time for historical LVs)

[1] fedora/~ # lvs -H -o name,full_ancestors,full_descendants,time_removed vg
  LV     FAncestors FDescendants RTime                     
  lvol1             lvol2                                  
  lvol2  lvol1                                             
  lvol4                                                    
  -lvol5            lvol6        2016-03-14 14:14:32 +0100 
  lvol6  -lvol5                                            
  pool 

(now, setting metadata/lvs_history_retention_time=10 - that means keep history records for 10 seconds per each history item)

[1] fedora/~ # lvmconfig metadata/lvs_history_retention_time
lvs_history_retention_time=10

(see that the current time is more than 10 seconds after removal time)

[1] fedora/~ # date
Mon Mar 14 14:16:31 CET 2016

(...and now any VG write operation that follows will remove outdated historical LVs automatically - I'll use vgchange --addtag which certainly changes metadata, but any command doing VG write will do this. In this case the historiacl LV vg/-lvol5 is removed automtically from history)

[1] fedora/~ # lvs -H -o name,full_ancestors,full_descendants vg
  LV    FAncestors FDescendants
  lvol1            lvol2       
  lvol2 lvol1                  
  lvol4                        
  lvol6                        
  pool

Comment 6 Mike McCune 2016-03-28 23:21:47 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions

Comment 8 Roman Bednář 2016-07-20 13:27:06 UTC
Marking as verified in lvm2-2.02.160-1.el7.x86_64


1) Change lvm.conf
Enable metadata section and set following options:

metadata {  
        record_lvs_history = 1
        lvs_history_retention_time = 3600
}

2) Testing layout
# lvs -o lv_name,full_descendants,full_ancestors,lv_layout,lv_role,lv_attr vg
  LV    FDescendants            FAncestors        Layout      Role                                                           Attr                       
  lvol1 lvol2,lvol3,lvol4,lvol5                   thin,sparse public,origin,thinorigin                                       Vwi-a-tz--             
  lvol2 lvol3,lvol4,lvol5       lvol1             thin,sparse public,origin,thinorigin,snapshot,thinsnapshot                 Vwi---tz-k            
  lvol3 lvol4,lvol5             lvol2,lvol1       thin,sparse public,origin,thinorigin,multithinorigin,snapshot,thinsnapshot Vwi---tz-k            
  lvol4                         lvol3,lvol2,lvol1 thin,sparse public,snapshot,thinsnapshot                                   Vwi---tz-k             
  lvol5                         lvol3,lvol2,lvol1 thin,sparse public,snapshot,thinsnapshot                                   Vwi---tz-k             
  pool                                            thin,pool   private                                                        twi-aotz--            

3) Descendants and ancestors matching full_descandants and full_ancestors
# lvs -o lv_name,descendants,ancestors vg
  LV    Descendants             Ancestors
  lvol1 lvol2,lvol3,lvol4,lvol5                   
  lvol2 lvol3,lvol4,lvol5       lvol1             
  lvol3 lvol4,lvol5             lvol2,lvol1      
  lvol4                         lvol3,lvol2,lvol1 
  lvol5                         lvol3,lvol2,lvol1 
  pool                                          

4) Remove LV
# lvremove -ff vg/lvol3
  Logical volume "lvol3" successfully removed
# lvs -o lv_name,descendants,ancestors vg
  LV    Descendants Ancestors              
  lvol1 lvol2                  
  lvol2             lvol1          
  lvol4                             
  lvol5                           
  pool                              

5) Check full_{ascendants|descendants} contain full LV listing
# lvs -o lv_name,full_descendants,full_ancestors vg
  LV    FDescendants      FAncestors            
  lvol1 lvol2,lvol4,lvol5             
  lvol2 lvol4,lvol5       lvol1      
  lvol4                   lvol2,lvol1 
  lvol5                   lvol2,lvol1 
  pool                                 

6) Check full_{ascendants|descendants} contain historical LVs with -H|--history option 
# lvs -H -o lv_name,full_descendants,full_ancestors,lv_layout,lv_role,lv_attr,lv_historical vg
  LV     FDescendants             FAncestors         Layout      Role                         Attr       Historical                      
  lvol1  lvol2,-lvol3,lvol4,lvol5                    thin,sparse public,origin,thinorigin     Vwi-a-tz--            
  lvol2  -lvol3,lvol4,lvol5       lvol1              thin,sparse public,snapshot,thinsnapshot Vwi---tz-k             
  -lvol3 lvol4,lvol5              lvol2,lvol1        none        public,history               ----h----- historical 
  lvol4                           -lvol3,lvol2,lvol1 thin,sparse public                       Vwi---tz-k            
  lvol5                           -lvol3,lvol2,lvol1 thin,sparse public                       Vwi---tz-k            
  pool                                               thin,pool   private                      twi-aotz--            

7) lv_time_removed attribute
# lvs -H -o lv_name,full_descendants,full_ancestors,lv_layout,lv_role,lv_attr,lv_historical,lv_time_removed vg
  LV     FDescendants             FAncestors         Layout      Role                         Attr       Historical RTime                     
  lvol1  lvol2,-lvol3,lvol4,lvol5                    thin,sparse public,origin,thinorigin     Vwi-a-tz--                                      
  lvol2  -lvol3,lvol4,lvol5       lvol1              thin,sparse public,snapshot,thinsnapshot Vwi---tz-k                                      
  -lvol3 lvol4,lvol5              lvol2,lvol1        none        public,history               ----h----- historical 2016-07-19 12:06:08 +0200 
  lvol4                           -lvol3,lvol2,lvol1 thin,sparse public                       Vwi---tz-k                                      
  lvol5                           -lvol3,lvol2,lvol1 thin,sparse public                       Vwi---tz-k                                      
  pool                                               thin,pool   private                      twi-aotz--      

8) lvremove --nohistory option
# lvremove --nohistory vg/lvol2 -ff
  Logical volume "lvol2" successfully removed
# lvs --history -o lv_name,full_descendants,full_ancestors,lv_layout,lv_role,lv_attr,lv_historical,lv_time_removed vg
  LV     FDescendants FAncestors Layout      Role           Attr       Historical RTime                     
  lvol1                          thin,sparse public         Vwi-a-tz--                                      
  -lvol3 lvol4,lvol5             none        public,history ----h----- historical 2016-07-19 12:06:08 +0200 
  lvol4               -lvol3     thin,sparse public         Vwi---tz-k                                      
  lvol5               -lvol3     thin,sparse public         Vwi---tz-k                                      
  pool                           thin,pool   private        twi-aotz--  

Notes: 
I) lvs history is turned off by default to prevent filling up metadata area
II) historical LVs acquire a hyphen as a prefix (e.g. '-lvol1')
III) historical LVs get removed once the lvs_history_retention_time is expired
IV) historical LVs can also be removed manually by lvremove (see man lvremove for more)
V) lvremove on historical LVs also removes all lvs in the chain if no existing ancestor or descendant is linked to them

example:
# lvs -o lv_name,full_descendants,full_ancestors vg -H
  LV     FDescendants        FAncestors          
  -lvol1 -lvol2,-lvol3,lvol4                     
  -lvol2 -lvol3,lvol4        -lvol1              
  -lvol3 lvol4               -lvol2,-lvol1       
  lvol4                      -lvol3,-lvol2,-lvol1
  pool                                           
# lvremove vg/lvol4 -ff
  Automatically removing historical logical volume vg/-lvol1.
  Automatically removing historical logical volume vg/-lvol2.
  Automatically removing historical logical volume vg/-lvol3.
  Automatically removing historical logical volume vg/-lvol4.
  Logical volume "lvol4" successfully removed

Comment 10 errata-xmlrpc 2016-11-04 04:09:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1445.html


Note You need to log in before you can comment on or make changes to this bug.