Bug 1777364

Summary: Rebase lvm2-python-boom to upstream version 1.1 (with kernel caching)
Product: Red Hat Enterprise Linux 7 Reporter: Bryn M. Reeves <bmr>
Component: lvm2Assignee: Bryn M. Reeves <bmr>
lvm2 sub component: Other QA Contact: cluster-qe <cluster-qe>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: agk, cmarthal, heinzm, jbrassow, mcsontos, msnitzer, prajnoha, pvlasin, rhandlin, zkabelac
Version: 7.7Keywords: Rebase
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: lvm2-2.02.187-3.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-29 19:55:48 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1784876    

Description Bryn M. Reeves 2019-11-27 13:53:39 UTC 
This bug was initially created as a copy of Bug #1771584

RHEL7 currently includes boom-0.8 which included basic functionality to create boot entries for snapshot volumes. The upstream boom-1.0 release includes more complete support for the Red Hat extensions to BLS including the ability to enable authentication for boom managed entries and more automatic configuration for users of Red Hat distributions:

  * Support for --grub-users, --grub-arg, --grub-class, and --grub-id extensions supported by Red Hat Grub2 builds

  * Automatic configuration of --uname-pattern for RHEL and Fedora distributions (used to select an OS profile based on the kernel version string)

  * Automatic enablement of Red Hat Grub2/BLS extensions on RHEL and Fedora


The release also includes new features and fixes not present in the 0.8 snapshot including the ability to define host profiles to customise boot entries on a per-installation basis, better support for systems that activate multiple logical volumes at boot time and the ability to add and remove custom kernel command line options.
Comment 6 Jonathan Earl Brassow 2019-12-13 22:11:54 UTC 
Bryn,

Since we will be waiting to put updated BOOM into rhel7.9, I've reassigned this bug, changed the flags, and changed the goal (and summary) to include BOOM 1.1 with caching support.

The other way to do this would be to close this bug and open a new one with BOOM + caching support for RHEL7.9.
Comment 18 Bryn M. Reeves 2020-05-22 16:06:34 UTC 
commit d788f82746826d091abcdf76b5f4884c22e3b7de (HEAD -> master, origin/master)
Author: Bryn M. Reeves <bmr>
Date:   Wed May 20 15:14:27 2020 +0100

    boom.cache: use correct variable in load_cache() warning branch
    
    Signed-off-by: Bryn M. Reeves <bmr>

diff --git a/boom/cache.py b/boom/cache.py
index 1dc5e07..cf7de47 100644
--- a/boom/cache.py
+++ b/boom/cache.py
@@ -238,7 +238,7 @@ def load_cache(verify=True, digests=False):
     images = cachedata["images"]
     for image_id in images.keys():
         if image_id not in ids:
-            _log_warn("Found orphan image_id '%s'" % image)
+            _log_warn("Found orphan image_id '%s'" % image_id)
             # clean up?
 
     _log_debug("Loaded %d cache paths and %d images" %
Comment 19 Bryn M. Reeves 2020-05-22 16:55:50 UTC 
Just to clarify: the patch in comment #18 is not required for RHEL7: it addresses a bug in a warning branch that detects a specific type of corruption in the cache metadata (an "orphan" image present in the file system but not indexed in the metadata).

Outside file system corruption or external modification of the cacheindex.json metadata it is not possible for this condition to occur in RHEL7: if a customer were to encounter it for one of those reasons then the resolution is very straightforward (removing the orphan file from /boot/boom/cache). There is no possibility of data corruption or loss as a result of this bug.
Comment 20 Corey Marthaler 2020-05-22 19:20:56 UTC 
Verified that --uname-pattern from host works (unlike in 8.2 bug 1819010) in the latest build as well as standard regression test scenarios. 

* Still need to check for "Support for --grub-users, --grub-arg, --grub-class, and --grub-id extensions supported by Red Hat Grub2 builds" *

[root@harding-03 ~]# boom profile create --from-host 
Created profile with os_id f150f3d:
  OS ID: "f150f3d6693495254255d46e20ecf5c690ec3262",
  Name: "Red Hat Enterprise Linux Server", Short name: "rhel",
  Version: "7.9 (Maipo)", Version ID: "7.9",
  Kernel pattern: "/vmlinuz-%{version}", Initramfs pattern: "/initramfs-%{version}.img",
  Root options (LVM2): "rd.lvm.lv=%{lvm_root_lv}",
  Root options (BTRFS): "rootflags=%{btrfs_subvolume}",
  Options: "root=%{root_device} ro %{root_opts}",
  Title: "%{os_name} %{os_version_id} (%{version})",
  Optional keys: "grub_users grub_arg grub_class id", UTS release pattern: "el7"

SCENARIO - [boom_sanity]
Create a snapshot and perform some boom sanity only checks as this wont be an actual bootable snapshot
Installing package lvm2-python-boom on harding-03...
Package lvm2-python-boom installed on node harding-03.
Making origin volume
lvcreate -L 300M snapper -n origin
Making snapshot of origin volume
lvcreate -s /dev/snapper/origin -c 128 -n boom_snap -L 100M

Check 0 a. Verify a new profile can be created (this is basicall a dup of the above cmd)
boom profile create --from-host --uname-pattern Linux
Check 0 b. Verify a new profile can be created automatically from just current host heuristic
boom profile create --from-host
Check 0 c. Verify a new profile can be created
boom profile create --from-host --uname-pattern el8

Check 1. Verify boom not configured for grub yet warning (Bug 1542952)
Boom already configured for grub. Skipping check.

Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-1141.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-1141.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-a41ec8c891a84179a7716bce7b6e893e
Found initrd image: /boot/initramfs-0-rescue-a41ec8c891a84179a7716bce7b6e893e.img
done

Check 2 a. Verify version numbers not matching an OS profile provide a helpful warning (Bug 1771584#c5)
boom create --title nomatchversion --rootlv snapper/boom_snap --version 2.5.0
Check 2 b. Verify fake OS profiles are properly identified (Bug 1540266)
boom create --profile fake_profile --title title --version 3.10-272.el7 --rootlv snapper/boom_snap

Check 3. Verify --rootlv and --root-device varations are both accepted (Bug 1543188)
--rootlv: boom create --title 1 --rootlv /dev/snapper/boom_snap
boom list ccb55fe
BootID  Version                  Name                            RootDevice            
ccb55fe 3.10.0-1141.el7.x86_64   Red Hat Enterprise Linux Server /dev/snapper/boom_snap
boom entry show ccb55fe
Boot Entry (boot_id=ccb55fe)
  title 1
  machine-id a41ec8c891a84179a7716bce7b6e893e
  version 3.10.0-1141.el7.x86_64
  linux /vmlinuz-3.10.0-1141.el7.x86_64
  initrd /initramfs-3.10.0-1141.el7.x86_64.img
  options root=/dev/snapper/boom_snap ro rd.lvm.lv=snapper/boom_snap
  grub_users $grub_users
  grub_arg kernel
  grub_class --unrestricted
boom delete ccb55fe

--root-lv: boom create --title 2 --root-lv /dev/snapper/boom_snap
boom list 507b4ff
BootID  Version                  Name                            RootDevice            
507b4ff 3.10.0-1141.el7.x86_64   Red Hat Enterprise Linux Server /dev/snapper/boom_snap
boom entry show 507b4ff
Boot Entry (boot_id=507b4ff)
  title 2
  machine-id a41ec8c891a84179a7716bce7b6e893e
  version 3.10.0-1141.el7.x86_64
  linux /vmlinuz-3.10.0-1141.el7.x86_64
  initrd /initramfs-3.10.0-1141.el7.x86_64.img
  options root=/dev/snapper/boom_snap ro rd.lvm.lv=snapper/boom_snap
  grub_users $grub_users
  grub_arg kernel
  grub_class --unrestricted
boom delete 507b4ff

--rootdevice: boom create --title 3 --rootdevice /dev/snapper/boom_snap
boom list ce879b5
BootID  Version                  Name                            RootDevice            
ce879b5 3.10.0-1141.el7.x86_64   Red Hat Enterprise Linux Server /dev/snapper/boom_snap
boom entry show ce879b5
Boot Entry (boot_id=ce879b5)
  title 3
  machine-id a41ec8c891a84179a7716bce7b6e893e
  version 3.10.0-1141.el7.x86_64
  linux /vmlinuz-3.10.0-1141.el7.x86_64
  initrd /initramfs-3.10.0-1141.el7.x86_64.img
  options root=/dev/snapper/boom_snap ro rd.lvm.lv=snapper/boom_snap
  grub_users $grub_users
  grub_arg kernel
  grub_class --unrestricted
boom delete ce879b5

--root-device: boom create --title 4 --root-device /dev/snapper/boom_snap
boom list 43463f2
BootID  Version                  Name                            RootDevice            
43463f2 3.10.0-1141.el7.x86_64   Red Hat Enterprise Linux Server /dev/snapper/boom_snap
boom entry show 43463f2
Boot Entry (boot_id=43463f2)
  title 4
  machine-id a41ec8c891a84179a7716bce7b6e893e
  version 3.10.0-1141.el7.x86_64
  linux /vmlinuz-3.10.0-1141.el7.x86_64
  initrd /initramfs-3.10.0-1141.el7.x86_64.img
  options root=/dev/snapper/boom_snap ro rd.lvm.lv=snapper/boom_snap
  grub_users $grub_users
  grub_arg kernel
  grub_class --unrestricted
boom delete 43463f2

Check 4. Verify --root-lv and --root-device verify the lv/device exists (Bug 1543186)
boom create --title no-lv --root-lv snapper/nonlv
boom create --title no-dev --root-device /dev/sdxyz
boom create --title 5 --no-dev --root-lv snapper/nonlv
boom delete abaab37

Check 5. Verify proper use of user providing --rootlv /dev/* format (Bug 1544914)
boom create --title 6 --rootlv /dev/snapper/boom_snap
Created entry with boot_id def37dc:
  title 6
  machine-id a41ec8c891a84179a7716bce7b6e893e
  version 3.10.0-1141.el7.x86_64
  linux /vmlinuz-3.10.0-1141.el7.x86_64
  initrd /initramfs-3.10.0-1141.el7.x86_64.img
  options root=/dev/snapper/boom_snap ro rd.lvm.lv=snapper/boom_snap
  grub_users $grub_users
  grub_arg kernel
  grub_class --unrestricted
boom delete def37dc

Check 6. Verify proper rd.lvm.lv entry w/ --root-device option (Also listed in Bug 1544914)
boom create --title 7 --root-device /dev/snapper/boom_snap
Created entry with boot_id bd3dfd8:
  title 7
  machine-id a41ec8c891a84179a7716bce7b6e893e
  version 3.10.0-1141.el7.x86_64
  linux /vmlinuz-3.10.0-1141.el7.x86_64
  initrd /initramfs-3.10.0-1141.el7.x86_64.img
  options root=/dev/snapper/boom_snap ro rd.lvm.lv=snapper/boom_snap
  grub_users $grub_users
  grub_arg kernel
  grub_class --unrestricted
boom delete bd3dfd8

Removing volume snapper/boom_snap
Removing origin snapper/origin
Comment 22 Corey Marthaler 2020-06-09 22:09:44 UTC 
Marking verified now that the bugs listed above are fixed in the latest 7.9 lvm2 build.

3.10.0-1149.el7.x86_64

lvm2-2.02.187-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
lvm2-libs-2.02.187-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
lvm2-cluster-2.02.187-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
lvm2-lockd-2.02.187-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
lvm2-python-boom-1.2-1.el7    BUILT: Sun Jun  7 08:15:48 CDT 2020
cmirror-2.02.187-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
device-mapper-1.02.170-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
device-mapper-libs-1.02.170-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
device-mapper-event-1.02.170-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020
device-mapper-event-libs-1.02.170-5.el7    BUILT: Sun Jun  7 08:13:11 CDT 2020


Also ran a quick boot regression check of an actual root volume snapshot:

[root@host-094 ~]# lvcreate -k n -a y --yes -s /dev/rhel_host-094/root -n boom_snap 
  WARNING: Sum of all thin volume sizes (9.57 GiB) exceeds the size of thin pool rhel_host-094/pool00 and the size of whole volume group (<7.00 GiB).
  WARNING: You have not turned on protection against thin pools running out of space.
  WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
  Logical volume "boom_snap" created.
[root@host-094 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize   Pool   Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices        
  boom_snap       rhel_host-094 Vwi-a-tz--  <4.79g pool00 root   61.86                                                  
  [lvol0_pmspare] rhel_host-094 ewi-------   4.00m                                                       /dev/vda2(205) 
  pool00          rhel_host-094 twi-aotz--  <4.79g               61.86  48.05                            pool00_tdata(0)
  [pool00_tdata]  rhel_host-094 Twi-ao----  <4.79g                                                       /dev/vda2(206) 
  [pool00_tmeta]  rhel_host-094 ewi-ao----   4.00m                                                       /dev/vda2(1431)
  root            rhel_host-094 Vwi-aotz--  <4.79g pool00        61.86                                                  
  swap            rhel_host-094 -wi-ao---- 820.00m                                                       /dev/vda2(0)   

[root@host-094 ~]# df -h
Filesystem                       Size  Used Avail Use% Mounted on
devtmpfs                         7.3G     0  7.3G   0% /dev
tmpfs                            7.3G     0  7.3G   0% /dev/shm
tmpfs                            7.3G  8.7M  7.3G   1% /run
tmpfs                            7.3G     0  7.3G   0% /sys/fs/cgroup
/dev/mapper/rhel_host--094-root  4.8G  2.9G  1.9G  61% /
/dev/vda1                       1014M  156M  859M  16% /boot
tmpfs                            1.5G     0  1.5G   0% /run/user/0

[root@host-094 ~]# boom create --title BOOM --rootlv /dev/rhel_host-094/boom_snap
Created entry with boot_id c9d13bf:
  title BOOM
  machine-id 6aa4719f9f24482b902819e419ce14a6
  version 3.10.0-1149.el7.x86_64
  linux /vmlinuz-3.10.0-1149.el7.x86_64
  initrd /initramfs-3.10.0-1149.el7.x86_64.img
  options root=/dev/rhel_host-094/boom_snap ro rd.lvm.lv=rhel_host-094/boom_snap
  grub_users $grub_users
  grub_arg --unrestricted
  grub_class kernel

[root@host-094 ~]# ls /dev/rhel_host-094/boom_snap
/dev/rhel_host-094/boom_snap
[root@host-094 ~]# boom list
BootID  Version                  Name                            RootDevice                  
c9d13bf 3.10.0-1149.el7.x86_64   Red Hat Enterprise Linux Server /dev/rhel_host-094/boom_snap
[root@host-094 ~]# grep boom /boot/grub2/grub.cfg
### BEGIN /etc/grub.d/42_boom ###
### END /etc/grub.d/42_boom ###
[root@host-094 ~]# grub2-mkconfig > /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-1149.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-1149.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-6aa4719f9f24482b902819e419ce14a6
Found initrd image: /boot/initramfs-0-rescue-6aa4719f9f24482b902819e419ce14a6.img
done
[root@host-094 ~]# sync
[root@host-094 ~]# reboot 


# Toggle down to BOOM
  Red Hat Enterprise Linux Server (3.10.0-1149.el7.x86_64) 7.9 (Maipo)      
  Red Hat Enterprise Linux Server (0-rescue-6aa4719f9f24482b902819e419ce14>
  Snapshots                                                                
      BOOM                                                                      

# Actually ran into bug 1749871 here before the eventual snap booting.

# Quick verification that the snap is the running root vol
[root@host-094 ~]# df -h
Filesystem                            Size  Used Avail Use% Mounted on
devtmpfs                              7.3G     0  7.3G   0% /dev
tmpfs                                 7.3G     0  7.3G   0% /dev/shm
tmpfs                                 7.3G  8.6M  7.3G   1% /run
tmpfs                                 7.3G     0  7.3G   0% /sys/fs/cgroup
/dev/mapper/rhel_host--094-boom_snap  4.8G  2.9G  2.0G  60% /
/dev/vda1                            1014M  156M  859M  16% /boot
tmpfs                                 1.5G     0  1.5G   0% /run/user/0
Comment 24 errata-xmlrpc 2020-09-29 19:55:48 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3927