Bug 2224108 - backport https://github.com/ostreedev/ostree/pull/2874/commits/de6fddc6adee09a93901243dc7074090828a1912 [rhel-8.6.0.z]
Summary: backport https://github.com/ostreedev/ostree/pull/2874/commits/de6fddc6adee09...
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: ostree
Version: 8.6
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Joseph Marrero
QA Contact: Aashish Radhakrishnan
URL:
Whiteboard:
Depends On: 2224102
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-07-19 19:27 UTC by RHEL Program Management Team
Modified: 2023-07-28 16:40 UTC (History)
6 users (show)

Fixed In Version: ostree-2022.1-4.el8_6
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 2224102
Environment:
Last Closed:
Type: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-162796 0 None None None 2023-07-19 19:29:47 UTC

Comment 1 Colin Walters 2023-07-24 21:44:34 UTC
For verification, I think it's fine to test the 8.6 build on rhcos-4.12-8.6 in the same way we're testing the 8.8 one.  But that said, it would *also* make sense to verify this with Edge.

Comment 4 Aashish Radhakrishnan 2023-07-25 16:52:36 UTC
Verification process:

aaradhak@fedora ~/rhcos/rhcos-4.12 $ cosa run --qemu-image rhcos-412.86.202305030814-0-qemu.x86_64.qcow2 --qemu-size '+3T' --qemu-memory 8192 -B reprovision-xfs.bu
COREOS_ASSEMBLER_CONTAINER=quay.io/coreos-assembler/coreos-assembler:rhcos-4.12
COREOS_ASSEMBLER_ADD_CERTS=y
+ podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v /var/home/aaradhak/rhcos/rhcos-4.12:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name cosa -v=/etc/pki/ca-trust:/etc/pki/ca-trust:ro quay.io/coreos-assembler/coreos-assembler:rhcos-4.12 run --qemu-image rhcos-412.86.202305030814-0-qemu.x86_64.qcow2 --qemu-size +3T --qemu-memory 8192 -B reprovision-xfs.bu
Red Hat Enterprise Linux CoreOS 412.86.202305030814-0
  Part of OpenShift 4.12, RHCOS is a Kubernetes native operating system
  managed by the Machine Config Operator (`clusteroperator/machine-config`).

WARNING: Direct SSH access to machines is not recommended; instead,
make configuration changes via `machineconfig` objects:
  https://docs.openshift.com/container-platform/4.12/architecture/architecture-rhcos.html

---
Last login: Tue Jul 25 16:18:44 2023
[core@cosa-devsh ~]$ sudo -i
[root@cosa-devsh ~]# xfs_info / |grep agcount=
meta-data=/dev/vda4              isize=512    agcount=4, agsize=187904768 blks
[root@cosa-devsh ~]# alias verify-ostree="podman run --rm -ti --pull=newer --privileged -v /:/rootfs --net=none quay.io/cgwalters/ostree-ext-dev provisional-repair repair --sysroot /rootfs/sysroot --dry-run"
[root@cosa-devsh ~]# verify-ostree 
Trying to pull quay.io/cgwalters/ostree-ext-dev:latest...
Getting image source signatures
Copying blob dbc9fb14b970 done  
Copying blob ad5077952f52 done  
Copying blob 19d2bba2169e done  
Copying config 991ff7104a done  
Writing manifest to image destination
Storing signatures
Running in container, assuming we can remount /rootfs/sysroot writable
OK no derived commits found.
[root@cosa-devsh ~]# rpm-ostree rebase --experimental ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable
'''
'''
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# verify-ostree 
Running in container, assuming we can remount /rootfs/sysroot writable
Found 1 derived commits
Backing filesystem information:
  File: "."
    ID: fc0400000000 Namelen: 255     Type: xfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 751252071  Free: 744866969  Available: 744866969
Inodes: Total: 300647616  Free: 300569834
Attempting analysis of ostree state for files that may be incorrectly linked.
For more information, see https://github.com/ostreedev/ostree/pull/2874/commits/de6fddc6adee09a93901243dc7074090828a1912

Gathering inodes for ostree objects...
Analyzed 25502 objects with > 32 bit inode numbers and 23380 objects with <= 32 bit inode numbers
warning: 1306 potentially colliding inodes found
Verifying 1 ostree-container images
Verifying with base ostree layer ostree/container/blob/sha256_3A_074b4ac01d1e6fd65aed6f9c9d81fd0bf0158a30b02a91c5cf674f7f013bc74f
Image has 0 derived layers
warning: Found corrupted merge commit
  inode clashes: 26
  unknown:       0
  ok:            2028

warning: Corrupted image docker://quay.io/fedora/fedora-coreos:stable
error: Found potential corruption, dry-run mode enabled
[root@cosa-devsh ~]# rpm -qa ostree
ostree-2022.2-5.el8.x86_64
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm-ostree usroverlay
Development mode enabled.  A writable overlayfs is now mounted on /usr.
All changes there will be discarded on reboot.
[root@cosa-devsh ~]# curl -L -k --fail --remote-name-all https://download.eng.bos.redhat.com/brewroot/work/tasks/4956/54144956/ostree-{,libs-,grub2-}2022.1-4.el8_6.x86_64.rpm
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  250k  100  250k    0     0   380k      0 --:--:-- --:--:-- --:--:--  381k
100  437k  100  437k    0     0   964k      0 --:--:-- --:--:-- --:--:-- 1700k
100 25188  100 25188    0     0   424k      0 --:--:-- --:--:-- --:--:--  424k
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm -Uvh ostree-*
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
	package ostree-libs-2022.2-5.el8.x86_64 (which is newer than ostree-libs-2022.1-4.el8_6.x86_64) is already installed
	package ostree-2022.2-5.el8.x86_64 (which is newer than ostree-2022.1-4.el8_6.x86_64) is already installed
	package ostree-grub2-2022.2-5.el8.x86_64 (which is newer than ostree-grub2-2022.1-4.el8_6.x86_64) is already installed
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm -Uvh ostree-* --oldpackage
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:ostree-libs-2022.1-4.el8_6       ################################# [ 17%]
   2:ostree-2022.1-4.el8_6            ################################# [ 33%]
   3:ostree-grub2-2022.1-4.el8_6      ################################# [ 50%]
Cleaning up / removing...
   4:ostree-grub2-2022.2-5.el8        ################################# [ 67%]
   5:ostree-2022.2-5.el8              ################################# [ 83%]
   6:ostree-libs-2022.2-5.el8         ################################# [100%]
[/usr/lib/tmpfiles.d/pkg-dbus-daemon.conf:1] Duplicate line for path "/var/lib/dbus", ignoring.
[/usr/lib/tmpfiles.d/tmp.conf:12] Duplicate line for path "/var/tmp", ignoring.
[/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring.
[/usr/lib/tmpfiles.d/var.conf:19] Duplicate line for path "/var/cache", ignoring.
[/usr/lib/tmpfiles.d/var.conf:21] Duplicate line for path "/var/lib", ignoring.
[/usr/lib/tmpfiles.d/var.conf:23] Duplicate line for path "/var/spool", ignoring.
"/home" already exists and is not a directory.
"/srv" already exists and is not a directory.
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# systemctl restart rpm-ostreed
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm -qa ostree
ostree-2022.1-4.el8_6.x86_64
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# verify-ostree 
Running in container, assuming we can remount /rootfs/sysroot writable
Found 1 derived commits
Backing filesystem information:
  File: "."
    ID: fc0400000000 Namelen: 255     Type: xfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 751252071  Free: 744864992  Available: 744864992
Inodes: Total: 300647616  Free: 300569689
Attempting analysis of ostree state for files that may be incorrectly linked.
For more information, see https://github.com/ostreedev/ostree/pull/2874/commits/de6fddc6adee09a93901243dc7074090828a1912

Gathering inodes for ostree objects...
Analyzed 25502 objects with > 32 bit inode numbers and 23380 objects with <= 32 bit inode numbers
warning: 1306 potentially colliding inodes found
Verifying 1 ostree-container images
Verifying with base ostree layer ostree/container/blob/sha256_3A_074b4ac01d1e6fd65aed6f9c9d81fd0bf0158a30b02a91c5cf674f7f013bc74f
Image has 0 derived layers
warning: Found corrupted merge commit
  inode clashes: 26
  unknown:       0
  ok:            2028

warning: Corrupted image docker://quay.io/fedora/fedora-coreos:stable
error: Found potential corruption, dry-run mode enabled
[root@cosa-devsh ~]# 

[root@cosa-devsh ~]# mount -o remount,rw /sysroot
[root@cosa-devsh ~]# 

[root@cosa-devsh ~]# rpm-ostree status
State: idle
Deployments:
  ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable
                   Digest: sha256:f01cbb6cb39878550a699468463809b7475631907af9dbae7e24c9a439dae132
                  Version: 38.20230625.3.0 (2023-07-25T16:25:01Z)
                     Diff: 337 upgraded, 13 downgraded, 153 removed, 76 added

* 2aa485c7967e1b147f05d41a773d725878989c826b3b3c8e927585d5b388d9ca
                  Version: 412.86.202305030814-0 (2023-05-03T08:17:34Z)
                 Unlocked: development


[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]#                 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# ostree refs --delete ostree/container
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm-ostree status
State: idle
Deployments:
  (error fetching image metadata)
                Timestamp: 2023-07-25T16:25:01Z
                     Diff: 337 upgraded, 13 downgraded, 153 removed, 76 added

* 2aa485c7967e1b147f05d41a773d725878989c826b3b3c8e927585d5b388d9ca
                  Version: 412.86.202305030814-0 (2023-05-03T08:17:34Z)
                 Unlocked: development
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm-ostree cleanup -p
Freed: 1.4 GB (pkgcache branches: 0)
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm-ostree status
State: idle
Deployments:
* 2aa485c7967e1b147f05d41a773d725878989c826b3b3c8e927585d5b388d9ca
                  Version: 412.86.202305030814-0 (2023-05-03T08:17:34Z)
                 Unlocked: development
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm-ostree rebase --experimental ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable       
'''
'''
[root@cosa-devsh ~]# verify-ostree 
Running in container, assuming we can remount /rootfs/sysroot writable
Found 1 derived commits
Backing filesystem information:
  File: "."
    ID: fc0400000000 Namelen: 255     Type: xfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 751252071  Free: 744865804  Available: 744865804
Inodes: Total: 300647616  Free: 300569726
Attempting analysis of ostree state for files that may be incorrectly linked.
For more information, see https://github.com/ostreedev/ostree/pull/2874/commits/de6fddc6adee09a93901243dc7074090828a1912

Gathering inodes for ostree objects...
Analyzed 25198 objects with > 32 bit inode numbers and 23646 objects with <= 32 bit inode numbers
warning: 1124 potentially colliding inodes found
Verifying 1 ostree-container images
Verifying with base ostree layer ostree/container/blob/sha256_3A_074b4ac01d1e6fd65aed6f9c9d81fd0bf0158a30b02a91c5cf674f7f013bc74f
Image has 0 derived layers
OK image docker://quay.io/fedora/fedora-coreos:stable (verified=11

OK no corrupted images found
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# 
[root@cosa-devsh ~]# rpm -qa ostree
ostree-2022.1-4.el8_6.x86_64

Comment 6 Aashish Radhakrishnan 2023-07-28 16:36:54 UTC
Verification done based on comment 4


Note You need to log in before you can comment on or make changes to this bug.