Description of problem: rootfs is created too small ``` # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 30G 0 disk ├─sda1 8:1 0 1M 0 part ├─sda2 8:2 0 127M 0 part ├─sda3 8:3 0 384M 0 part /boot ├─sda4 8:4 0 3G 0 part │ └─root 253:0 0 3G 0 crypt /sysroot └─sda5 8:5 0 65M 0 part ``` when using the documented procedure for NBDE https://github.com/openshift/openshift-docs/blob/enterprise-4.7/modules/installation-special-config-encrypt-disk-tang.adoc without NBDE, rootfs is spaning the full disk size: ``` $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 30G 0 disk ├─sda1 8:1 0 1M 0 part ├─sda2 8:2 0 127M 0 part ├─sda3 8:3 0 384M 0 part /boot ├─sda4 8:4 0 29.4G 0 part /sysroot └─sda5 8:5 0 65M 0 part ``` Version-Release number of selected component (if applicable): 4.8-nightly How reproducible: everytime Steps to Reproduce: 1. follow official steps to enable nbde Actual results: mcd crash because not enough disk space Expected results: installation to succeed Additional info:
See also: https://github.com/openshift/os/pull/514 and https://gitlab.cee.redhat.com/coreos/redhat-coreos/-/merge_requests/1234 The fix for this landed in RHCOS 48.83.202103111918-0
Verified on RHCOS 48.83.202103122318-0 [core@localhost ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 252:0 0 20G 0 disk |-vda1 252:1 0 1M 0 part |-vda2 252:2 0 127M 0 part |-vda3 252:3 0 384M 0 part /boot `-vda4 252:4 0 19.5G 0 part `-root 253:0 0 19.5G 0 crypt /sysroot Also need to verify this after the boot image bump.
There's a bug in RHEL's lsblk (which we've hit before; see https://github.com/coreos/coreos-installer/pull/453) breaking the new code in ignition-ostree-growfs.sh. Working on a patch to work around it. But sadly, we'll need a new bootimage bump for this. Re-moving to ASSIGNED.
The fix for this landed in RHCOS 48.83.202103221318-0
nvme disk ---------------- [core@cosa-devsh ~]$ rpm-ostree status State: idle Deployments: ● ostree://328a44d7c259ca1e3ed31ae020f09d922f460be998657a92f684f6760443077b Version: 48.83.202103221318-0 (2021-03-22T13:22:02Z) [core@cosa-devsh ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom nvme0n1 259:0 0 16G 0 disk |-nvme0n1p1 259:1 0 1M 0 part |-nvme0n1p2 259:2 0 127M 0 part |-nvme0n1p3 259:3 0 384M 0 part /boot `-nvme0n1p4 259:4 0 15.5G 0 part `-root 253:0 0 15.5G 0 crypt /sysroot [core@cosa-devsh ~]$ sudo cryptsetup luksDump /dev/disk/by-partlabel/root LUKS header information Version: 2 Epoch: 6 Metadata area: 16384 [bytes] Keyslots area: 16744448 [bytes] UUID: 68c753b1-8f2f-427d-afc4-f3b9d0db550b Label: (no label) Subsystem: (no subsystem) Flags: (no flags) Data segments: 0: crypt offset: 16777216 [bytes] length: (whole device) cipher: aes-cbc-essiv:sha256 sector: 512 [bytes] Keyslots: 1: luks2 Key: 256 bits Priority: normal Cipher: aes-cbc-essiv:sha256 Cipher key: 256 bits PBKDF: argon2i Time cost: 4 Memory: 1048576 Threads: 4 Salt: d8 78 43 e1 1a 40 b1 40 a3 91 b1 fb 25 66 2a 05 b1 dc 4f f8 20 5a 7b 83 bc 99 5c 10 e5 d5 92 1d AF stripes: 4000 AF hash: sha256 Area offset:163840 [bytes] Area length:131072 [bytes] Digest ID: 0 Tokens: 0: clevis Keyslot: 1 Digests: 0: pbkdf2 Hash: sha256 Iterations: 203212 Salt: 8c b8 d3 99 97 1f f5 8b 6b b8 a7 d8 ba b4 57 5e 57 59 3c 6e 30 87 bc 6e 30 62 ba 44 90 95 d3 83 Digest: 6e 07 4f bb cb 1f 8f d0 d6 46 6b 35 d3 d6 ef 08 18 70 29 68 2b 88 3d 9e f8 4e 47 c0 18 fb b2 22 [core@cosa-devsh ~]$ clevis luks list -d /dev/disk/by-partlabel/root Device /dev/disk/by-partlabel/root does not exist or access denied. Device /dev/disk/by-partlabel/root does not exist or access denied. Device /dev/disk/by-partlabel/root does not exist or access denied. /dev/disk/by-partlabel/root is not a supported LUKS device! No used slots detected for device /dev/disk/by-partlabel/root! [core@cosa-devsh ~]$ sudo clevis luks list -d /dev/disk/by-partlabel/root 1: sss '{"t":1,"pins":{"tang":[{"url":"http://192.168.1.176"}]}}' [core@cosa-devsh ~]$ findmnt /var TARGET SOURCE FSTYPE OPTIONS /var /dev/mapper/root[/ostree/deploy/rhcos/var] xfs rw,relatime,seclabel,at [core@cosa-devsh ~]$ findmnt /var | less TARGET SOURCE FSTYPE OPTIONS /var /dev/mapper/root[/ostree/deploy/rhcos/var] xfs rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,prjquota [core@cosa-devsh ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom nvme0n1 259:0 0 16G 0 disk |-nvme0n1p1 259:1 0 1M 0 part |-nvme0n1p2 259:2 0 127M 0 part |-nvme0n1p3 259:3 0 384M 0 part /boot `-nvme0n1p4 259:4 0 15.5G 0 part `-root 253:0 0 15.5G 0 crypt /sysroot regular disk ------------------- [core@localhost ~]$ rpm-ostree status State: idle Deployments: ● ostree://328a44d7c259ca1e3ed31ae020f09d922f460be998657a92f684f6760443077b Version: 48.83.202103221318-0 (2021-03-22T13:22:02Z) [core@localhost ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 252:0 0 20G 0 disk ├─vda1 252:1 0 1M 0 part ├─vda2 252:2 0 127M 0 part ├─vda3 252:3 0 384M 0 part /boot └─vda4 252:4 0 19.5G 0 part └─root 253:0 0 19.5G 0 crypt /sysroot [core@localhost ~]$ sudo cryptsetup luksDump /dev/disk/by-partlabel/root LUKS header information Version: 2 Epoch: 6 Metadata area: 16384 [bytes] Keyslots area: 16744448 [bytes] UUID: d8404c5d-8db6-418e-bf57-5f9b3e43534d Label: (no label) Subsystem: (no subsystem) Flags: (no flags) Data segments: 0: crypt offset: 16777216 [bytes] length: (whole device) cipher: aes-cbc-essiv:sha256 sector: 512 [bytes] Keyslots: 1: luks2 Key: 256 bits Priority: normal Cipher: aes-cbc-essiv:sha256 Cipher key: 256 bits PBKDF: argon2i Time cost: 4 Memory: 629526 Threads: 2 Salt: e0 8d 9a f2 99 f0 43 d2 46 95 37 a4 2e fa e6 9f 16 b4 33 83 05 f7 3c 29 42 d2 d2 b1 89 d3 9e dd AF stripes: 4000 AF hash: sha256 Area offset:163840 [bytes] Area length:131072 [bytes] Digest ID: 0 Tokens: 0: clevis Keyslot: 1 Digests: 0: pbkdf2 Hash: sha256 Iterations: 191625 Salt: 76 6a de 2e 82 74 12 3c f9 95 a2 cd 1a bf 4b 4d 65 63 77 8a 4f 88 b0 27 26 53 3e 21 92 bb 93 d9 Digest: ae 5e 0b 24 88 06 be ee 6c 2e 84 0d ba e9 08 e2 83 ac d3 01 92 4e c1 06 47 00 a1 ad dd 8d be 52 [core@localhost ~]$ sudo clevis luks list -d /dev/disk/by-partlabel Device /dev/disk/by-partlabel is not compatible. Device /dev/disk/by-partlabel is not compatible. Device /dev/disk/by-partlabel is not compatible. /dev/disk/by-partlabel is not a supported LUKS device! No used slots detected for device /dev/disk/by-partlabel! [core@localhost ~]$ sudo clevis luks list -d /dev/disk/by-partlabel/root 1: sss '{"t":1,"pins":{"tang":[{"url":"http://192.168.1.176"}]}}' [core@localhost ~]$ findmnt /var | less [core@localhost ~]$ findmnt /var TARGET SOURCE FSTYPE OPTIONS /var /dev/mapper/root[/ostree/deploy/rhcos/var] xfs rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,prjquota
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438