Bug 1940704

Assigning to jlebon for initial evaluation.

At the "Ignition distro integration" level, the assumption is that as soon as you reprovision the root filesystem, you're choosing your own adventure. So e.g. you should be able to add whatever mount options you'd like.

There's at least two problems here though:
1. So far `prjquota` has mostly been an implementation detail we haven't exposed to users. This would require documenting it everywhere root reprovisioning is documented.
2. Changing mount options right now for the rootfs is painful. We don't support the `mountOptions` flag (https://github.com/coreos/fedora-coreos-config/issues/805) nor Ignition kargs (but soon: https://github.com/coreos/ignition/issues/1168). So right now, this would require using `rpm-ostree kargs` in a systemd service as documented in https://docs.fedoraproject.org/en-US/fedora-coreos/kernel-args/.

It's tempting to try to get https://github.com/coreos/fedora-coreos-config/issues/805 into 4.7, because at a technical level it's pretty trivial to do. That would leave (1), i.e. updating all the documented MachineConfigs and RCC snippets to include a mount_options/mountOptions. Though specifically for prjquota, it doesn't seem like there's any harm in just *always* turning it on at the OS level if the rootfs is XFS, regardless of whether it was reprovisioned or not. AFAICT there isn't really any overhead or performance issues associated with this (I mean... this *has* been the default for a long time). So then it remains an implementation detail and avoids documentation churn. Or another way to frame this is that it simplifies the rootflags messaging to just: "by default, we use prjquota if the rootfs is XFS".

And when we implement https://github.com/coreos/fedora-coreos-config/issues/805 (which is generally useful), anyone who for whatever reason *doesn't* want prjquota can just do `mountOptions: []` (short-term, they can fallback to using `rpm-ostree kargs` to modify the `rootflags` karg).

I'll take a look at this.

PR to retain prjquota on XFS: https://github.com/coreos/fedora-coreos-config/pull/903
PR to support mountOptions: https://github.com/coreos/fedora-coreos-config/pull/904 (though not planning to backport that one since it's not strictly needed).

This fix landed in RHCOS 48.83.202103221318-0

Booted RHCOS 48.83.202103221318-0 with luks enabled and prjquota option is set


[core@localhost ~]$ rpm-ostree status
State: idle
Deployments:
* ostree://328a44d7c259ca1e3ed31ae020f09d922f460be998657a92f684f6760443077b
                   Version: 48.83.202103221318-0 (2021-03-22T13:22:02Z)

[core@localhost ~]$ findmnt /var | less
TARGET SOURCE                                     FSTYPE OPTIONS
/var   /dev/mapper/root[/ostree/deploy/rhcos/var] xfs    rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,prjquota

[core@localhost ~]$ sudo cryptsetup luksDump /dev/disk/by-partlabel/root
LUKS header information
Version:       	2
Epoch:         	6
Metadata area: 	16384 [bytes]
Keyslots area: 	16744448 [bytes]
UUID:          	d8404c5d-8db6-418e-bf57-5f9b3e43534d
Label:         	(no label)
Subsystem:     	(no subsystem)
Flags:       	(no flags)

Data segments:
  0: crypt
	offset: 16777216 [bytes]
	length: (whole device)
	cipher: aes-cbc-essiv:sha256
	sector: 512 [bytes]

Keyslots:
  1: luks2
	Key:        256 bits
	Priority:   normal
	Cipher:     aes-cbc-essiv:sha256
	Cipher key: 256 bits
	PBKDF:      argon2i
	Time cost:  4
	Memory:     629526
	Threads:    2
	Salt:       e0 8d 9a f2 99 f0 43 d2 46 95 37 a4 2e fa e6 9f 
	            16 b4 33 83 05 f7 3c 29 42 d2 d2 b1 89 d3 9e dd 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:163840 [bytes]
	Area length:131072 [bytes]
	Digest ID:  0
Tokens:
  0: clevis
	Keyslot:  1
Digests:
  0: pbkdf2
	Hash:       sha256
	Iterations: 191625
	Salt:       76 6a de 2e 82 74 12 3c f9 95 a2 cd 1a bf 4b 4d 
	            65 63 77 8a 4f 88 b0 27 26 53 3e 21 92 bb 93 d9 
	Digest:     ae 5e 0b 24 88 06 be ee 6c 2e 84 0d ba e9 08 e2 
	            83 ac d3 01 92 4e c1 06 47 00 a1 ad dd 8d be 52 
[core@localhost ~]$ sudo lsblk
NAME     MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
vda      252:0    0   20G  0 disk  
|-vda1   252:1    0    1M  0 part  
|-vda2   252:2    0  127M  0 part  
|-vda3   252:3    0  384M  0 part  /boot
`-vda4   252:4    0 19.5G  0 part  
  `-root 253:0    0 19.5G  0 crypt /sysroot


[core@localhost ~]$ sudo clevis luks list -d /dev/disk/by-partlabel/root
1: sss '{"t":1,"pins":{"tang":[{"url":"http://192.168.1.176"}]}}'

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438