Description of problem: Currently it is not possible to add and delete kernel args simultaneously with MachineConfigs [0]. This is already fixed in rpm-ostree [1], but the fix is not available in RHCOS for OCP 4.4 yet. [0] https://github.com/openshift/machine-config-operator/issues/1265 [1] https://github.com/coreos/rpm-ostree/pull/1940 Version-Release number of selected component (if applicable): OCP 4.4 nightly How reproducible: always Steps to Reproduce: 1. Create a MachineConfig with kernel args, wait for MachineConfigPool getting updated. 2. Update the MachineConfig with added and deleted kernel args Actual results: MachineConfigPool is stuck in degraded state Expected results: MachineConfigPool updated again, nodes rebooted with updated kernel args. Additional info:
Related - https://bugzilla.redhat.com/show_bug.cgi?id=1776823
The KNI folks would like this fix as part of 4.4, ahead of the RHEL 8.2 release. Colin, can we get a fixed version of `rpm-ostree` built and included in the RHAOS 4.4 puddle?
So that's only in 2019.7, but the latest errata https://errata.devel.redhat.com/advisory/47351 is for 2019.6. The tracker bug for that is: https://bugzilla.redhat.com/show_bug.cgi?id=1756192 I think at this point we need an exception or something to do another build?
OK I filed a new bug https://bugzilla.redhat.com/show_bug.cgi?id=1807487
The fixed package landed in RHCOS 44.81.202003062006-0; all subsequent builds should have the fix.
This can be moved to VERIFIED once it hits ON_QA ``` Red Hat Enterprise Linux CoreOS 44.81.202003110830-0 Part of OpenShift 4.4, RHCOS is a Kubernetes native operating system managed by the Machine Config Operator (`clusteroperator/machine-config`). WARNING: Direct SSH access to machines is not recommended; instead, make configuration changes via `machineconfig` objects: https://docs.openshift.com/container-platform/4.4/architecture/architecture-rhcos.html --- Last login: Thu Mar 12 14:15:36 2020 from 192.168.124.1 [core@localhost ~]$ rpm-ostree kargs rhcos.root=crypt_rootfs console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu rd.luks.options=discard $ignition_firstboot ostree=/ostree/boot.0/rhcos/8e33da004cf9c41250a7fb2f8f125e9767bfe7343252ae5d2488b08a858f8d8a/0 foo=bar bar=foo pizza=life [core@localhost ~]$ sudo rpm-ostree kargs --delete pizza=life --append beer=life Staging deployment... done Kernel arguments updated. Run "systemctl reboot" to start a reboot [core@localhost ~]$ rpm-ostree kargs rhcos.root=crypt_rootfs console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu rd.luks.options=discard $ignition_firstboot ostree=/ostree/boot.0/rhcos/8e33da004cf9c41250a7fb2f8f125e9767bfe7343252ae5d2488b08a858f8d8a/0 foo=bar bar=foo beer=life [core@localhost ~]$ sudo systemctl reboot ... Warning: Permanently added '192.168.124.105' (ECDSA) to the list of known hosts. Red Hat Enterprise Linux CoreOS 44.81.202003110830-0 Part of OpenShift 4.4, RHCOS is a Kubernetes native operating system managed by the Machine Config Operator (`clusteroperator/machine-config`). WARNING: Direct SSH access to machines is not recommended; instead, make configuration changes via `machineconfig` objects: https://docs.openshift.com/container-platform/4.4/architecture/architecture-rhcos.html --- Last login: Thu Mar 12 14:21:22 2020 from 192.168.124.1 [core@localhost ~]$ rpm-ostree kargs rhcos.root=crypt_rootfs console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu rd.luks.options=discard $ignition_firstboot ostree=/ostree/boot.1/rhcos/8e33da004cf9c41250a7fb2f8f125e9767bfe7343252ae5d2488b08a858f8d8a/0 foo=bar bar=foo beer=life [core@localhost ~]$ cat /proc/cmdline BOOT_IMAGE=(hd0,gpt1)/ostree/rhcos-8e33da004cf9c41250a7fb2f8f125e9767bfe7343252ae5d2488b08a858f8d8a/vmlinuz-4.18.0-147.5.1.el8_1.x86_64 rhcos.root=crypt_rootfs console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu rd.luks.options=discard ostree=/ostree/boot.1/rhcos/8e33da004cf9c41250a7fb2f8f125e9767bfe7343252ae5d2488b08a858f8d8a/0 foo=bar bar=foo beer=life ```
Marking VERIFIED per comment #13
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581