Description of problem: If I follow [1] to create a ContainerRuntimeConfig that sets an overlaySize, that setting is ignored and container root filesystems can grow higher than overlaySize. Version-Release number of selected component (if applicable): Confirmed on 4.7.31 and latest nightly, likely to occur in any version after the commit I'll post in the additional notes. How reproducible: Always in the right versions Steps to Reproduce: 1. Follow instructions on [1] 2. 3. Actual results: Container root filesystem sizes not limited to overlaySize Expected results: Container root filesystem sizes limited to overlaySize Additional info: It seems that this commit[2] introduced `storage_driver` and `storage_options` settings on crio config, in order to copy them from defaults. The problem is that, by doing so, all the storage options from /etc/containers/storage.conf are overridden. This includes the "size" setting under "storage.options" section of "/etc/containers/storage.conf" that is configured whenever an overlaySize is specified[3]. Versions prior to that commit (e.g. 4.7.19) don't reproduce the bug. If I also tune cri-o config to remove those options, the issue also doesn't happen. As a last note, the wrong commit was introduced to have cri-o defaults managed by MCO in a safe way. Some recent versions also ship an /etc/crio/crio.conf that includes the same options, overriding /etc/containers/storage.conf in the very same way. [1] - https://docs.openshift.com/container-platform/4.9/post_installation_configuration/machine-configuration-tasks.html#set-the-default-max-container-root-partition-size-for-overlay-with-crio_post-install-machine-configuration-tasks [2] - https://github.com/openshift/machine-config-operator/commit/d6809a18ab361da4b8985a15ae752379bb70bf7e [3] - https://github.com/openshift/machine-config-operator/blob/d6809a18ab361da4b8985a15ae752379bb70bf7e/pkg/controller/container-runtime-config/helpers.go#L279
I coded PR[1] to get this fixed. It basically does 2 things: - Removes crio options from /etc/crio/crio.conf.d/00-default that collide with /etc/containers/storage.conf from both default master and worker MCP default machineconfigs - It blanks /etc/crio/crio.conf. I understand this may sound as a quite radical approach, but recent cri-o versions also include the offending settings on it. I already checked that /etc/crio/crio.conf.d/00-default contains any default value needed from /etc/crio/crio.conf , so the whole file is redundant. In fact, as per the commit that introduced the issue[2], this seems to be the original plan. [1] - https://github.com/openshift/machine-config-operator/pull/2811 [2] - https://github.com/openshift/machine-config-operator/commit/d6809a18ab361da4b8985a15ae752379bb70bf7e
I believe this is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2012838
Indeed it is. Thanks and sorry. *** This bug has been marked as a duplicate of bug 2012838 ***
Reopening bug because, although it is technically a duplicate, the suggested set of changes still makes sense alone (it is good to not have duplicated settings). Check this PR comments for more context[1][2] [1] - https://github.com/openshift/machine-config-operator/pull/2811#issuecomment-953047861 [2] - https://github.com/openshift/machine-config-operator/pull/2811#issuecomment-953857563
verified on 4.10.0-0.nightly-2021-12-01-032405 sh-4.4# chroot /host sh-4.4# cat /etc/crio/crio.conf # The CRI-O configuration file specifies all of the available configuration # options and command-line flags for the crio(8) OCI Kubernetes Container Runtime # daemon, but in a TOML format that can be more easily modified and versioned. # # Please refer to crio.conf(5) for details of all configuration options. # CRI-O supports partial configuration reload during runtime, which can be # done by sending SIGHUP to the running process. Currently supported options # are explicitly mentioned with: 'This option supports live configuration # reload'. # CRI-O reads its storage defaults from the containers-storage.conf(5) file # located at /etc/containers/storage.conf. Modify this storage configuration if # you want to change the system's defaults. If you want to modify storage just # for CRI-O, you can change the storage configuration options here. [crio] # The crio.runtime table contains settings pertaining to the OCI runtime used # and options for how to set up and manage the OCI runtime. [crio.runtime] # If true, SELinux will be used for pod separation on the host. selinux = true ... sh-4.4# cat /etc/crio/crio.conf.d/00-default [crio] internal_wipe = true version_file_persist = "/var/lib/crio/version" [crio.api] stream_address = "" stream_port = "10010"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056