Description of the problem: Multicluster_operators_hub_subscription pod having various issues due to the constantly increasing /tmp storage usage. This includes: - Error: Kubelet may be retrying requests that are timing out in CRI-O due to system load: context deadline exceeded: error reserving ctr name k8s_multicluster-operators-hub-subscription_multicluster-operators-hub-subscription-..._open-cluster-management_... for id 3844...c6b9: name is reserved - Pod constantly being evicted with error: The node was low on resource: ephemeral-storage. Container multicluster-operators-hub-subscription was using 16392Ki, which exceeds its request of 0. - Pod logging errors similar to: E0209 13:08:19.899823 1 gitrepo_sync.go:388] lstat /tmp/data-services/release/internal: no such file or directoryFailed to sort kubernetes resources and helm charts. E0209 13:08:19.899861 1 gitrepo_sync.go:112] lstat /tmp/data-services/release/internal: no such file or directory E0209 13:08:19.899888 1 mcmhub_controller.go:626] subscription-hub-reconciler "msg"="failed to process on doMCMHubReconcile" "error"="lstat /tmp/data-services/release/internal: no such file or directory" Release version: ACM v2.4.1 (OCP 4.8 (ARO)) Operator snapshot version: OCP version: 4.8.29 (ARO) Browser Info: Steps to reproduce: 1. Configure applications to be deployed via HelmRepo channel 2. Access multicluster_operators_hub_subscription via rsh 3. Check the /tmp folder - observe that subfolders named charts* will be constantly generated Actual results: New subfolders charts* will be constantly created in /tmp folder. Even in the moderate environments (6 managed clusters, ~30 apps) /tmp will run out of free space in just few days (~70G, 250k charts* folders in about 4 days) Expected results: /tmp usage does not grow Additional info: @Roke Jung proposed solution to include CHARTS_DIR environment variable ("/tmp") value which seems to help with the problem. I think this should be included OOB.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat Advanced Cluster Management 2.4.5 security updates and bug fixes), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5201