+++ This bug was initially created as a clone of Bug #2093040 +++ RHCOS Version at Install Time: 4.11 dev build Platform: qemu Architecture: x86_64 What are you trying to do? What is your use case? Run `toolbox` on a single RHCOS node that has `podman` 4.0 installed. What happened? What went wrong or what did you expect? The `toolbox` command reports that the container is in an unknown state `created`: ``` [core@cosa-devsh ~]$ rpm -q podman toolbox podman-4.0.2-6.module+el8.6.0+14877+f643d2d6.x86_64 toolbox-0.0.9-1.rhaos4.11.el8.noarch [core@cosa-devsh ~]$ toolbox Trying to pull registry.redhat.io/rhel8/support-tools:latest... Getting image source signatures Checking if image destination supports signatures Copying blob cae7320018ab done Copying blob f70d60810c69 done Copying blob 545277d80005 done Copying config a8419564b7 done Writing manifest to image destination Storing signatures a8419564b7cc1417d26300d4560ce5812dccaee0c5ec88e22956ad5e67b32037 Spawning a container 'toolbox-core' with image 'registry.redhat.io/rhel8/support-tools' Detected RUN label in the container image. Using that as the default... 220b8a1616e3993fcfe479ad283fec70847ac65be533e6d99d15f0afe84bf1fd Container 'toolbox-core' in unknown state: 'created' ``` What are the steps to reproduce your issue? Please try to reduce these steps to something that can be reproduced with a single RHCOS node. 1. Build RHCOS 4.11 with RHEL 8.6 content, including `podman` 4.0 2. `cosa run` 3. `toolbox` It looks like in `podman` 4.0 when you do `podman create` the new state reported is `created` but the `toolbox` code is not checking for that state https://github.com/coreos/toolbox/blob/main/rhcos-toolbox#L56 Workaround: Manually run the toolbox container: podman run -it --name toolbox-root --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=toolbox-root -e IMAGE=registry.redhat.io/rhel8/support-tools -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-tools --- Additional comment from Micah Abbott on 2022-06-02 19:27:13 UTC --- This is going to be a problem when we move to RHEL 8.6 --- Additional comment from Aashish Radhakrishnan on 2022-06-13 18:53:59 UTC --- Addressed the issue by adding the state 'created' in the conditional statements of the toolbox state : https://github.com/coreos/toolbox/pull/78 --- Additional comment from Micah Abbott on 2022-07-05 14:52:43 UTC --- While it would nice to get this as part of OCP 4.11, the code freeze deadline has passed and we'll have to target this as part of OCP 4.12 We can easily backport this to 4.11.z in the near future.
Addressed the issue by adding the state 'created' in the conditional statements of the toolbox state : https://github.com/coreos/toolbox/pull/78
Test with scratch build in https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=47840764, toolbox works well [core@cosa-devsh ~]$ rpm -q toolbox toolbox-0.1.0-1.rhaos4.11.el8.noarch [core@cosa-devsh ~]$ toolbox Trying to pull registry.redhat.io/rhel8/support-tools:latest... Getting image source signatures Checking if image destination supports signatures Copying blob 1b3417e31a5e done Copying blob 809fe483e885 done Copying blob 4c54166b2fbb done Copying config 99f45b4332 done Writing manifest to image destination Storing signatures 99f45b4332e9bd0c20f726c38b03e52a9d4e0ed28350b770c2b3f858174b47db Spawning a container 'toolbox-core' with image 'registry.redhat.io/rhel8/support-tools' Detected RUN label in the container image. Using that as the default... aefb6845361cb1729e356f92d05ffa2bee3b9e66f9e897d6f525ea9ab1940b56 toolbox-core Container started successfully. To exit, type 'exit'.
Moving back to MODIFIED as it's now in a 4.11 build.
Verified on 4.11.0-0.nightly-2022-09-22-034852 which is running RHCOS 411.86.202209211811-0 with podman-4.0.2-6.rhaos4.11.el8.x86_64 and toolbox-0.1.0-1.rhaos4.11.el8.noarch $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.11.0-0.nightly-2022-09-22-034852 True False 8m11s Cluster version is 4.11.0-0.nightly-2022-09-22-034852 $ oc get nodes NAME STATUS ROLES AGE VERSION ip-10-0-129-129.us-east-2.compute.internal Ready worker 18m v1.24.0+3882f8f ip-10-0-139-157.us-east-2.compute.internal Ready worker 16m v1.24.0+3882f8f ip-10-0-145-187.us-east-2.compute.internal Ready master 23m v1.24.0+3882f8f ip-10-0-160-68.us-east-2.compute.internal Ready master 24m v1.24.0+3882f8f ip-10-0-221-10.us-east-2.compute.internal Ready master 23m v1.24.0+3882f8f ip-10-0-234-2.us-east-2.compute.internal Ready worker 17m v1.24.0+3882f8f $ oc debug node/ip-10-0-139-157.us-east-2.compute.internal Warning: would violate PodSecurity "restricted:latest": host namespaces (hostNetwork=true, hostPID=true), privileged (container "container-00" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "container-00" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "container-00" must set securityContext.capabilities.drop=["ALL"]), restricted volume types (volume "host" uses restricted volume type "hostPath"), runAsNonRoot != true (pod or container "container-00" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "container-00" must not set runAsUser=0), seccompProfile (pod or container "container-00" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") Starting pod/ip-10-0-139-157us-east-2computeinternal-debug ... To use host binaries, run `chroot /host` Pod IP: 10.0.139.157 If you don't see a command prompt, try pressing enter. sh-4.4# chroot host sh-4.4# rpm-ostree status State: idle Deployments: * pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:42049fccd994c2bd4be11005a788c6e3e9590e32b2c95bd8174984da45585804 CustomOrigin: Managed by machine-config-operator Version: 411.86.202209211811-0 (2022-09-21T18:13:54Z) sh-4.4# rpm -q podman toolbox podman-4.0.2-6.rhaos4.11.el8.x86_64 toolbox-0.1.0-1.rhaos4.11.el8.noarch sh-4.4# toolbox Trying to pull registry.redhat.io/rhel8/support-tools:latest... Getting image source signatures Checking if image destination supports signatures Copying blob 4c54166b2fbb done Copying blob 1b3417e31a5e done Copying blob 809fe483e885 done Copying config 99f45b4332 done Writing manifest to image destination Storing signatures 99f45b4332e9bd0c20f726c38b03e52a9d4e0ed28350b770c2b3f858174b47db Spawning a container 'toolbox-root' with image 'registry.redhat.io/rhel8/support-tools' Detected RUN label in the container image. Using that as the default... b9a5b2fa5c9e83cd909f4d5fd0aa01fb11a1d25a53d30b9d3bb90daa72097ad3 toolbox-root Container started successfully. To exit, type 'exit'. [root@ip-10-0-139-157 /]# [root@ip-10-0-139-157 /]# cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.6 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.6" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.6" [root@ip-10-0-139-157 /]# exit exit sh-4.4# exit exit sh-4.4# exit exit Removing debug pod ...
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.11.6 packages update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:6658