Bug 2093040

Summary: unable to start `toolbox` on RHCOS using `podman` 4.0
Product: OpenShift Container Platform Reporter: Micah Abbott <miabbott>
Component: RHCOSAssignee: Timothée Ravier <travier>
Status: CLOSED ERRATA QA Contact: Michael Nguyen <mnguyen>
Severity: high Docs Contact:
Priority: high    
Version: 4.11CC: aaradhak, dornelas, hhei, jdohmann, jligon, miabbott, mrussell, nstielau, travier
Target Milestone: ---   
Target Release: 4.12.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
* Previously, updating to Podman 4.0 prevented users from running the `toolbox` command on {op-system}. This fix updates the toolbox library code to account for the new Podman behavior, so users can now run `toolbox` on {op-system} as expected. (link:https://bugzilla.redhat.com/show_bug.cgi?id=2093040[*BZ#2093040*])
 Story Points: ---
Clone Of:
: 2104116 (view as bug list) Environment:
Last Closed: 2023-01-17 19:49:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2048789, 2104116, 2105456    

Description Micah Abbott 2022-06-02 19:26:52 UTC 
RHCOS Version at Install Time: 4.11 dev build
Platform: qemu
Architecture: x86_64


What are you trying to do? What is your use case?

Run `toolbox` on a single RHCOS node that has `podman` 4.0 installed.

What happened? What went wrong or what did you expect?

The `toolbox` command reports that the container is in an unknown state `created`:

```
[core@cosa-devsh ~]$ rpm -q podman toolbox
podman-4.0.2-6.module+el8.6.0+14877+f643d2d6.x86_64
toolbox-0.0.9-1.rhaos4.11.el8.noarch
[core@cosa-devsh ~]$ toolbox
Trying to pull registry.redhat.io/rhel8/support-tools:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob cae7320018ab done  
Copying blob f70d60810c69 done  
Copying blob 545277d80005 done  
Copying config a8419564b7 done  
Writing manifest to image destination
Storing signatures
a8419564b7cc1417d26300d4560ce5812dccaee0c5ec88e22956ad5e67b32037
Spawning a container 'toolbox-core' with image 'registry.redhat.io/rhel8/support-tools'
Detected RUN label in the container image. Using that as the default...
220b8a1616e3993fcfe479ad283fec70847ac65be533e6d99d15f0afe84bf1fd
Container 'toolbox-core' in unknown state: 'created'
```

What are the steps to reproduce your issue? Please try to reduce these steps to something that can be reproduced with a single RHCOS node.


1.  Build RHCOS 4.11 with RHEL 8.6 content, including `podman` 4.0
2.  `cosa run`
3.  `toolbox`


It looks like in `podman` 4.0 when you do `podman create` the new state reported is `created` but the `toolbox` code is not checking for that state

https://github.com/coreos/toolbox/blob/main/rhcos-toolbox#L56


Workaround:

Manually run the toolbox container:

podman run -it --name toolbox-root --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=toolbox-root -e IMAGE=registry.redhat.io/rhel8/support-tools -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-tools
Comment 1 Micah Abbott 2022-06-02 19:27:13 UTC 
This is going to be a problem when we move to RHEL 8.6
Comment 2 Aashish Radhakrishnan 2022-06-13 18:53:59 UTC 
Addressed the issue by adding the state 'created' in the conditional statements of the toolbox state : https://github.com/coreos/toolbox/pull/78
Comment 3 Micah Abbott 2022-07-05 14:52:43 UTC 
While it would nice to get this as part of OCP 4.11, the code freeze deadline has passed and we'll have to target this as part of OCP 4.12

We can easily backport this to 4.11.z in the near future.
Comment 5 HuijingHei 2022-09-05 03:40:25 UTC 
Test with dev build 412.86.202209030446-0

[core@cosa-devsh ~]$ rpm -q podman toolbox
podman-4.2.0-1.rhaos4.12.el8.x86_64
toolbox-0.1.0-1.rhaos4.12.el8.noarch
[core@cosa-devsh ~]$ toolbox
Trying to pull registry.redhat.io/rhel8/support-tools:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob db0f4cd41250 done  
Copying blob 7e3624512448 done  
Copying blob 1f10f4923dcd done  
Copying config 5ef18a77eb done  
Writing manifest to image destination
Storing signatures
5ef18a77eb271a205c3c4604a67209126dd4d0831f63891ba1e33808c1fd36d7
Spawning a container 'toolbox-core' with image 'registry.redhat.io/rhel8/support-tools'
Detected RUN label in the container image. Using that as the default...
fcaaf733962366cdef6b5ccb55b7e955290331050388331f1018c0172ca3d9b6
toolbox-core
Container started successfully. To exit, type 'exit'.

[root@cosa-devsh /]# cat /etc/os-release 
NAME="Red Hat Enterprise Linux"
VERSION="8.6 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.6"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
Comment 8 errata-xmlrpc 2023-01-17 19:49:30 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7399