Bug 2093037

OCP Version at Install Time: 4.11
RHCOS Version at Install Time: 411.85.202206011118-0
Platform: AWS
Architecture: x86_64


What are you trying to do? What is your use case?


Run `sos report` from a `toolbox` container in the `oc debug node` environment


What happened? What went wrong or what did you expect?

After entering the `chroot /host` on the node, the `sos` command is not present as expected.


```
$ oc debug node/ip-10-0-151-23.ec2.internal
Warning: would violate PodSecurity "restricted:latest": host namespaces (hostNetwork=true, hostPID=true), privileged (container "container-00" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "container-00" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "container-00" must set securityContext.capabilities.drop=["ALL"]), restricted volume types (volume "host" uses restricted volume type "hostPath"), runAsNonRoot != true (pod or container "container-00" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "container-00" must not set runAsUser=0), seccompProfile (pod or container "container-00" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
Starting pod/ip-10-0-151-23ec2internal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.151.23
If you don't see a command prompt, try pressing enter.
sh-4.4# chroot /host
sh-4.4# toolbox
Checking if there is a newer version of registry.redhat.io/rhel8/support-tools available...
Container 'toolbox-root' already exists. Trying to start...
(To remove the container and start with a fresh toolbox, run: sudo podman rm 'toolbox-root')
toolbox-root
Container started successfully. To exit, type 'exit'.
[root@ip-10-0-151-23 /]# sos
bash: sos: command not found
[root@ip-10-0-151-23 /]# sos report
bash: sos: command not found
```

What are the steps to reproduce your issue? Please try to reduce these steps to something that can be reproduced with a single RHCOS node.

This doesn't reproduce on a RHCOS node via `cosa run`; I suspect it is something to do with the namespaces that are created via `oc debug node` and how they affect the ability to use `toolbox` container started via `podman`.

On an OCP cluster:

```
$ oc debug node/<node name>
$ chroot /host
# toolbox
# sos report
```

This was originally reported in a comment to bug 1915537


Workaround:

Manually start the `toolbox` container via `podman`:

```
sh-4.4# podman run -it --name toolbox-root --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=toolbox-root -e IMAGE=registry.redhat.io/rhel8/support-tools -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-ools
[root@ip-10-0-151-23 /]# sos report

sosreport (version 4.2)

This command will collect diagnostic and configuration information from
this Red Hat CoreOS system.

An archive containing the collected information will be generated in
/host/var/tmp/sos.soifck4b and may be provided to a Red Hat support
representative.

Any information provided to Red Hat will be treated in accordance with
the published support policies at:

        Distribution Website : https://www.redhat.com/
        Commercial Support   : https://www.access.redhat.com/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.

Press ENTER to continue, or CTRL-C to quit.
^CExiting on user cancel
```