Bug 1998191
| Summary: | Suggest a way forward if coreos/toolbox was used | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Alex Jia <ajia> | |
| Component: | toolbox | Assignee: | Debarshi Ray <debarshir> | |
| Status: | CLOSED ERRATA | QA Contact: | atomic-bugs <atomic-bugs> | |
| Severity: | high | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 8.5 | CC: | debarshir, dornelas, lmiksik, sbarcomb, tpopela | |
| Target Milestone: | rc | Keywords: | Triaged | |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
|
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | container-tools-rhel8-8050020210902170952.faa19cc5 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2000914 2009626 (view as bug list) | Environment: | ||
| Last Closed: | 2021-11-09 17:40:16 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2000914, 2009626 | |||
(In reply to Alex Jia from comment #0) > [root@kvm-07-guest10 ~]# podman rm -f support-tools > WARN[0000] lstat > /sys/fs/cgroup/devices/machine.slice/libpod- > 26b987c9d039968bb4e3206ff572d08c5d4d886eee079922a419f5bfb398c827.scope: no > such file or directory > 26b987c9d039968bb4e3206ff572d08c5d4d886eee079922a419f5bfb398c827 > BTW, is it a necessary WARN in here? I actually think that this was touched during one of the meetings that we had in past about the rebase with Derrick and Scott. My impression was that the backwards compatibility is not a problem if it's documented. On the other hand if it would be easy to somehow maintain the backwards compatibility, then it would be appreciated by our customers. The new toolbox will completely error out if old toolbox was ever run on the system. Reproducer: 1. Run old bash script toolbox once and exit: # /tmp/rhcos-toolbox Spawning a container 'toolbox-root' with image 'registry.redhat.io/rhel8/support-tools' Detected RUN label in the container image. Using that as the default... [root@rhel84 /]# exit exit 2. Run new toolbox: # rpm -q toolbox toolbox-0.0.99.3-0.3.module+el8.5.0+12372+12f82d56.x86_64 # toolbox --version toolbox version 0.0.99.2 # toolbox create Error: TOOLBOX_PATH not set The cause is that using old toolbox we mounted host /run to container /run (-v /run:/run), and the container creates a /run/.containerenv file that's supposed to only live in the container. We basically leaked it out of the container. Removing the file on the host fixes the problem: # rm -f /run/.containerenv # toolbox create Created container: support-tools-latest Enter with: toolbox enter We essentially break everyone that's used toolbox before. I feel like we should handle this in some way. The backwards compatibility discussions we had previously were around the config options and UX/CLI changes. Those are acceptable changes. > We essentially break everyone that's used toolbox before.
> I feel like we should handle this in some way. The backwards
> compatibility discussions we had previously were around
> the config options and UX/CLI changes. Those are acceptable
> changes.
How do you want to handle it?
Is it enough if the following error message said something else:
[root@kvm-07-guest10 ~]# toolbox enter support-tools
Error: container support-tools is too old and no longer supported
Recreate it with Toolbox version 0.0.17 or newer.
The fact that a /run/.containerenv gets placed on the host is a bit thorny. We might be able to carefully side-step it, but it depends on what we exactly decide to do.
I think the simplest solution might be: if env variables $TOOLBOX_PATH and $container are not set and file /run/.containerenv exists, then print an error indicating the user may need to manually remove /run/.containerenv from their system. This at least hints at how to resolve the issue. We'll likely create a kbase solution customers can search for that will explain the issue in more detail. > I think the simplest solution might be: if env variables
> $TOOLBOX_PATH and $container are not set and file
> /run/.containerenv exists, then print an error indicating
> the user may need to manually remove /run/.containerenv
> from their system.
Ok, this sounds doable.
I got a patch that does: [root@kvm-07-guest10 ~]# toolbox enter support-tools Error: /run/.containerenv found on what looks like the host If this is the host, then remove /run/.containerenv and try again. Otherwise, contact your system administrator or file a bug. Does it look OK? Looks good to me, thanks! Built toolbox-0.0.99.3-0.3.module+el8.5.0+12477+44413d02: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1717786 This bug has been verified on toolbox-0.0.99.3-0.3.module+el8.5.0+12477+44413d02. [root@hpe-dl380pgen8-02-vm-8 ~]# rpm -q toolbox toolbox-0.0.99.3-0.3.module+el8.5.0+12477+44413d02.x86_64 [root@hpe-dl380pgen8-02-vm-8 ~]# toolbox Error: /run/.containerenv found on what looks like the host If this is the host, then remove /run/.containerenv and try again. Otherwise, contact your system administrator or file a bug. [root@hpe-dl380pgen8-02-vm-8 ~]# rm -rf /run/.containerenv [root@hpe-dl380pgen8-02-vm-8 ~]# toolbox ⬢[root@toolbox ~]# echo $HOST /run/host ⬢[root@toolbox ~]# exit logout [root@hpe-dl380pgen8-02-vm-8 ~]# echo $? 0 Closing this bug as VERIFIED per Comment 15. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:4154 |
Description of problem: This issue is found by Debarshi, we never talked about backwards compatibility with existing containers created by the older implementation. Version-Release number of selected component (if applicable): [root@kvm-07-guest10 ~]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.5 Beta (Ootpa) [root@kvm-07-guest10 ~]# rpm -q toolbox podman runc kernel toolbox-0.0.99.3-0.1.module+el8.5.0+12381+e822eb26.x86_64 podman-3.3.1-1.module+el8.5.0+12381+e822eb26.x86_64 runc-1.0.2-1.module+el8.5.0+12381+e822eb26.x86_64 kernel-4.18.0-335.el8.x86_64 upgrading toolbox, podman, runc, ... [root@kvm-07-guest10 ~]# rpm -q toolbox podman runc kernel toolbox-0.0.99.3-0.3.module+el8.5.0+12372+12f82d56.x86_64 podman-4.0.0-0.1.module+el8.5.0+12372+12f82d56.x86_64 runc-1.0.1-5.module+el8.5.0+12234+e4609207.x86_64 kernel-4.18.0-335.el8.x86_64 How reproducible: always Steps to Reproduce: 1. creating support-tools container by an old toolbox version 2. upgrade toolbox to an new version 3. toolbox enter previous existing support-tools container 4. podman rm -f support-tools 5. toolbox create --image registry.redhat.io/rhel8/support-tools 6. toolbox enter Actual results: 1. toolbox-0.0.99.3-0.1.module+el8.5.0+12381+e822eb26 [root@kvm-07-guest10 ~]# podman images REPOSITORY TAG IMAGE ID CREATED SIZE registry.redhat.io/rhel8/support-tools latest 6ad911cd1a8b 3 weeks ago 279 MB [root@kvm-07-guest10 ~]# toolbox create --image registry.redhat.io/rhel8/support-tools Created container: support-tools Enter with: toolbox enter support-tools [root@kvm-07-guest10 ~]# toolbox enter support-tools ⬢[root@toolbox ~]# ls /run/host bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var 2. upgrading toolbox to 0.0.99.3-0.3.module+el8.5.0+12372+12f82d56 [root@kvm-07-guest10 ~]# toolbox create --image registry.redhat.io/rhel8/support-tools Error: TOOLBOX_PATH not set [root@kvm-07-guest10 ~]# rm -rf /run/.containerenv [root@kvm-07-guest10 ~]# toolbox create --image registry.redhat.io/rhel8/support-tools Error: container support-tools already exists Enter with: toolbox enter support-tools Run 'toolbox --help' for usage. [root@kvm-07-guest10 ~]# toolbox enter support-tools Error: container support-tools is too old and no longer supported Recreate it with Toolbox version 0.0.17 or newer. [root@kvm-07-guest10 ~]# podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 26b987c9d039 registry.redhat.io/rhel8/support-tools:latest /usr/bin/bash 11 hours ago Up 7 minutes ago support-tools [root@kvm-07-guest10 ~]# podman exec -i support-tools cat /etc/redhat-release Red Hat Enterprise Linux release 8.4 (Ootpa) [root@kvm-07-guest10 ~]# toolbox create --image registry.access.redhat.com/ubi8 Error: TOOLBOX_PATH not set [root@kvm-07-guest10 ~]# podman rm -f support-tools WARN[0000] lstat /sys/fs/cgroup/devices/machine.slice/libpod-26b987c9d039968bb4e3206ff572d08c5d4d886eee079922a419f5bfb398c827.scope: no such file or directory 26b987c9d039968bb4e3206ff572d08c5d4d886eee079922a419f5bfb398c827 [root@kvm-07-guest10 ~]# toolbox create --image registry.redhat.io/rhel8/support-tools Error: TOOLBOX_PATH not set [root@kvm-07-guest10 ~]# ls /run/.containerenv -lah -rwxr-xr-x. 1 root root 0 Aug 26 09:40 /run/.containerenv [root@kvm-07-guest10 ~]# rpm -qf /run/.containerenv file /run/.containerenv is not owned by any package [root@kvm-07-guest10 ~]# rm -rf /run/.containerenv [root@kvm-07-guest10 ~]# toolbox create --image registry.redhat.io/rhel8/support-tools Created container: support-tools Enter with: toolbox enter support-tools [root@kvm-07-guest10 ~]# toolbox enter support-tools ⬢[root@toolbox ~]# echo $HOST /run/host Expected results: fix it. Additional info: Currently, on a freshly booted system, we will get: # toolbox enter toolbox-root Error: container toolbox-root is too old and no longer supported Recreate it with Toolbox version 0.0.17 or newer. If the system isn't freshly booted, and the older implementation was used, then we will get less elegant failure: # toolbox enter toolbox-root Error: TOOLBOX_PATH not set