Bug 1825283
| Summary: | Running sosreport in container without setting ENV_HOST_SYSROOT fails with traceback | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Robb Manes <rmanes> |
| Component: | sos | Assignee: | Pavel Moravec <pmoravec> |
| Status: | CLOSED ERRATA | QA Contact: | Maros Kopec <makopec> |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | 8.1 | CC: | agk, bmr, dornelas, fedoraproject, jhunsaker, mhradile, plambri, pmoravec, sbradley |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
| Target Release: | 8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | sos-3.9.1-2.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-11-04 01:57:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1186913 | ||
The traceback is definitely unwelcome, but how are you running this and in what container? The support-tools container is (currently) the only supported way to run sos in a container environment on Red Hat products, and these environments are expected to set HOST (and possibly CONTAINER too, although that has not always been reliable). If that's not getting set then that's potentially going to be a problem depending on what you're expecting since sos will end up inspecting only the container namespace. The output of 'env' from the container would be useful. Sure; container is running like so: $ podman run -it --rm --name sosreport registry.redhat.io/rhel8/support-tools bash [root@86989c8c709a /]# env LANG=C.utf8 HOSTNAME=86989c8c709a container=oci PWD=/ HOME=/root TERM=xterm SHLVL=1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin _=/usr/bin/env We're not using it mounted with `toolbox` options, which work as expected since the `toolbox` method of running support-tools explicitly sets ENV_HOST_SYSROOT to `/' and provides a mount to gather from the host (which ostensibly has no sosreport package). I'm testing this in CoreOS now. This started from a scenario where we were attempting to gather data from inside of a container, to get other values. An argument could be made that gathering `podman` and `docker` commands could give us most values; if we only support this running in a support-tools image container, we can update docs to reflect this (or I can find them and provide them to interested parties); I did not know that. Perhaps defaulting to just explicitly saying "only can run in support-tools" when executing sosreport is enough; see the discussion I've had with Jake in the PR listed. Thanks! Another comment; this was spawned from a customer running `sosreport` in a rhel8 application base image container for an unrelated issue, we were trying to get information from the containerized environment and someone asked for it. They installed it from base repos as it was on a RHEL host. I am using support-tools just to reproduce. Thanks for investigation and upstream PR. So far I plan to have it fixed in 8.3. From testing perspective: is there a better scenario than "unset ENV_HOST_SYSROOT in a container and call sosreport there"? (In reply to Pavel Moravec from comment #4) > Thanks for investigation and upstream PR. So far I plan to have it fixed in > 8.3. > > From testing perspective: is there a better scenario than "unset > ENV_HOST_SYSROOT in a container and call sosreport there"? Hi Robb, could you please provide some verification steps or confirm you will verify a candidate build by yourself, once the build is available? (In reply to Robb Manes from comment #0) > > Welcome to thoughts and changes; if it's not intended for us to run in a > container this way, we should catch this and throw an error instead of the > traceback. > Sorry if I've missed something here, but I think the correct thing to do here is throw an error and exit when sosreport is not run via toolbox[1] (or more specifically, when it's not run via the support-tools "run" label). # podman inspect registry.redhat.io/rhel8/support-tools | jq '.[].Labels.run' "podman run -it --name NAME --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=NAME -e IMAGE=IMAGE -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host IMAGE" # podman inspect registry.redhat.io/rhel8/support-tools | jq '.[].Labels.usage' "podman container runlabel RUN rhel8/support-tools" # podman container runlabel RUN rhel8/support-tools command: podman run -it --name support-tools --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=support-tools -e IMAGE=registry.redhat.io/rhel8/support-tools:latest -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-tools:latest [root@podman2 /]# echo $HOST, $NAME, $IMAGE /host, support-tools, registry.redhat.io/rhel8/support-tools:latest I don't see much value in being able to run sosreport in a container for the express purpose of getting container-only data. We can either start with an existing image or recreate the image from a Dockerfile/Containerfile, and from there reproduce the customer's issue. The only kind of data we would be missing using this method might be application source code and/or custom binaries, and sosreport wouldn't gather that data anyways. All config data we would be interested in should usually be available as part of the image build artifacts (or possibly volume mounts at run time). I would like to avoid people trying to run sosreport in a container reguarly, expecting to generate any useful data. For one, I think that will likely generate more questions/bug reports for sos. [1] https://github.com/coreos/toolbox/blob/0.0.7/rhcos-toolbox Hello, I decided to backport the upstream PR to 8.3. As particular reproducer steps are not known to us, could you please verify the bugfix by yourself? A yum repository for the build of sos-3.9.1-2.el8 (task 28860028) is available at: http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/2.el8/ You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory: http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/2.el8/sos-3.9.1-2.el8.repo RPMs and build logs can be found in the following locations: http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/2.el8/noarch/ The full list of available rpms is: http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/2.el8/noarch/sos-3.9.1-2.el8.src.rpm http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/2.el8/noarch/sos-3.9.1-2.el8.noarch.rpm http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/2.el8/noarch/sos-audit-3.9.1-2.el8.noarch.rpm The repository will be available for the next 45 days. Tested this on the RHEL-8.3.0-20200706.n.0 compose + podman-2.0.2-1.module+el8.3.0+7303+7fef20f0.x86_64 # podman create --name test -ti ubi8 bash Trying to pull registry.access.redhat.com/ubi8... Getting image source signatures Copying blob 264aaa453271 done Copying blob 7f24bdb73d53 done Copying config a523835cfc done Writing manifest to image destination Storing signatures d838ab22dbd449db5ac872efac9472c3934de1032dc33dcf687354638ddcf80f # podman cp /etc/yum.repos.d/compose.repo test:/etc/yum.repos.d/compose.repo # podman start -ai test [root@d838ab22dbd4 /]# yum install sos Updating Subscription Management repositories. Unable to read consumer identity Subscription Manager is operating in container mode. This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. latest-RHEL-8.3.0 BaseOS 40 MB/s | 2.3 MB 00:00 latest-RHEL-8.3.0 AppStream 52 MB/s | 6.3 MB 00:00 Red Hat Universal Base Image 8 (RPMs) - BaseOS 2.8 MB/s | 766 kB 00:00 Red Hat Universal Base Image 8 (RPMs) - AppStream 18 MB/s | 3.9 MB 00:00 Red Hat Universal Base Image 8 (RPMs) - CodeReady Builder 112 kB/s | 11 kB 00:00 Dependencies resolved. ================================================================================================================================================================ Package Architecture Version Repository Size ================================================================================================================================================================ Installing: sos noarch 3.9.1-4.el8 baseos 545 k Installing dependencies: bzip2 x86_64 1.0.6-26.el8 baseos 60 k xz x86_64 5.2.4-3.el8 baseos 153 k Transaction Summary ================================================================================================================================================================ Install 3 Packages Total download size: 759 k Installed size: 2.1 M Is this ok [y/N]: y Downloading Packages: (1/3): bzip2-1.0.6-26.el8.x86_64.rpm 3.7 MB/s | 60 kB 00:00 (2/3): xz-5.2.4-3.el8.x86_64.rpm 8.0 MB/s | 153 kB 00:00 (3/3): sos-3.9.1-4.el8.noarch.rpm 22 MB/s | 545 kB 00:00 ---------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 27 MB/s | 759 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : xz-5.2.4-3.el8.x86_64 1/3 Installing : bzip2-1.0.6-26.el8.x86_64 2/3 Installing : sos-3.9.1-4.el8.noarch 3/3 Running scriptlet: sos-3.9.1-4.el8.noarch 3/3 Verifying : bzip2-1.0.6-26.el8.x86_64 1/3 Verifying : sos-3.9.1-4.el8.noarch 2/3 Verifying : xz-5.2.4-3.el8.x86_64 3/3 Installed products updated. Installed: bzip2-1.0.6-26.el8.x86_64 sos-3.9.1-4.el8.noarch xz-5.2.4-3.el8.x86_64 Complete! [root@d838ab22dbd4 /]# sosreport sosreport (version 3.9) WARNING: unable to set option for disabled or non-existing plugin (boot) WARNING: unable to set option for disabled or non-existing plugin (boot) This command will collect diagnostic and configuration information from this Red Hat Enterprise Linux system and installed applications. An archive containing the collected information will be generated in /var/tmp/sos.y3i2og36 and may be provided to a Red Hat support representative. Any information provided to Red Hat will be treated in accordance with the published support policies at: https://access.redhat.com/support/ The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. Press ENTER to continue, or CTRL-C to quit. Please enter the case id that you are generating this report for []: Setting up archive ... Setting up plugins ... [plugin:networking] skipped command 'nft list ruleset': required kmods missing: nf_tables. [plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec. Use '--allow-system-changes' to enable collection. [plugin:networking] skipped command 'ss -peaonmi': required kmods missing: udp_diag, unix_diag, tcp_diag, netlink_diag, inet_diag, af_packet_diag. Use '--allow-system-changes' to enable collection. [plugin:networking] skipped command 'iptables -vnxL': required kmods missing: nf_tables, iptable_filter. [plugin:networking] skipped command 'ip6tables -vnxL': required kmods missing: ip6table_filter, nf_tables. Running plugins. Please wait ... Finishing plugins [Running: subscription_manager] Finished running plugins Creating compressed archive... Your sosreport has been generated and saved in: /var/tmp/sosreport-d838ab22dbd4-2020-07-09-riktmlm.tar.xz Size 3.31MiB Owner root md5 01e882db34e68297e5ba51ed2a5a2f21 Please send this file to your support representative. So the original traceback is resolved, but I would still rather see an error/warning that explains how to run sosreport correctly in a container. (In reply to Derrick Ornelas from comment #10) > So the original traceback is resolved, but I would still rather see an > error/warning that explains how to run sosreport correctly in a container. I would expect "run it as a regular command there", since sosreport does detect well (I hope) it is running inside a container: https://github.com/sosreport/sos/blob/master/sos/policies/redhat.py#L142-L144 and it sets sysroot and similar stuff accordingly, then. Is this procedure invalid for some use case? (there was a change around that lines recently, maybe relevant? see https://github.com/sosreport/sos/pull/2028) Or do you require a dedicated log in stdout "sosreport is running inside a container" whenever this happens? Running sosreport in this way just collects a small amount of data from the container, but the intent (in almost all cases) is for sosreport running in the container to collect data from the host. The container will not have any access to the host if it is not started with the correct options. These options are pre-configured in the support-tools container image, and they are utilized properly when 'podman container runlabel RUN ...' is used with the image. So, if you just install and run sosreport in an unprivileged container all it does is collect data about the container. The data will be limited, because the contents of the container are very limited by design. Our base images do not even have procps-ng installed, so you won't even get something as simple as the ps output. The two problems I foresee are: 1. customers may start running sosreport in containers which are not configured correctly and giving those to support, and there is confusion about why the data does not match the host that is being debugged 2. customers may start running soreport in containers as part of troubleshooting a container issue, and the data captured is not useful and delays resolution I think we could avoid these issues by giving a warning. I hope that makes sense. (In reply to Derrick Ornelas from comment #12) > Running sosreport in this way just collects a small amount of data from the > container, but the intent (in almost all cases) is for sosreport running in > the container to collect data from the host. The container will not have > any access to the host if it is not started with the correct options. These > options are pre-configured in the support-tools container image, and they > are utilized properly when 'podman container runlabel RUN ...' is used with > the image. > > So, if you just install and run sosreport in an unprivileged container all > it does is collect data about the container. The data will be limited, > because the contents of the container are very limited by design. Our base > images do not even have procps-ng installed, so you won't even get something > as simple as the ps output. > > The two problems I foresee are: > > 1. customers may start running sosreport in containers which are not > configured correctly and giving those to support, and there is confusion > about why the data does not match the host that is being debugged > > 2. customers may start running soreport in containers as part of > troubleshooting a container issue, and the data captured is not useful and > delays resolution > > I think we could avoid these issues by giving a warning. I hope that makes > sense. I would agree to put such warning. As a container noob, could you please describe for me particular conditions detecting either scenario? I.e. what to fill in "if <test-of-unpriviledged-container> then raise warning" ? Jake, since you work on both sosreport and the support-tools image can you help with this topic? I would say that, at a minimum, sosreport could/should check for $HOST and a valid rootfs at that location. Does sosreport honor $HOST (meaning I can say 'podman ... -e HOST=/myroot -v /:/myroot ...') or is the default /host in the RUN label all we really can support? For checking that the container is running as privileged the best I can come up with is checking that the sosreport process itself is running with the spc_t[1] selinux context type (https://danwalsh.livejournal.com/74754.html) A quick looks says this will need either python3-psutil or procps-ng, neither of which is required by sos or installed by default in the support-tools image. I'll open a new BZ to get procps-ng added to the image either way. Afaik you can just peel those values straight from /proc/self/attr/current - no need for extra packages: $ cat /proc/self/attr/current unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 However the values will be specific to the LSM in use - for us that's SELinux and spc_t but if this is going to be a part of sosreport we'll need to be able to do the right thing with other security mechanisms too. sos /should/ honour setting HOST (or the equivalent --sysroot SYSROOT command line option) - this is how it's designed to work. I just noticed there's possibly a small problem right now when running with sos collector since the value of sos_path_strip is not updated to use a value from HOST/--sysroot, although I haven't tested this to confirm the behaviour I expect. (In reply to Derrick Ornelas from comment #14) > I would say that, at a minimum, sosreport could/should check for $HOST and a > valid rootfs at that location. Does sosreport honor $HOST (meaning I can > say 'podman ... -e HOST=/myroot -v /:/myroot ...') or is the default /host > in the RUN label all we really can support? > sos will check for the `HOST` env var and *IF* the `container` env var is also set, then the value of $HOST is honored by the policy and collections will be from the host, and not the container. It currently does not matter what the `container` env var is set to. If either/both of these are missing and sos is run in a container, then in 3.x sos will abort (in earlier versions, ungracefully) and in 4.0+ it will collect information from the container (see the discussion in https://github.com/sosreport/sos/pull/2028). For our purposes here, podman (and docker for that matter) automatically adds the `container` env var, and support-tools adds the `HOST` variable when run via the RUN label. Manually setting or changing this inside an already running container should result in the same behavior, if perhaps at a different location. (In reply to Bryn M. Reeves from comment #15) > I just noticed > there's possibly a small problem right now when running with sos collector > since the value of sos_path_strip is not updated to use a value from > HOST/--sysroot, although I haven't tested this to confirm the behaviour I > expect. I think you're right - the sos_path_strip default is set assuming the end user is running via support-tools. I don't see a use case where setting $HOST to anything aside from the host's / would be useful, but it'd be a good idea to reflect changes there regardless. Manually tested on RHEL-8.2 x84_64 with podman version 1.9.3. I was not able to reproduce the traceback on podman with different version.
[root@ci-vm-10-0-138-202 ~]# podman run -it --rm --name sosreport registry.redhat.io/rhel8/support-tools bash
[root@2c00362db49c /]# env
LANG=C.utf8
HOSTNAME=2c00362db49c
container=oci
PWD=/
HOME=/root
TERM=xterm
SHLVL=1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/env
[root@2c00362db49c /]# rpm -qa sos
sos-3.8-7.el8_2.noarch
[root@2c00362db49c /]# sosreport
sosreport (version 3.8)
Traceback (most recent call last):
File "/usr/sbin/sosreport", line 19, in <module>
main(sys.argv[1:])
File "/usr/lib/python3.6/site-packages/sos/sosreport.py", line 1372, in main
sos = SoSReport(args)
File "/usr/lib/python3.6/site-packages/sos/sosreport.py", line 313, in __init__
self.policy = sos.policies.load(sysroot=self.opts.sysroot)
File "/usr/lib/python3.6/site-packages/sos/policies/__init__.py", line 44, in load
cache['policy'] = policy(sysroot=sysroot)
File "/usr/lib/python3.6/site-packages/sos/policies/redhat.py", line 252, in __init__
super(RHELPolicy, self).__init__(sysroot=sysroot)
File "/usr/lib/python3.6/site-packages/sos/policies/redhat.py", line 50, in __init__
sysroot = self._container_init()
File "/usr/lib/python3.6/site-packages/sos/policies/redhat.py", line 138, in _container_init
host_tmp_dir = os.path.abspath(self._host_sysroot + self._tmp_dir)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
[root@ci-vm-10-0-138-202 ~]# podman run -it --rm --name sosreport registry.redhat.io/rhel8/support-tools bash
[root@3b71799ab94a ~]# dnf install sos-3.9.1-6.el8.noarch.rpm
Updating Subscription Management repositories.
Unable to read consumer identity
Subscription Manager is operating in container mode.
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Red Hat Universal Base Image 8 (RPMs) - BaseOS 906 kB/s | 767 kB 00:00
Red Hat Universal Base Image 8 (RPMs) - AppStream 4.2 MB/s | 4.0 MB 00:00
Red Hat Universal Base Image 8 (RPMs) - CodeReady Builder 23 kB/s | 12 kB 00:00
Dependencies resolved.
==============================================================================================================================================================================================
Package Architecture Version Repository Size
==============================================================================================================================================================================================
Installing:
sos noarch 3.9.1-6.el8 @commandline 546 k
Installing dependencies:
bzip2 x86_64 1.0.6-26.el8 ubi-8-baseos 60 k
xz x86_64 5.2.4-3.el8 ubi-8-baseos 153 k
Transaction Summary
==============================================================================================================================================================================================
Install 3 Packages
Total size: 759 k
Total download size: 213 k
Installed size: 2.1 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): bzip2-1.0.6-26.el8.x86_64.rpm 199 kB/s | 60 kB 00:00
(2/2): xz-5.2.4-3.el8.x86_64.rpm 381 kB/s | 153 kB 00:00
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 524 kB/s | 213 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : bzip2-1.0.6-26.el8.x86_64 1/3
Installing : xz-5.2.4-3.el8.x86_64 2/3
Installing : sos-3.9.1-6.el8.noarch 3/3
Running scriptlet: sos-3.9.1-6.el8.noarch 3/3
Verifying : xz-5.2.4-3.el8.x86_64 1/3
Verifying : bzip2-1.0.6-26.el8.x86_64 2/3
Verifying : sos-3.9.1-6.el8.noarch 3/3
Installed products updated.
Installed:
bzip2-1.0.6-26.el8.x86_64 sos-3.9.1-6.el8.noarch xz-5.2.4-3.el8.x86_64
Complete!
[root@f8927b4f7dc0 ~]# sosreport --batch
sosreport (version 3.9)
WARNING: unable to set option for disabled or non-existing plugin (boot)
WARNING: unable to set option for disabled or non-existing plugin (boot)
This command will collect diagnostic and configuration information from
this Red Hat Enterprise Linux system and installed applications.
An archive containing the collected information will be generated in
/var/tmp/sos.xbzhs59g and may be provided to a Red Hat support
representative.
Any information provided to Red Hat will be treated in accordance with
the published support policies at:
https://access.redhat.com/support/
The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.
No changes will be made to system configuration.
Setting up archive ...
Setting up plugins ...
caught exception in plugin method "block.setup()"
caught exception in plugin method "block.setup()"
writing traceback to sos_logs/block-plugin-errors.txt
writing traceback to sos_logs/block-plugin-errors.txt
caught exception in plugin method "cgroups.setup()"
caught exception in plugin method "cgroups.setup()"
writing traceback to sos_logs/cgroups-plugin-errors.txt
writing traceback to sos_logs/cgroups-plugin-errors.txt
caught exception in plugin method "crypto.setup()"
caught exception in plugin method "crypto.setup()"
writing traceback to sos_logs/crypto-plugin-errors.txt
writing traceback to sos_logs/crypto-plugin-errors.txt
caught exception in plugin method "dbus.setup()"
caught exception in plugin method "dbus.setup()"
writing traceback to sos_logs/dbus-plugin-errors.txt
writing traceback to sos_logs/dbus-plugin-errors.txt
caught exception in plugin method "dnf.setup()"
caught exception in plugin method "dnf.setup()"
writing traceback to sos_logs/dnf-plugin-errors.txt
writing traceback to sos_logs/dnf-plugin-errors.txt
caught exception in plugin method "filesys.setup()"
caught exception in plugin method "filesys.setup()"
writing traceback to sos_logs/filesys-plugin-errors.txt
writing traceback to sos_logs/filesys-plugin-errors.txt
caught exception in plugin method "hardware.setup()"
caught exception in plugin method "hardware.setup()"
writing traceback to sos_logs/hardware-plugin-errors.txt
writing traceback to sos_logs/hardware-plugin-errors.txt
caught exception in plugin method "host.setup()"
caught exception in plugin method "host.setup()"
writing traceback to sos_logs/host-plugin-errors.txt
writing traceback to sos_logs/host-plugin-errors.txt
caught exception in plugin method "i18n.setup()"
caught exception in plugin method "i18n.setup()"
writing traceback to sos_logs/i18n-plugin-errors.txt
writing traceback to sos_logs/i18n-plugin-errors.txt
caught exception in plugin method "kernel.setup()"
caught exception in plugin method "kernel.setup()"
writing traceback to sos_logs/kernel-plugin-errors.txt
writing traceback to sos_logs/kernel-plugin-errors.txt
caught exception in plugin method "krb5.setup()"
caught exception in plugin method "krb5.setup()"
writing traceback to sos_logs/krb5-plugin-errors.txt
writing traceback to sos_logs/krb5-plugin-errors.txt
caught exception in plugin method "kvm.setup()"
caught exception in plugin method "kvm.setup()"
writing traceback to sos_logs/kvm-plugin-errors.txt
writing traceback to sos_logs/kvm-plugin-errors.txt
caught exception in plugin method "ldap.setup()"
caught exception in plugin method "ldap.setup()"
writing traceback to sos_logs/ldap-plugin-errors.txt
writing traceback to sos_logs/ldap-plugin-errors.txt
caught exception in plugin method "libraries.setup()"
caught exception in plugin method "libraries.setup()"
writing traceback to sos_logs/libraries-plugin-errors.txt
writing traceback to sos_logs/libraries-plugin-errors.txt
caught exception in plugin method "libvirt.setup()"
caught exception in plugin method "libvirt.setup()"
writing traceback to sos_logs/libvirt-plugin-errors.txt
writing traceback to sos_logs/libvirt-plugin-errors.txt
caught exception in plugin method "login.setup()"
caught exception in plugin method "login.setup()"
writing traceback to sos_logs/login-plugin-errors.txt
writing traceback to sos_logs/login-plugin-errors.txt
caught exception in plugin method "logrotate.setup()"
caught exception in plugin method "logrotate.setup()"
writing traceback to sos_logs/logrotate-plugin-errors.txt
writing traceback to sos_logs/logrotate-plugin-errors.txt
caught exception in plugin method "logs.setup()"
caught exception in plugin method "logs.setup()"
writing traceback to sos_logs/logs-plugin-errors.txt
writing traceback to sos_logs/logs-plugin-errors.txt
caught exception in plugin method "lvm2.setup()"
caught exception in plugin method "lvm2.setup()"
writing traceback to sos_logs/lvm2-plugin-errors.txt
writing traceback to sos_logs/lvm2-plugin-errors.txt
caught exception in plugin method "md.setup()"
caught exception in plugin method "md.setup()"
writing traceback to sos_logs/md-plugin-errors.txt
writing traceback to sos_logs/md-plugin-errors.txt
caught exception in plugin method "memory.setup()"
caught exception in plugin method "memory.setup()"
writing traceback to sos_logs/memory-plugin-errors.txt
writing traceback to sos_logs/memory-plugin-errors.txt
caught exception in plugin method "multipath.setup()"
caught exception in plugin method "multipath.setup()"
writing traceback to sos_logs/multipath-plugin-errors.txt
writing traceback to sos_logs/multipath-plugin-errors.txt
caught exception in plugin method "networking.setup()"
caught exception in plugin method "networking.setup()"
writing traceback to sos_logs/networking-plugin-errors.txt
writing traceback to sos_logs/networking-plugin-errors.txt
caught exception in plugin method "openhpi.setup()"
caught exception in plugin method "openhpi.setup()"
writing traceback to sos_logs/openhpi-plugin-errors.txt
writing traceback to sos_logs/openhpi-plugin-errors.txt
caught exception in plugin method "pam.setup()"
caught exception in plugin method "pam.setup()"
writing traceback to sos_logs/pam-plugin-errors.txt
writing traceback to sos_logs/pam-plugin-errors.txt
caught exception in plugin method "pci.setup()"
caught exception in plugin method "pci.setup()"
writing traceback to sos_logs/pci-plugin-errors.txt
writing traceback to sos_logs/pci-plugin-errors.txt
caught exception in plugin method "process.setup()"
caught exception in plugin method "process.setup()"
writing traceback to sos_logs/process-plugin-errors.txt
writing traceback to sos_logs/process-plugin-errors.txt
caught exception in plugin method "processor.setup()"
caught exception in plugin method "processor.setup()"
writing traceback to sos_logs/processor-plugin-errors.txt
writing traceback to sos_logs/processor-plugin-errors.txt
caught exception in plugin method "release.setup()"
caught exception in plugin method "release.setup()"
writing traceback to sos_logs/release-plugin-errors.txt
writing traceback to sos_logs/release-plugin-errors.txt
caught exception in plugin method "rpm.setup()"
caught exception in plugin method "rpm.setup()"
writing traceback to sos_logs/rpm-plugin-errors.txt
writing traceback to sos_logs/rpm-plugin-errors.txt
caught exception in plugin method "scsi.setup()"
caught exception in plugin method "scsi.setup()"
writing traceback to sos_logs/scsi-plugin-errors.txt
writing traceback to sos_logs/scsi-plugin-errors.txt
caught exception in plugin method "selinux.setup()"
caught exception in plugin method "selinux.setup()"
writing traceback to sos_logs/selinux-plugin-errors.txt
writing traceback to sos_logs/selinux-plugin-errors.txt
caught exception in plugin method "services.setup()"
caught exception in plugin method "services.setup()"
writing traceback to sos_logs/services-plugin-errors.txt
writing traceback to sos_logs/services-plugin-errors.txt
caught exception in plugin method "ssh.setup()"
caught exception in plugin method "ssh.setup()"
writing traceback to sos_logs/ssh-plugin-errors.txt
writing traceback to sos_logs/ssh-plugin-errors.txt
caught exception in plugin method "subscription_manager.setup()"
caught exception in plugin method "subscription_manager.setup()"
writing traceback to sos_logs/subscription_manager-plugin-errors.txt
writing traceback to sos_logs/subscription_manager-plugin-errors.txt
caught exception in plugin method "system.setup()"
caught exception in plugin method "system.setup()"
writing traceback to sos_logs/system-plugin-errors.txt
writing traceback to sos_logs/system-plugin-errors.txt
caught exception in plugin method "systemd.setup()"
caught exception in plugin method "systemd.setup()"
writing traceback to sos_logs/systemd-plugin-errors.txt
writing traceback to sos_logs/systemd-plugin-errors.txt
caught exception in plugin method "sysvipc.setup()"
caught exception in plugin method "sysvipc.setup()"
writing traceback to sos_logs/sysvipc-plugin-errors.txt
writing traceback to sos_logs/sysvipc-plugin-errors.txt
caught exception in plugin method "udev.setup()"
caught exception in plugin method "udev.setup()"
writing traceback to sos_logs/udev-plugin-errors.txt
writing traceback to sos_logs/udev-plugin-errors.txt
caught exception in plugin method "xfs.setup()"
caught exception in plugin method "xfs.setup()"
writing traceback to sos_logs/xfs-plugin-errors.txt
writing traceback to sos_logs/xfs-plugin-errors.txt
caught exception in plugin method "yum.setup()"
caught exception in plugin method "yum.setup()"
writing traceback to sos_logs/yum-plugin-errors.txt
writing traceback to sos_logs/yum-plugin-errors.txt
Running plugins. Please wait ...
Starting 1/46 block [Running: block]
Starting 2/46 cgroups [Running: block cgroups]
Starting 3/46 crypto [Running: block crypto]
Starting 4/46 dbus [Running: block dbus]
Starting 5/46 devicemapper [Running: block devicemapper]
Starting 6/46 devices [Running: block devicemapper devices]
Starting 7/46 dnf [Running: block devicemapper devices dnf]
Starting 8/46 ebpf [Running: block devicemapper devices ebpf]
Starting 9/46 filesys [Running: block devicemapper ebpf filesys]
Starting 10/46 hardware [Running: block devicemapper ebpf hardware]
Starting 11/46 host [Running: block devicemapper ebpf host]
Starting 12/46 i18n [Running: block devicemapper host i18n]
Starting 13/46 kernel [Running: block devicemapper host kernel]
Starting 14/46 krb5 [Running: devicemapper host kernel krb5]
Starting 15/46 kvm [Running: devicemapper host kernel kvm]
Starting 16/46 ldap [Running: devicemapper host kernel ldap]
Starting 17/46 libraries [Running: devicemapper host kernel libraries]
Starting 18/46 libvirt [Running: devicemapper host kernel libvirt]
Starting 19/46 login [Running: devicemapper host kernel login]
Starting 20/46 logrotate [Running: host kernel login logrotate]
Starting 21/46 logs [Running: kernel login logs]
Starting 23/46 md [Running: kernel login md]
Starting 22/46 lvm2 [Running: kernel login md lvm2]
Starting 24/46 memory [Running: kernel login lvm2 memory]
Starting 25/46 multipath [Running: kernel login lvm2 multipath]
Starting 26/46 networking [Running: kernel login lvm2 networking]
Starting 27/46 openhpi [Running: kernel login lvm2 openhpi]
Starting 28/46 pam [Running: kernel login lvm2 pam]
Starting 29/46 pci [Running: login lvm2 pci]
Starting 31/46 processor [Running: login lvm2 processor]
Starting 30/46 process [Running: login lvm2 process]
Starting 32/46 python [Running: login lvm2 python]
Starting 33/46 release [Running: login lvm2 python release]
Starting 35/46 scsi [Running: login python scsi]
Starting 34/46 rpm [Running: python rpm]
Starting 36/46 selinux [Running: python selinux]
Starting 37/46 services [Running: python services]
Starting 38/46 ssh [Running: python ssh]
Starting 39/46 subscription_manager [Running: python subscription_manager]
Starting 40/46 system [Running: python system]
Starting 41/46 systemd [Running: python systemd]
Starting 42/46 sysvipc [Running: python systemd sysvipc]
Starting 43/46 udev [Running: python systemd udev]
Starting 44/46 vhostmd [Running: python systemd vhostmd]
Starting 45/46 xfs [Running: python systemd vhostmd xfs]
Finishing plugins [Running: python systemd vhostmd]
Starting 46/46 yum [Running: python systemd vhostmd yum]
Finishing plugins [Running: python systemd vhostmd]
Finishing plugins [Running: python systemd]
Finishing plugins [Running: systemd]
Finished running plugins
caught exception in plugin method "filesys.postproc()"
caught exception in plugin method "filesys.postproc()"
writing traceback to sos_logs/filesys-plugin-errors.txt
writing traceback to sos_logs/filesys-plugin-errors.txt
caught exception in plugin method "ldap.postproc()"
caught exception in plugin method "ldap.postproc()"
writing traceback to sos_logs/ldap-plugin-errors.txt
writing traceback to sos_logs/ldap-plugin-errors.txt
caught exception in plugin method "openhpi.postproc()"
caught exception in plugin method "openhpi.postproc()"
writing traceback to sos_logs/openhpi-plugin-errors.txt
writing traceback to sos_logs/openhpi-plugin-errors.txt
Creating compressed archive...
Your sosreport has been generated and saved in:
/var/tmp/sosreport-f8927b4f7dc0-2020-09-16-jpnkjok.tar.xz
Size 17.70KiB
Owner root
md5 3bdc630cf68cd344846c32a40fb30d90
Please send this file to your support representative.
[root@f8927b4f7dc0 ~]#
Thanks Derrick Ornelas for checking if the fix works. I am marking the bug as VERIFIED based on comment 10 and comment 17. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (sos bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2020:4534 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |
Description of problem: Running sosreport in a container without expliciltly setting ENV_HOST_SYSROOT to '/' will result in a traceback: $ podman run -it --rm --name sosreport registry.redhat.io/rhel8/support-tools bash [root@9e2c299cf1c4 /]# sosreport Traceback (most recent call last): File "/usr/sbin/sosreport", line 19, in <module> main(sys.argv[1:]) File "/usr/lib/python3.6/site-packages/sos/sosreport.py", line 1383, in main sos = SoSReport(args) File "/usr/lib/python3.6/site-packages/sos/sosreport.py", line 287, in __init__ self.policy = sos.policies.load(sysroot=self.opts.sysroot) File "/usr/lib/python3.6/site-packages/sos/policies/__init__.py", line 44, in load cache['policy'] = policy(sysroot=sysroot) File "/usr/lib/python3.6/site-packages/sos/policies/redhat.py", line 259, in __init__ super(RHELPolicy, self).__init__(sysroot=sysroot) File "/usr/lib/python3.6/site-packages/sos/policies/redhat.py", line 50, in __init__ sysroot = self._container_init() File "/usr/lib/python3.6/site-packages/sos/policies/redhat.py", line 145, in _container_init host_tmp_dir = os.path.abspath(self._host_sysroot + self._tmp_dir) TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' It appears we explicitly check for root-path sysroot only, and if it is the default None or any other value it will fail, when I think we explicitly want this to be set when there is or isn't a ENV_HOST_SYSROOT: class RedHatPolicy(LinuxPolicy): - - - - 8< - - - - def _container_init(self): """Check if sos is running in a container and perform container specific initialisation based on ENV_HOST_SYSROOT. """ if ENV_CONTAINER in os.environ: if os.environ[ENV_CONTAINER] in ['docker', 'oci']: self._in_container = True if ENV_HOST_SYSROOT in os.environ: self._host_sysroot = os.environ[ENV_HOST_SYSROOT] use_sysroot = self._in_container and self._host_sysroot != '/' if use_sysroot: host_tmp_dir = os.path.abspath(self._host_sysroot + self._tmp_dir) We should do something like this: use_sysroot = self._in_container and self._host_sysroot is not None Welcome to thoughts and changes; if it's not intended for us to run in a container this way, we should catch this and throw an error instead of the traceback. Version-Release number of selected component (if applicable): sos-3.7-8.el8_1.noarch All versions currently affected. How reproducible: Every time Steps to Reproduce: Provided above. Actual results: Tracebacks instead of gathers sosreport Expected results: Should either fail gracefully or otherwise collect sosreport. Additional info: PR resolving this behavior is here: https://github.com/sosreport/sos/pull/2028