Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1881118

Summary:	Collect contents of /etc/crio/crio.conf.d/ in crio plugin
Product:	Red Hat Enterprise Linux 8	Reporter:	Derrick Ornelas <dornelas>
Component:	sos	Assignee:	Pavel Moravec <pmoravec>
Status:	CLOSED ERRATA	QA Contact:	Miroslav Hradílek <mhradile>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	8.2	CC:	agk, bmr, dornelas, jhunsaker, mhradile, plambri, sbradley
Target Milestone:	rc	Keywords:	OtherQA, Triaged
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	sos-4.0-4.el8	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-05-18 14:48:58 UTC	Type:	Enhancement
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1186913

Description Derrick Ornelas 2020-09-21 14:52:59 UTC

CRI-O now supports/uses /etc/crio/crio.conf.d/ as a drop-in config directory.  User modifications/overrides are stored in this directory, and /etc/crio/crio.conf is not modified.  Please collect the contents of this directory when collecting data on RHEL CoreOS.

Comment 3 Pavel Moravec 2020-09-21 15:37:16 UTC

Upstream PR created.

By default, it will land in RHEL8.5 and 9.0. If you need a faster backport, please mention that (and ideally share the will to verify the fix by OtherQE, to ensure qa_ack+).

Comment 4 Derrick Ornelas 2020-09-21 20:19:31 UTC

There will likely be about 4 OpenShift releases between now and RHEL 8.5  :-)  Could we please look at getting this into 8.4?  I can do OtherQE

Comment 5 Pavel Moravec 2020-09-22 06:36:00 UTC

Re-scheduled, thanks for confirmation with OtherQE.

Comment 9 Pavel Moravec 2020-10-29 11:55:44 UTC

Hello,
can you please verify the bug against below build? Thanks in advance.


A yum repository for the build of sos-4.0-2.el8 (task 32548242) is available at:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/sos-4.0-2.el8.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-4.0-2.el8.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-4.0-2.el8.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-audit-4.0-2.el8.noarch.rpm

The repository will be available for the next 60 days. Scratch build output will be deleted
earlier, based on the Brew scratch build retention policy.

Comment 10 Derrick Ornelas 2020-12-09 18:02:41 UTC

Tested with sos-4.0-2.el8.noarch.rpm on OCP 4.6 (RHCOS 46.82.202010091720-0) and I'm not seeing /etc/crio/crio.conf.d/ being collected


# ls -l /etc/crio/crio.conf /etc/crio/crio.conf.d/
-rw-r--r--. 1 root root 15229 Nov 17 22:50 /etc/crio/crio.conf

/etc/crio/crio.conf.d/:
total 4
-rw-r--r--. 1 root root 868 Nov 17 22:47 00-default


# podman pull --authfile=/var/lib/kubelet/config.json registry.redhat.io/rhel8/support-tools
Trying to pull registry.redhat.io/rhel8/support-tools...
Getting image source signatures
Copying blob 6500ac87b29f done  
Copying blob 34ea9dcbf96b done  
Copying blob 1b8dabac56ed done  
Copying config 89e8127ded done  
Writing manifest to image destination
Storing signatures
89e8127deda47f6a0157c0794916e371eea4b4980d01c4ba17e929f95226976f


# toolbox 
Spawning a container 'toolbox-root' with image 'registry.redhat.io/rhel8/support-tools'
Detected RUN label in the container image. Using that as the default...
command: podman run -it --name toolbox-root --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=toolbox-root -e IMAGE=registry.redhat.io/rhel8/support-tools:latest -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-tools:latest

[root@dell-r430-08 /]# rpm -Uvh http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-4.0-2.el8.noarch.rpm
Retrieving http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-4.0-2.el8.noarch.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:sos-4.0-2.el8                    ################################# [ 33%]
Cleaning up / removing...
   2:sos-collector-1.8-2.el8          ################################# [ 67%]
   3:sos-3.9.1-6.el8                  ################################# [100%]


[root@dell-r430-08 /]# sosreport -version
Please note the 'sosreport' command has been deprecated in favor of the new 'sos' command, E.G. 'sos report'.
Redirecting to 'sos report -version'
WARNING: Unable to read configuration file /etc/sos/sos.conf : No such file or directory

sosreport (version 4.0)

a non-existing plugin (rsion) was specified in the command line


[root@dell-r430-08 /]# sos report
WARNING: Unable to read configuration file /etc/sos/sos.conf : No such file or directory

sosreport (version 4.0)

This command will collect diagnostic and configuration information from
this Red Hat CoreOS system.

An archive containing the collected information will be generated in
/host/var/tmp/sos.7oxeojfj and may be provided to a Red Hat support
representative.

Any information provided to Red Hat will be treated in accordance with
the published support policies at:

  https://access.redhat.com/support/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.

Press ENTER to continue, or CTRL-C to quit.

Please enter the case id that you are generating this report for []: 

 Setting up archive ...
 Setting up plugins ...
[plugin:networking] skipped command 'ip6tables -t nat -nvL': required kmods missing: nf_tables, ip6table_nat. 
[plugin:networking] skipped command 'ip6tables -t mangle -nvL': required kmods missing: ip6table_mangle, nf_tables. 
[plugin:networking] skipped command 'ip6tables -t filter -nvL': required kmods missing: ip6table_filter, nf_tables. 
[plugin:networking] skipped command 'nft list ruleset': required kmods missing: nf_tables. 
[plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec.  Use '--allow-system-changes' to enable collection.
[plugin:networking] skipped command 'ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag.  Use '--allow-system-changes' to enable collection.
[plugin:networking] skipped command 'iptables -vnxL': required kmods missing: nf_tables, iptable_filter. 
[plugin:networking] skipped command 'ip6tables -vnxL': required kmods missing: ip6table_filter, nf_tables. 
[plugin:networking] skipped command 'ip netns exec 2f99aa26-8a92-4532-81f1-72fcf55e65ce ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec badd1cdc-0449-4593-afc8-ba7c27694e0a ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 2217df1a-dbd7-4204-ba07-8a5bb4ded7eb ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 17804370-c541-4bc5-ba77-bc842a242554 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 39b7b597-034d-4c4a-b74c-d568d11621ce ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 3a4c181f-74d9-436f-9bb2-c03cf8670b9e ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 427a59ae-c75d-4b70-a92d-b0b140fa85f3 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 8c8113e1-c720-4c2e-803d-ae824abd5a98 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec bfc770f3-c2fd-4c7b-a9f6-43a17c37a994 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 1777a414-fd9e-4c78-b903-a60f1e21a8cf ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec ad102751-9a2e-4e45-88cf-0e38451b3ddc ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 959a8cfb-03b5-49e3-93e5-02a4913f3162 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 02b521ec-261a-4331-b09d-0eae57560b95 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 419e6c31-4526-43ab-b2d9-8d5c3ca332f2 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec a75b4299-62b9-42bd-a8d7-5ff7d7b1dc88 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 8e462e60-cbb3-447a-aca3-e9f2a413078b ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec bcc18096-0552-4d77-8353-9ef523ebce7c ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 40fd7b96-b7aa-407d-a0c8-82b31779f8b3 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec a9e97f67-ee96-4ffc-9d3d-254b79bf446e ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 7a868a08-d9d2-4086-be8a-74f433e4b9ec ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 92916b96-d544-4a7f-8c56-622325a2eb3e ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec c534cbcf-a1d0-4dc9-ad73-d5b2e66917a7 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 94462fe3-abae-4a8f-8e74-608bda4224b3 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec adc5305d-7b30-473e-8ce7-72fe68ad0759 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 59f8d034-82ce-452b-a885-92d2c741edbc ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 38e63eeb-4028-4017-b6bc-f94eda5692e8 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec ed6ea8a5-b71b-4f47-940f-970c81220ce3 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 7175b13e-065d-482b-a5e2-6e84f45f97ac ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec f0c360f0-5763-491b-bd1c-8408c8f4c061 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 5ccb0faa-b39c-4b6d-b9ed-967835c9467d ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 4a856e51-8585-40ca-b116-dcefef2d71cc ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 3c1c62a1-eb37-4f32-8f1c-a6babb59a2f4 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 78351d22-4741-4cd8-9196-d77946884259 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec dbc9987e-0c9c-407e-92f8-d14709230f38 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 5d06ab65-a12f-46b8-afc8-9b1f08bb4687 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec a632425c-2cfd-4c1f-9e37-cc64c68f74b7 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec 20a675de-3e30-4f45-ba94-b653a70a19d1 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:networking] skipped command 'ip netns exec be068b46-df6a-44cf-996b-28037f097400 ss -peaonmi': required kmods missing: udp_diag, tcp_diag, af_packet_diag, unix_diag, netlink_diag, inet_diag. 
[plugin:openshift] Not logged in to OCP API, and no login token provided. Will not collect `oc` commands
[plugin:sssd] skipped command 'sssctl config-check': required services missing: sssd. 
caught exception in plugin method "system.setup()"
writing traceback to sos_logs/system-plugin-errors.txt
[plugin:systemd] skipped command 'systemd-resolve --status': required services missing: systemd-resolved. 
[plugin:systemd] skipped command 'systemd-resolve --statistics': required services missing: systemd-resolved. 
 Running plugins. Please wait ...

  Starting 8/87  cifs            [Running: block cgroups chrony cifs]                     caught exception in plugin method "cgroups.collect()"
writing traceback to sos_logs/cgroups-plugin-errors.txt
  Starting 78/87 systemd         [Running: cgroups logs networking selinux systemd]       caught exception in plugin method "systemd.collect()"
writing traceback to sos_logs/systemd-plugin-errors.txt
  Finishing plugins              [Running: cgroups systemd]                               ged]
Creating compressed archive...

Your sosreport has been generated and saved in:
	/host/var/tmp/sosreport-dell-r430-08-2020-12-09-vvzpemg.tar.xz

 Size	456.91MiB
 Owner	root
 md5	168ff81217378bb6c10fa75e9de891b5

Please send this file to your support representative.


[root@dell-r430-08 /]# exit
exit


# cd /var/tmp && tar xf sosreport-dell-r430-08-2020-12-09-vvzpemg.tar.xz


# ls -lR sosreport-dell-r430-08-2020-12-09-vvzpemg/etc/crio/
sosreport-dell-r430-08-2020-12-09-vvzpemg/etc/crio/:
total 28
-rw-r--r--. 1 root root 15229 Nov 17 22:50 crio.conf
-rw-r--r--. 1 root root 11937 Nov 17 22:50 seccomp.json



The sos.txt log says the directory was collected:

sosreport-dell-r430-08-2020-12-09-vvzpemg/sos_reports/sos.txt:
---
crio
-  commands executed:
[...]
-  files copied:
  * /host/etc/sysconfig/crio-metrics
  * /host/etc/sysconfig/crio-network
  * /host/etc/containers/registries.conf
  * /host/etc/crictl.yaml
  * /host/etc/crio/crio.conf.d/00-default
  * /host/etc/crio/crio.conf
  * /host/etc/containers/registries.d/default.yaml
  * /host/etc/sysconfig/crio-storage
  * /host/etc/containers/policy.json
  * /host/etc/containers/storage.conf
  * /host/etc/crio/seccomp.json
---

Comment 11 Derrick Ornelas 2020-12-09 18:08:21 UTC

Testing again

[root@dell-r430-08 /]# sos report --batch -v -o crio 
WARNING: Unable to read configuration file /etc/sos/sos.conf : No such file or directory

sosreport (version 4.0)

This command will collect diagnostic and configuration information from
this Red Hat CoreOS system.

An archive containing the collected information will be generated in
/host/var/tmp/sos.t3dxp7xm and may be provided to a Red Hat support
representative.

Any information provided to Red Hat will be treated in accordance with
the published support policies at:

  https://access.redhat.com/support/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.


 Setting up archive ...
[archive:TarFileArchive] initialised empty FileCacheArchive at '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs'
[sos.report:setup] executing 'sos report --batch -v -o crio'
[sos.report:setup] using 'ocp' preset defaults (--all-logs --verify)
[sos.report:setup] effective options now: --all-logs --batch --only-plugins crio -v --verify
 Setting up plugins ...
[plugin:crio] added cmd output 'journalctl --no-pager  --unit crio'
[plugin:crio] added cmd output 'ls -alhR /etc/cni'
[plugin:crio] added cmd output 'crictl info'
[plugin:crio] added cmd output 'crictl images'
[plugin:crio] added cmd output 'crictl pods'
[plugin:crio] added cmd output 'crictl ps'
[plugin:crio] added cmd output 'crictl ps -a'
[plugin:crio] added cmd output 'crictl ps -v'
[plugin:crio] added cmd output 'crictl stats'
[plugin:crio] added cmd output 'crictl version'
[plugin:crio] added cmd output 'crictl inspect 69170aa14104d14d9bd89d3f596c41680cef47e6d3462015612d60606592a79d'
[plugin:crio] added cmd output 'crictl inspect 59a9e67adc2be3b152539e0cc08174cd0cbb8a50d209571084690feb98be6565'
[plugin:crio] added cmd output 'crictl inspect fde30ac88d3e7dff36849bae87f3cf6ec37f51470a3f46611373ec7197be58c9'
[plugin:crio] added cmd output 'crictl inspect 44a2df4b9fd37371002094ee9c2bd0b1c5f9ea6c975b97fd189e62cff7a6d1ad'
[plugin:crio] added cmd output 'crictl inspect 7a959038caae355994737487ac16011dab68eb602f71e5298efb018db0aa9a45'
[plugin:crio] added cmd output 'crictl inspect a74688f424244aa868c0d2001e2c0196a8e155672019f74160c5ad8d99544be1'
[plugin:crio] added cmd output 'crictl inspect fc4e9ac5b121df8489564e6e94e6982932403bde6cd88b0d60f778c8c8dfeeb0'
[plugin:crio] added cmd output 'crictl inspect 56d7f1bd079e9992e15fe5f3fdab286882c2d43964e9f08d4904884a4ce00a33'
[plugin:crio] added cmd output 'crictl inspect c653b04c8616af02a25de0f95adc3f4cc4a2e7b101481c30509dfbd511aa252b'
[plugin:crio] added cmd output 'crictl inspect b29c79d24a365af0c9fd713bf510180031d8edc722d5352ba97dcfd7ef0a65fd'
[plugin:crio] added cmd output 'crictl inspect 2f67ddb30860b23392a91bddef2e6d7baecc620adab179aadcf48f9a74904494'
[plugin:crio] added cmd output 'crictl inspect 6b44fff419cc4c982f640060815503c3683d315506718a8e539c013fdf538121'
[plugin:crio] added cmd output 'crictl inspect 981f46523451ef0571759729282dd48692be2496a47e5dd36cc76ef4900cb53a'
[plugin:crio] added cmd output 'crictl inspect c6801d2cb1d7a8ed7ec0f9dc4d8a6f7be920345fd8aebc857ab88825fc4e9d81'
[plugin:crio] added cmd output 'crictl inspect 4bb41f72e2d07c11da77b52cb4c42f28bb2eee493a723c92f87e7df5b2d0f0b9'
[plugin:crio] added cmd output 'crictl inspect 65cabedb077a9a52c3a987f3b9c5fa6ce78d0582198ccad82391a59736d54cc7'
[plugin:crio] added cmd output 'crictl inspect bd2a9bff124b26c955452cb3ed0f84c448b17c17190ee851fdee3e130f0400c3'
[plugin:crio] added cmd output 'crictl inspect 1d2f5742bb02536aedc7e9b1d58f4a638476062fecaba31ea107c88234ed6f8c'
[plugin:crio] added cmd output 'crictl inspect 99ef38f6ef8f8c2eac3d1fb2f8c5c1940fb9e7cd431012e2e6b093aef017e633'
[plugin:crio] added cmd output 'crictl inspect d1f451397b4815902a1e02883b53bcd77382d6faf554ca25d7b2d4d459dcccbd'
[plugin:crio] added cmd output 'crictl inspect f853b2431386cbb39e0e722653b5cd66e5e9302bcc3ddfabb041dfd145581e20'
[plugin:crio] added cmd output 'crictl inspect 8ff1f2943ca09ef0d845ebd9392e6b1fc035af047ad5874dd17317c4c82bf330'
[plugin:crio] added cmd output 'crictl inspect 6a70a0edb7846fc93c26a31ea52940ad577ea3da4a820282ea4522f55f1fcf53'
[plugin:crio] added cmd output 'crictl inspect 48f082b7e5a78c10a6b38629af011aeb0f32f7882724f5ffcd49954c8d9a766a'
[plugin:crio] added cmd output 'crictl inspect abdebdd02e1af43e5d03b672375ff77d7a0860fe0ad96d684b867e8e842f391a'
[plugin:crio] added cmd output 'crictl inspect 03a7f195d0b052b53002bd413717583729583a11c0c57a3e667f27cadb4ad037'
[plugin:crio] added cmd output 'crictl inspect ef8e973c311181c51084f37d5142a0d492c7862946d5079b1b7604489288b752'
[plugin:crio] added cmd output 'crictl inspect cdf979813984ec281c784867addc74a430976419a0642e84c3ad9f2e0282684a'
[plugin:crio] added cmd output 'crictl inspect f8c21c497ec4c9bf479460904bab85c630c11d7879eb8a65c0efdf445cd2ac15'
[plugin:crio] added cmd output 'crictl inspect b83290c647cf3ef623297120c107352f586201be4670d8ae686514eb10209afc'
[plugin:crio] added cmd output 'crictl inspect ec805db82f25957b0bec5ab63e5e58e9a035095a863a73771a599c80bc47dcf3'
[plugin:crio] added cmd output 'crictl inspect 5a3b4bd0e23ffaaa9c5e09807cfff556e3d742652f8dc280b40a6924f305b83b'
[plugin:crio] added cmd output 'crictl inspect 7c8aedbfdab90472241ab4bf9c2ed61bc29ebebbe36f9e4346e458b7f351b1e4'
[plugin:crio] added cmd output 'crictl inspect bfa165d7f8d0de89269559bb73d00fef7bc2ef96436b92e524ea32dd6f9e9b19'
[plugin:crio] added cmd output 'crictl inspect 09279cf1e70ce38c2efb644aa117c8e1580cf659175f2eebd433eec33d39d88d'
[plugin:crio] added cmd output 'crictl inspect d3b2600d44d2f5ef2712f27ef96847fbd220ab0ec2b8f8553f64344a1ff5f6ca'
[plugin:crio] added cmd output 'crictl inspect 605251eafb1e78fb9e46b95f2690c07216ca8827491cb10d54a8d0b94ff7d184'
[plugin:crio] added cmd output 'crictl inspect 5e0e163de40f7fa1de8eb716d60d3b41270e6fdca34a022679a1ebb85def2285'
[plugin:crio] added cmd output 'crictl inspect 5f753f860af7c5ee4dcd8e44c3b6f61f3e155090d5d8a0656b891cb3b310b160'
[plugin:crio] added cmd output 'crictl inspect 2866f151feb763efec85336b59bd7f9d2c8f395e6ad271b631f05699b555f50c'
[plugin:crio] added cmd output 'crictl inspect b056b80802703a2c8b7f7b27bd987e914119419678f10bd2f4b0c765afb9e709'
[plugin:crio] added cmd output 'crictl inspect f3a5476cb72e80bbd65b780ce580fdc68d51625c027d53ee695b0170db429058'
[plugin:crio] added cmd output 'crictl inspect b92804d7ad516351b1e8e0bd27e2577fbfb628b8fe7f7bfc61139bd60cd45956'
[plugin:crio] added cmd output 'crictl inspect cbd39e7a9566577f8ec97f08b063060bcf7e6f81e8dc7dd093b4ade4657a8d1d'
[plugin:crio] added cmd output 'crictl inspect 9afb24750d672bb431b5ff8871c753eceae97e6c49db4637d4d1a685f428d828'
[plugin:crio] added cmd output 'crictl inspect da567fda23c0b7caef669f49ccfaef7f84bfeaf27c4de672a65efcb02adf5c62'
[plugin:crio] added cmd output 'crictl inspect c11f150adf57e2acbbd627783276fe7a776622eada7a4ef764253204d4ee0feb'
[plugin:crio] added cmd output 'crictl inspect e3442d6cd977817fbb9ee2cc2f986c262d20839bd939d35306a5f5d31c410158'
[plugin:crio] added cmd output 'crictl inspect f5f1aab2fdc3828bd89105ff4110410cda80adf1cfa94e942c50104e9c769d77'
[plugin:crio] added cmd output 'crictl inspect 2bff47a20fcce2ba72e944f0bf8bcc4039370837239315296837e36451f5e7de'
[plugin:crio] added cmd output 'crictl inspect 333b58865c6b3c5f2d15601a73016fd4000f063b0f69c0877afe049066951fb3'
[plugin:crio] added cmd output 'crictl inspect 92cee5912790382b87f4d97e53df721dd6fa86f155d5f9ddce94f4a4455a1547'
[plugin:crio] added cmd output 'crictl inspect d2c443208e6f3823105029982fb8b2b413929c676780ca46aea3b59714a28d0f'
[plugin:crio] added cmd output 'crictl inspect ce4ada2146c187e6fe6e5fb74974938a24ddbc0894e93aded9e27d39fac84ea9'
[plugin:crio] added cmd output 'crictl inspect 8db1624537fbf52a32dfb35a8f1727abf06b442fc28a2f3c5b7d21a0775ef34b'
[plugin:crio] added cmd output 'crictl inspect 455a5c039afabc15e59e07400a535a1dee949da75bd7bf7d4a292af6df7844c8'
[plugin:crio] added cmd output 'crictl inspect 9a492e62cef1052f9ea8f094f709cd54066511307df46f9f5b6478c69466062d'
[plugin:crio] added cmd output 'crictl inspect fddbd301593c0be82f6be7b61d4ff07b19161b82b612bfff02949d3113ed4ce6'
[plugin:crio] added cmd output 'crictl inspect 8fb30497a3e8378d329fdc4d4d90fffd5db2d988cc932dedc1e91f8026b7883e'
[plugin:crio] added cmd output 'crictl inspect e72edeb8b56d06eddca369faeb4f6e4d5cc3f17f193d1b6029cc60bbadf9a014'
[plugin:crio] added cmd output 'crictl inspect 6e0b37a22b6ba915172b31a4e87d6e9616862b35a266320cce77c4bc9e347b94'
[plugin:crio] added cmd output 'crictl inspect 869a3d268a043d7cbe9c38dbde225b87b42d4834dc85645be241a0f66ba45ee7'
[plugin:crio] added cmd output 'crictl inspect 88ca2ae9822aeede26f667c4e2aedc4a99db9dcb2e11cd74ebbfdc11b5bad76f'
[plugin:crio] added cmd output 'crictl inspect ab78c567c0db7a7a9f3d3751cdc925c51e1bb5e137716b7e1910daef42ef685f'
[plugin:crio] added cmd output 'crictl inspect 60f934c6c2dd7a14196212e1e3f2df6926ad59a9ceab2d0ae9088342bf309217'
[plugin:crio] added cmd output 'crictl inspect edd220700913fb67a2dbbccbe7f97b21eb51f96e9aac5e220e1f4f094f40651a'
[plugin:crio] added cmd output 'crictl inspect 7e07f1d6834eaacbe4bcb436bef5f64f90fa316602e51bb3758244256bd02ccd'
[plugin:crio] added cmd output 'crictl inspect cdbd6d76d8c2aa2d2f597eeca30a2f0ab6e5120a5c0a619a6834f8213b195aca'
[plugin:crio] added cmd output 'crictl inspect 73ee0c1af4907bb495a7bfc99f14f407cacb7484705d138c34dc2c22868051fc'
[plugin:crio] added cmd output 'crictl inspect 726d0bd2f108fc71ebd095852ebcbbd41b00b7172ecf0631e3d7ccb633d170af'
[plugin:crio] added cmd output 'crictl inspect 1cbf89ce8eaec433bde30a00b9c38b8e1f069a65804344bd289af3e8609c612c'
[plugin:crio] added cmd output 'crictl inspect 0f107301e1dd59a0a67fb0d99be048a0f2ede9904d3ee52f0c2dea21b0685ce8'
[plugin:crio] added cmd output 'crictl inspect b33d5af31fa26e1ad13513f4440f3cc7d64656a0629df161a71e20396ddec8aa'
[plugin:crio] added cmd output 'crictl inspect b84a02b3f76697a7be30051b922f271666b8ff9a4559486260c4a356c0a77097'
[plugin:crio] added cmd output 'crictl inspect 9c7beffeb13525b426814d891a73189d59416cd838ded346f6a54f051fe86239'
[plugin:crio] added cmd output 'crictl inspecti 034f77b7b6206d6a19a4b45f6789aaf7881e0a13b723a3f924fddde99acfe1ae'
[plugin:crio] added cmd output 'crictl inspecti 09a01faefd4fcc313b7653d468878bf3427265d995cf7ca621e7e18992d47c51'
[plugin:crio] added cmd output 'crictl inspecti 3dbbd23e9657a5d059b04771dce535755cf9a3568785a0d309d29f14b3229944'
[plugin:crio] added cmd output 'crictl inspecti fbdfe329e9eb24a20023c7b84fccbffdd698a12ffd17e51ffa7728d15f2e2136'
[plugin:crio] added cmd output 'crictl inspecti c8328a0ded0e04dfe01d6424488cff688426587bb86b5c08079a5f7bc7f218cc'
[plugin:crio] added cmd output 'crictl inspecti fe3f46a31be413f0f5810ca8ea5fe54ab83ad8e7965980ea22af0c6c0371d3e8'
[plugin:crio] added cmd output 'crictl inspecti 85d381f7a4f998891f91af11b1658b16e34f52d3c05a8563d4eb8db27f79f286'
[plugin:crio] added cmd output 'crictl inspecti 7491f5f484164674f3d1610e58dab752dae275fabd5fe68fd8a13f9f52814217'
[plugin:crio] added cmd output 'crictl inspecti fdd1ce49495579b4c58c587bb180a69b854a971c7b2a9a39046d6960247f1123'
[plugin:crio] added cmd output 'crictl inspecti e654362b4a3c7ccac1da74df2d5ef8f50c1be1223ba8b31129ba56cfd010c444'
[plugin:crio] added cmd output 'crictl inspecti 10e525c5b8882b6a8606da0d1f98892d326c9b29d7c0b4b3fabbcb980ab57ee6'
[plugin:crio] added cmd output 'crictl inspecti cf43cf40cf48e8c192aa5f855b150fa580c8bb764788857d814c69b0598e0815'
[plugin:crio] added cmd output 'crictl inspecti 71d17f2c24c7409883769b1fe06812063da7c9615a5b1e5df54d6ef8d33fbbf3'
[plugin:crio] added cmd output 'crictl inspecti 1f9649341c2276f0bcde0c125059eafc68fe1449bafdf44ce0b7519013c0dbe9'
[plugin:crio] added cmd output 'crictl inspecti 3faaab80cd610654276b199994986568feb6376cdf72832a3595bc8b0b26497e'
[plugin:crio] added cmd output 'crictl inspecti a88630bfa771d811633766c29c2d953f8484a956b80460e659b345de57e8a114'
[plugin:crio] added cmd output 'crictl inspecti 1411a8630b920c81400e175a2b82bad9b6662ac049042f90c2e686e718c73d5b'
[plugin:crio] added cmd output 'crictl inspecti 7b8c808ab27a609df1a5983a070efd372d2365933aea93e03c01985971850e25'
[plugin:crio] added cmd output 'crictl inspecti 0d689230e34d1b56ac9149226728932862f9f4188d19c1337f5490aed4e78307'
[plugin:crio] added cmd output 'crictl inspecti 2abf30ff6ba99c2a03e2c6b4b9ccd6f389c1aa9f6bd0f020a21e5f7f0f5a493a'
[plugin:crio] added cmd output 'crictl inspecti 1c20b9ba40e4052f46ac37d1b41897db4f576ed56dcb68d6333c885214d3d5e1'
[plugin:crio] added cmd output 'crictl inspecti ee1ad20f0063bb88b3bffd9cfd0ceddeca5ef14d9fb7e1b033fac4d8e61bc140'
[plugin:crio] added cmd output 'crictl inspecti a30dfb279cfc7014c96a3c143eba4b62e4843ecda6d957f3f5f251c0b5f43b83'
[plugin:crio] added cmd output 'crictl inspecti 3264a20df581bff1f88671f55733f7f58bb4841702b6e6659858af640bc6855b'
[plugin:crio] added cmd output 'crictl inspecti c52903ef9ea83e0f043f4bcb1ef089411840f78000addb0ef1edda4862267145'
[plugin:crio] added cmd output 'crictl inspecti 806a0bf7684a4a9e1b20f3c422d4fb96513412e7d11632c1a482d29734b89ba7'
[plugin:crio] added cmd output 'crictl inspecti a43a61ece277a722fcdcc8e14449cdb95ee6710e32525c4d904ae851a68f0b77'
[plugin:crio] added cmd output 'crictl inspecti c374de0d959263981871ea0134ef048de3a20b212c1b9f4cf9eda78f9a603e53'
[plugin:crio] added cmd output 'crictl inspecti 7307cec9948467ed333fddb457eff62cc03fd4060fd57adcc378a91ed1d0dd0b'
[plugin:crio] added cmd output 'crictl inspecti 90bf71031466153a92fd9dc71e064f6aac63e9fce27a2636f036a09ca7ff26ea'
[plugin:crio] added cmd output 'crictl inspecti dbd125e25c8ade62f2938361146b23740fd7874c9e33714a29646b12cd3e017f'
[plugin:crio] added cmd output 'crictl inspecti 140253c0c1a51e61bcb113e1b837496e8958b7b90c027fec8f4b5ddb08e060fc'
[plugin:crio] added cmd output 'crictl inspecti b9f516edc1b156b0504465b6075adc9814f447debec401890318f8ac31054dfe'
[plugin:crio] added cmd output 'crictl inspecti 95534a9fb872628fbeda0f52f8d06462042ca8dd838b0ebefecc9c56b92218eb'
[plugin:crio] added cmd output 'crictl inspecti e2d8c35ce60c1e429e8f6160507bbde2f7a17635e138192f9cf731eda2b849a4'
[plugin:crio] added cmd output 'crictl inspecti 154226eaa544c7cd8fc91996d77fac472b1f149378399529553c62d7630b2158'
[plugin:crio] added cmd output 'crictl inspecti 24ff3011f60ea1ade69afd61b3a8e3eb2ee6b1fe2c342aaf0f9076a3bf710ca3'
[plugin:crio] added cmd output 'crictl inspecti 182e2d06a92b449a358f734fbd0aa94e9968f5e9dcbacea334a3ff64d2da54b3'
[plugin:crio] added cmd output 'crictl inspecti 21162c320f9459d85ce6bd7070433ffa31d62358ed40afe074b790ccddcab782'
[plugin:crio] added cmd output 'crictl inspecti 8f13cacfce95f5b2a0c2b50df05869e3236b07dfc9a0c5102b41b04d07b8fea0'
[plugin:crio] added cmd output 'crictl inspecti bdb981388da5ffedc0e8b0069e0309b4d3978550ef9ee46609bb4f58880e527e'
[plugin:crio] added cmd output 'crictl inspecti 4675029e660ab72a42c14e8032de66f14fafb663b7b42560b79e837c56b9389f'
[plugin:crio] added cmd output 'crictl inspecti bdf8e0b08db5ed5f436f9bed5fb6f4981326e428cfb6769e450360db64a29e75'
[plugin:crio] added cmd output 'crictl inspecti e666790521c3d46bc09d46da58d71c8fbe1187a47dcb7c10fffa64a9124b8c7b'
[plugin:crio] added cmd output 'crictl inspecti 6202cfe10d67716d76162dd34a924c4f43335036cbb1d2d9a13f2fe894cb0da2'
[plugin:crio] added cmd output 'crictl inspecti 2129063b76c3703ca046cf31a4b59b231dae9253971ad65e2517f8c78c16f86a'
[plugin:crio] added cmd output 'crictl inspecti f0e053c66e8bc4afc5f6b73a71447cc54cd399191a8aaaf62dc2cbc6900a4c21'
[plugin:crio] added cmd output 'crictl inspecti ffb8ac0907237aac782d9c1e796f717f8ce3824f76bbdd8277c43b24ab9ec907'
[plugin:crio] added cmd output 'crictl inspecti 89e8127deda47f6a0157c0794916e371eea4b4980d01c4ba17e929f95226976f'
[plugin:crio] added cmd output 'crictl inspectp b08884374aa3cc9e11e697919688eefebfabea4dce9cb9e61876b052d65ec942'
[plugin:crio] added cmd output 'crictl inspectp c01ed02be1a3cd9f8d87c18f155912ea4857622e7c99ce4f6798f799b177dc9d'
[plugin:crio] added cmd output 'crictl inspectp e429a27ca6cd2c3ef6d1dc79df4e4f1cf1dc188c6450ded8ce0209ebff668b66'
[plugin:crio] added cmd output 'crictl inspectp b25c44221a233cb233ddcd25a6eda445fce540a85b18e752868acaf67bcb50c6'
[plugin:crio] added cmd output 'crictl inspectp 25ff819c050b6fb0d0cd830d0cf41f3a7a4113bfeb93d16c38ac2ab775362df3'
[plugin:crio] added cmd output 'crictl inspectp fb3c4ec8ddc91e9607aae99e11566addb4d988660561525281ce03db5b4d69ff'
[plugin:crio] added cmd output 'crictl inspectp 703f481bd38d187730b049cd5327d1fcd11bbc4bc73f1fe8984dc4830dd95e7d'
[plugin:crio] added cmd output 'crictl inspectp 0ebe623d23a413d1aff6eafd8aa093580389c6be80c21dbd9eb50238045a2b32'
[plugin:crio] added cmd output 'crictl inspectp ca280a7fba950ee532527bb427e079fd91a30c85bc92bf629b71f9f593e091e5'
[plugin:crio] added cmd output 'crictl inspectp b007b18daf6293b4d8b5d185d5ec72442bf1b0125bd80b52914bf0b80caecb9f'
[plugin:crio] added cmd output 'crictl inspectp e1be83b1b9b62b46964ddadb4d5b7e3406a1ea550bae9278ab500ed6d9d06c39'
[plugin:crio] added cmd output 'crictl inspectp eb8a798ff18e9e3586abd8941555e8153f02bb46fc3178a1ff499152adb672cb'
[plugin:crio] added cmd output 'crictl inspectp 878357fa4458cfc9ea6f13342d00639315171bcd4fe6eb814829601c632d64c2'
[plugin:crio] added cmd output 'crictl inspectp 877f831a6eb9ebda8722e841360c90611ea3827ee1820e3edc94976ee601514e'
[plugin:crio] added cmd output 'crictl inspectp 95acbc095f712d127d36dfa5b4458af29c194ad103b44130753f256f76c9b4bb'
[plugin:crio] added cmd output 'crictl inspectp 3f7cc61df55d554f260aee849bab1cb15324d99a300663b8ea2bc5e85074f100'
[plugin:crio] added cmd output 'crictl inspectp c9f5c132cb0ee9f416d00684ed30e58d8a386c93b7a533e4ca4271b1ad31936e'
[plugin:crio] added cmd output 'crictl inspectp 87eb87538debf0642a7a7f362aa9608ee54b716de62611e71eb58d99412a2c0c'
[plugin:crio] added cmd output 'crictl inspectp aa7b71c520c46212d1d5e6ac8e51bea69abea5f88545e61c430f95c351d9b675'
[plugin:crio] added cmd output 'crictl inspectp f98490a20b6cba2c331b76182b91d94cdfc8be9797602a80ad065882aee48a5b'
[plugin:crio] added cmd output 'crictl inspectp ebd8e69aa0f6576adf4acf9eb0627ebbb49bc0427457df74e286f033222954a8'
[plugin:crio] added cmd output 'crictl inspectp 51acae7314c39947aab6c63efcc16b04809dc1f503e3749c99ec718e5232a539'
[plugin:crio] added cmd output 'crictl inspectp 650ae38f26a8efd2e91769eec1b79b5b01a3f62ca70d82dfed41d6fb7ca459bb'
[plugin:crio] added cmd output 'crictl inspectp fe4bd0644eff21f6fa8ca7925d426e0c97c02debd9f18b33b23ac6b05357a03b'
[plugin:crio] added cmd output 'crictl inspectp 5f6e7bd39d2896521f0eb0b4d27a93bd352210ad8fe813ba65a7299669132f7b'
[plugin:crio] added cmd output 'crictl inspectp 2789070a6c91f35fca6c6024a5ea4b0742dbda084b15f86e105032468296e00b'
[plugin:crio] added cmd output 'crictl inspectp 4f456db3b8a9705c1c4a859af8ba8046a403027b87c291eac161199723270b6a'
[plugin:crio] added cmd output 'crictl inspectp 56ec9d460947cbdee8b7b1b3ae0c7e891bdf5ae0e3b067eb26d3c4345868b79c'
[plugin:crio] added cmd output 'crictl inspectp 0b5ce1bd4c72933e06cf50ed14dd4d243828d94208b645234d3a4502d0fc9f83'
[plugin:crio] added cmd output 'crictl inspectp 97e49d48a2383b787c06ae8a701ed67883e8e760dc932c03f0b4f7e7c32f4537'
[plugin:crio] added cmd output 'crictl inspectp a398bbe05cc39a9f795d331c9556c287a8b91b3eabdd4d9d7ca103f0b5873fcb'
[plugin:crio] added cmd output 'crictl inspectp 06ce271f732d5db79a841777ab7204ead8ff27b63d808f3a4b16ae46c05fb232'
[plugin:crio] added cmd output 'crictl inspectp a45b9b7166ac32056975b8fb384e8b77aabd2175ee41edb10f97c0ba7d93903e'
[plugin:crio] added cmd output 'crictl inspectp d6a70055acf3718eb4623a333dfb5231f5b9411adeb3439d3285022097375e75'
[plugin:crio] added cmd output 'crictl inspectp a0ff09e1d825352c327fd7278c96b5b80c619404d7f31b838e28f420648ae0fe'
[plugin:crio] added cmd output 'crictl inspectp d752e107bb2d04d8a666c5ff8980658e6dfd4d29c55d644d2f567e5f3af7e31b'
[plugin:crio] added cmd output 'crictl inspectp 7d277f768f783bfb2a51248831252573c8ced5d8736ad6105d427e7d8b7cd599'
[plugin:crio] added cmd output 'crictl inspectp 02479e5b22b15a832b2c62d7d37790b3727bc30746f72793a31b239f09304122'
[plugin:crio] added cmd output 'crictl inspectp c1fdf28d00c1684847101d3c52525d78d51b6ce4f44ce76729d4eabbd6e3f9cd'
[plugin:crio] added cmd output 'crictl inspectp 946c0a5a45cf8eb481e623d30773852fe36d32a20e0ae9c762d974da3ed9f64c'
[plugin:crio] added cmd output 'crictl inspectp 39d5554bd633166e3fd0dca627ce887a225f4c43b73fd423148b2e6a03880529'
[plugin:crio] added cmd output 'crictl inspectp daf57ff6f914a8111b843093f3afa59b084e1d79ceaf2ed3b5f80222996b2853'
[plugin:crio] added cmd output 'crictl inspectp 0ce17bc1dd4dcdba7521c9d4f3da3e565e9698057b807a4110ef24485e906419'
[plugin:crio] added cmd output 'crictl inspectp e8f6f151d6085e02b37237c99d8c7c460ddcb2eadc52c02df69da94f5fd5e83a'
[plugin:crio] added cmd output 'crictl inspectp f03020fb7ee043d529be5382b5722c53327c74c8b72cfd7e76c2ec45f91b4b4e'
[plugin:crio] added cmd output 'crictl inspectp 3e7b37019427bb1407f0cc91e067d24c4f7aa28ec1b170384337eb6e8cc36dc2'
[plugin:crio] added cmd output 'crictl inspectp b2d93108e7d406567571bac8554e2af4a16f66866228b031d9e667fc14be300b'
[plugin:crio] added cmd output 'crictl inspectp 63cebf61e3759c6071083a836df7023e9a33baefa7e636d1047bae335ac7ceb2'
[plugin:crio] added cmd output 'crictl inspectp 6305b17a1d13b00841c11fe3dd327200a797e95c1a956668fd2f2a0734d240ee'
[plugin:crio] added cmd output 'crictl inspectp 3ec2aaa8580c092acbde97961c8e0948d454a385f0337924fe6c2732646aa750'
[plugin:crio] added cmd output 'crictl inspectp ec476f107f853fe6a9dc65bb1060c13784670fb507fcd32501a87d6edbbf9871'
[plugin:crio] added cmd output 'crictl inspectp 3170b07475934d343dd7ea20baff3b99e27a126c4887efbd5d540e5b6d7f2ab2'
[plugin:crio] added cmd output 'crictl inspectp edbbc3d48bf10d4c2481b5dc8fca5d4ca5a6ac47f18990eff73659efcea787e1'
[plugin:crio] added cmd output 'crictl inspectp 23151de9a086f3f4d9bc62166761b83b5e08e502df74a0125c1821b31e50e4b7'
[plugin:crio] added cmd output 'rpm -V cri-o cri-tools'
 Running plugins. Please wait ...

  Starting 1/1   crio            [Running: crio]
[plugin:crio] collecting path '/host/etc/crictl.yaml'
[plugin:crio] collecting path '/host/etc/sysconfig/crio-network'
[plugin:crio] collecting path '/host/etc/crio/crio.conf.d/00-default'
[archive:TarFileArchive] File not collected: '[Errno 2] No such file or directory: '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs/etc/crio/crio.conf.d/00-default''
[plugin:crio] collecting path '/host/etc/crio/seccomp.json'
[plugin:crio] collecting path '/host/etc/containers/registries.d'
[archive:TarFileArchive] File not collected: '[Errno 2] No such file or directory: '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs/etc/containers/registries.d/default.yaml''
[plugin:crio] collecting path '/host/etc/containers/registries.conf'
[plugin:crio] collecting path '/host/etc/sysconfig/crio-storage'
[plugin:crio] collecting path '/host/etc/crio/crio.conf'
[plugin:crio] collecting path '/host/etc/containers/oci/hooks.d'
[plugin:crio] collecting path '/host/etc/containers/certs.d'
[plugin:crio] collecting path '/host/etc/sysconfig/crio-metrics'
[plugin:crio] collecting path '/host/etc/containers/policy.json'
[plugin:crio] collecting path '/host/etc/containers/storage.conf'
[plugin:crio] collecting output of 'journalctl --no-pager  --unit crio'
[plugin:crio] collecting output of 'ls -alhR /etc/cni'
[plugin:crio] collecting output of 'crictl info'
[plugin:crio] collecting output of 'crictl images'
[plugin:crio] collecting output of 'crictl pods'
[plugin:crio] collecting output of 'crictl ps'
[plugin:crio] collecting output of 'crictl ps -a'
[plugin:crio] collecting output of 'crictl ps -v'
[plugin:crio] collecting output of 'crictl stats'
[plugin:crio] collecting output of 'crictl version'
[plugin:crio] collecting output of 'crictl inspect 69170aa14104d14d9bd89d3f596c41680cef47e6d3462015612d60606592a79d'
[plugin:crio] collecting output of 'crictl inspect 59a9e67adc2be3b152539e0cc08174cd0cbb8a50d209571084690feb98be6565'
[plugin:crio] collecting output of 'crictl inspect fde30ac88d3e7dff36849bae87f3cf6ec37f51470a3f46611373ec7197be58c9'
[plugin:crio] collecting output of 'crictl inspect 44a2df4b9fd37371002094ee9c2bd0b1c5f9ea6c975b97fd189e62cff7a6d1ad'
[plugin:crio] collecting output of 'crictl inspect 7a959038caae355994737487ac16011dab68eb602f71e5298efb018db0aa9a45'
[plugin:crio] collecting output of 'crictl inspect a74688f424244aa868c0d2001e2c0196a8e155672019f74160c5ad8d99544be1'
[plugin:crio] collecting output of 'crictl inspect fc4e9ac5b121df8489564e6e94e6982932403bde6cd88b0d60f778c8c8dfeeb0'
[plugin:crio] collecting output of 'crictl inspect 56d7f1bd079e9992e15fe5f3fdab286882c2d43964e9f08d4904884a4ce00a33'
[plugin:crio] collecting output of 'crictl inspect c653b04c8616af02a25de0f95adc3f4cc4a2e7b101481c30509dfbd511aa252b'
[plugin:crio] collecting output of 'crictl inspect b29c79d24a365af0c9fd713bf510180031d8edc722d5352ba97dcfd7ef0a65fd'
[plugin:crio] collecting output of 'crictl inspect 2f67ddb30860b23392a91bddef2e6d7baecc620adab179aadcf48f9a74904494'
[plugin:crio] collecting output of 'crictl inspect 6b44fff419cc4c982f640060815503c3683d315506718a8e539c013fdf538121'
[plugin:crio] collecting output of 'crictl inspect 981f46523451ef0571759729282dd48692be2496a47e5dd36cc76ef4900cb53a'
[plugin:crio] collecting output of 'crictl inspect c6801d2cb1d7a8ed7ec0f9dc4d8a6f7be920345fd8aebc857ab88825fc4e9d81'
[plugin:crio] collecting output of 'crictl inspect 4bb41f72e2d07c11da77b52cb4c42f28bb2eee493a723c92f87e7df5b2d0f0b9'
[plugin:crio] collecting output of 'crictl inspect 65cabedb077a9a52c3a987f3b9c5fa6ce78d0582198ccad82391a59736d54cc7'
[plugin:crio] collecting output of 'crictl inspect bd2a9bff124b26c955452cb3ed0f84c448b17c17190ee851fdee3e130f0400c3'
[plugin:crio] collecting output of 'crictl inspect 1d2f5742bb02536aedc7e9b1d58f4a638476062fecaba31ea107c88234ed6f8c'
[plugin:crio] collecting output of 'crictl inspect 99ef38f6ef8f8c2eac3d1fb2f8c5c1940fb9e7cd431012e2e6b093aef017e633'
[plugin:crio] collecting output of 'crictl inspect d1f451397b4815902a1e02883b53bcd77382d6faf554ca25d7b2d4d459dcccbd'
[plugin:crio] collecting output of 'crictl inspect f853b2431386cbb39e0e722653b5cd66e5e9302bcc3ddfabb041dfd145581e20'
[plugin:crio] collecting output of 'crictl inspect 8ff1f2943ca09ef0d845ebd9392e6b1fc035af047ad5874dd17317c4c82bf330'
[plugin:crio] collecting output of 'crictl inspect 6a70a0edb7846fc93c26a31ea52940ad577ea3da4a820282ea4522f55f1fcf53'
[plugin:crio] collecting output of 'crictl inspect 48f082b7e5a78c10a6b38629af011aeb0f32f7882724f5ffcd49954c8d9a766a'
[plugin:crio] collecting output of 'crictl inspect abdebdd02e1af43e5d03b672375ff77d7a0860fe0ad96d684b867e8e842f391a'
[plugin:crio] collecting output of 'crictl inspect 03a7f195d0b052b53002bd413717583729583a11c0c57a3e667f27cadb4ad037'
[plugin:crio] collecting output of 'crictl inspect ef8e973c311181c51084f37d5142a0d492c7862946d5079b1b7604489288b752'
[plugin:crio] collecting output of 'crictl inspect cdf979813984ec281c784867addc74a430976419a0642e84c3ad9f2e0282684a'
[plugin:crio] collecting output of 'crictl inspect f8c21c497ec4c9bf479460904bab85c630c11d7879eb8a65c0efdf445cd2ac15'
[plugin:crio] collecting output of 'crictl inspect b83290c647cf3ef623297120c107352f586201be4670d8ae686514eb10209afc'
[plugin:crio] collecting output of 'crictl inspect ec805db82f25957b0bec5ab63e5e58e9a035095a863a73771a599c80bc47dcf3'
[plugin:crio] collecting output of 'crictl inspect 5a3b4bd0e23ffaaa9c5e09807cfff556e3d742652f8dc280b40a6924f305b83b'
[plugin:crio] collecting output of 'crictl inspect 7c8aedbfdab90472241ab4bf9c2ed61bc29ebebbe36f9e4346e458b7f351b1e4'
[plugin:crio] collecting output of 'crictl inspect bfa165d7f8d0de89269559bb73d00fef7bc2ef96436b92e524ea32dd6f9e9b19'
[plugin:crio] collecting output of 'crictl inspect 09279cf1e70ce38c2efb644aa117c8e1580cf659175f2eebd433eec33d39d88d'
[plugin:crio] collecting output of 'crictl inspect d3b2600d44d2f5ef2712f27ef96847fbd220ab0ec2b8f8553f64344a1ff5f6ca'
[plugin:crio] collecting output of 'crictl inspect 605251eafb1e78fb9e46b95f2690c07216ca8827491cb10d54a8d0b94ff7d184'
[plugin:crio] collecting output of 'crictl inspect 5e0e163de40f7fa1de8eb716d60d3b41270e6fdca34a022679a1ebb85def2285'
[plugin:crio] collecting output of 'crictl inspect 5f753f860af7c5ee4dcd8e44c3b6f61f3e155090d5d8a0656b891cb3b310b160'
[plugin:crio] collecting output of 'crictl inspect 2866f151feb763efec85336b59bd7f9d2c8f395e6ad271b631f05699b555f50c'
[plugin:crio] collecting output of 'crictl inspect b056b80802703a2c8b7f7b27bd987e914119419678f10bd2f4b0c765afb9e709'
[plugin:crio] collecting output of 'crictl inspect f3a5476cb72e80bbd65b780ce580fdc68d51625c027d53ee695b0170db429058'
[plugin:crio] collecting output of 'crictl inspect b92804d7ad516351b1e8e0bd27e2577fbfb628b8fe7f7bfc61139bd60cd45956'
[plugin:crio] collecting output of 'crictl inspect cbd39e7a9566577f8ec97f08b063060bcf7e6f81e8dc7dd093b4ade4657a8d1d'
[plugin:crio] collecting output of 'crictl inspect 9afb24750d672bb431b5ff8871c753eceae97e6c49db4637d4d1a685f428d828'
[plugin:crio] collecting output of 'crictl inspect da567fda23c0b7caef669f49ccfaef7f84bfeaf27c4de672a65efcb02adf5c62'
[plugin:crio] collecting output of 'crictl inspect c11f150adf57e2acbbd627783276fe7a776622eada7a4ef764253204d4ee0feb'
[plugin:crio] collecting output of 'crictl inspect e3442d6cd977817fbb9ee2cc2f986c262d20839bd939d35306a5f5d31c410158'
[plugin:crio] collecting output of 'crictl inspect f5f1aab2fdc3828bd89105ff4110410cda80adf1cfa94e942c50104e9c769d77'
[plugin:crio] collecting output of 'crictl inspect 2bff47a20fcce2ba72e944f0bf8bcc4039370837239315296837e36451f5e7de'
[plugin:crio] collecting output of 'crictl inspect 333b58865c6b3c5f2d15601a73016fd4000f063b0f69c0877afe049066951fb3'
[plugin:crio] collecting output of 'crictl inspect 92cee5912790382b87f4d97e53df721dd6fa86f155d5f9ddce94f4a4455a1547'
[plugin:crio] collecting output of 'crictl inspect d2c443208e6f3823105029982fb8b2b413929c676780ca46aea3b59714a28d0f'
[plugin:crio] collecting output of 'crictl inspect ce4ada2146c187e6fe6e5fb74974938a24ddbc0894e93aded9e27d39fac84ea9'
[plugin:crio] collecting output of 'crictl inspect 8db1624537fbf52a32dfb35a8f1727abf06b442fc28a2f3c5b7d21a0775ef34b'
[plugin:crio] collecting output of 'crictl inspect 455a5c039afabc15e59e07400a535a1dee949da75bd7bf7d4a292af6df7844c8'
[plugin:crio] collecting output of 'crictl inspect 9a492e62cef1052f9ea8f094f709cd54066511307df46f9f5b6478c69466062d'
[plugin:crio] collecting output of 'crictl inspect fddbd301593c0be82f6be7b61d4ff07b19161b82b612bfff02949d3113ed4ce6'
[plugin:crio] collecting output of 'crictl inspect 8fb30497a3e8378d329fdc4d4d90fffd5db2d988cc932dedc1e91f8026b7883e'
[plugin:crio] collecting output of 'crictl inspect e72edeb8b56d06eddca369faeb4f6e4d5cc3f17f193d1b6029cc60bbadf9a014'
[plugin:crio] collecting output of 'crictl inspect 6e0b37a22b6ba915172b31a4e87d6e9616862b35a266320cce77c4bc9e347b94'
[plugin:crio] collecting output of 'crictl inspect 869a3d268a043d7cbe9c38dbde225b87b42d4834dc85645be241a0f66ba45ee7'
[plugin:crio] collecting output of 'crictl inspect 88ca2ae9822aeede26f667c4e2aedc4a99db9dcb2e11cd74ebbfdc11b5bad76f'
[plugin:crio] collecting output of 'crictl inspect ab78c567c0db7a7a9f3d3751cdc925c51e1bb5e137716b7e1910daef42ef685f'
[plugin:crio] collecting output of 'crictl inspect 60f934c6c2dd7a14196212e1e3f2df6926ad59a9ceab2d0ae9088342bf309217'
[plugin:crio] collecting output of 'crictl inspect edd220700913fb67a2dbbccbe7f97b21eb51f96e9aac5e220e1f4f094f40651a'
[plugin:crio] collecting output of 'crictl inspect 7e07f1d6834eaacbe4bcb436bef5f64f90fa316602e51bb3758244256bd02ccd'
[plugin:crio] collecting output of 'crictl inspect cdbd6d76d8c2aa2d2f597eeca30a2f0ab6e5120a5c0a619a6834f8213b195aca'
[plugin:crio] collecting output of 'crictl inspect 73ee0c1af4907bb495a7bfc99f14f407cacb7484705d138c34dc2c22868051fc'
[plugin:crio] collecting output of 'crictl inspect 726d0bd2f108fc71ebd095852ebcbbd41b00b7172ecf0631e3d7ccb633d170af'
[plugin:crio] collecting output of 'crictl inspect 1cbf89ce8eaec433bde30a00b9c38b8e1f069a65804344bd289af3e8609c612c'
[plugin:crio] collecting output of 'crictl inspect 0f107301e1dd59a0a67fb0d99be048a0f2ede9904d3ee52f0c2dea21b0685ce8'
[plugin:crio] collecting output of 'crictl inspect b33d5af31fa26e1ad13513f4440f3cc7d64656a0629df161a71e20396ddec8aa'
[plugin:crio] collecting output of 'crictl inspect b84a02b3f76697a7be30051b922f271666b8ff9a4559486260c4a356c0a77097'
[plugin:crio] collecting output of 'crictl inspect 9c7beffeb13525b426814d891a73189d59416cd838ded346f6a54f051fe86239'
[plugin:crio] collecting output of 'crictl inspecti 034f77b7b6206d6a19a4b45f6789aaf7881e0a13b723a3f924fddde99acfe1ae'
[plugin:crio] collecting output of 'crictl inspecti 09a01faefd4fcc313b7653d468878bf3427265d995cf7ca621e7e18992d47c51'
[plugin:crio] collecting output of 'crictl inspecti 3dbbd23e9657a5d059b04771dce535755cf9a3568785a0d309d29f14b3229944'
[plugin:crio] collecting output of 'crictl inspecti fbdfe329e9eb24a20023c7b84fccbffdd698a12ffd17e51ffa7728d15f2e2136'
[plugin:crio] collecting output of 'crictl inspecti c8328a0ded0e04dfe01d6424488cff688426587bb86b5c08079a5f7bc7f218cc'
[plugin:crio] collecting output of 'crictl inspecti fe3f46a31be413f0f5810ca8ea5fe54ab83ad8e7965980ea22af0c6c0371d3e8'
[plugin:crio] collecting output of 'crictl inspecti 85d381f7a4f998891f91af11b1658b16e34f52d3c05a8563d4eb8db27f79f286'
[plugin:crio] collecting output of 'crictl inspecti 7491f5f484164674f3d1610e58dab752dae275fabd5fe68fd8a13f9f52814217'
[plugin:crio] collecting output of 'crictl inspecti fdd1ce49495579b4c58c587bb180a69b854a971c7b2a9a39046d6960247f1123'
[plugin:crio] collecting output of 'crictl inspecti e654362b4a3c7ccac1da74df2d5ef8f50c1be1223ba8b31129ba56cfd010c444'
[plugin:crio] collecting output of 'crictl inspecti 10e525c5b8882b6a8606da0d1f98892d326c9b29d7c0b4b3fabbcb980ab57ee6'
[plugin:crio] collecting output of 'crictl inspecti cf43cf40cf48e8c192aa5f855b150fa580c8bb764788857d814c69b0598e0815'
[plugin:crio] collecting output of 'crictl inspecti 71d17f2c24c7409883769b1fe06812063da7c9615a5b1e5df54d6ef8d33fbbf3'
[plugin:crio] collecting output of 'crictl inspecti 1f9649341c2276f0bcde0c125059eafc68fe1449bafdf44ce0b7519013c0dbe9'
[plugin:crio] collecting output of 'crictl inspecti 3faaab80cd610654276b199994986568feb6376cdf72832a3595bc8b0b26497e'
[plugin:crio] collecting output of 'crictl inspecti a88630bfa771d811633766c29c2d953f8484a956b80460e659b345de57e8a114'
[plugin:crio] collecting output of 'crictl inspecti 1411a8630b920c81400e175a2b82bad9b6662ac049042f90c2e686e718c73d5b'
[plugin:crio] collecting output of 'crictl inspecti 7b8c808ab27a609df1a5983a070efd372d2365933aea93e03c01985971850e25'
[plugin:crio] collecting output of 'crictl inspecti 0d689230e34d1b56ac9149226728932862f9f4188d19c1337f5490aed4e78307'
[plugin:crio] collecting output of 'crictl inspecti 2abf30ff6ba99c2a03e2c6b4b9ccd6f389c1aa9f6bd0f020a21e5f7f0f5a493a'
[plugin:crio] collecting output of 'crictl inspecti 1c20b9ba40e4052f46ac37d1b41897db4f576ed56dcb68d6333c885214d3d5e1'
[plugin:crio] collecting output of 'crictl inspecti ee1ad20f0063bb88b3bffd9cfd0ceddeca5ef14d9fb7e1b033fac4d8e61bc140'
[plugin:crio] collecting output of 'crictl inspecti a30dfb279cfc7014c96a3c143eba4b62e4843ecda6d957f3f5f251c0b5f43b83'
[plugin:crio] collecting output of 'crictl inspecti 3264a20df581bff1f88671f55733f7f58bb4841702b6e6659858af640bc6855b'
[plugin:crio] collecting output of 'crictl inspecti c52903ef9ea83e0f043f4bcb1ef089411840f78000addb0ef1edda4862267145'
[plugin:crio] collecting output of 'crictl inspecti 806a0bf7684a4a9e1b20f3c422d4fb96513412e7d11632c1a482d29734b89ba7'
[plugin:crio] collecting output of 'crictl inspecti a43a61ece277a722fcdcc8e14449cdb95ee6710e32525c4d904ae851a68f0b77'
[plugin:crio] collecting output of 'crictl inspecti c374de0d959263981871ea0134ef048de3a20b212c1b9f4cf9eda78f9a603e53'
[plugin:crio] collecting output of 'crictl inspecti 7307cec9948467ed333fddb457eff62cc03fd4060fd57adcc378a91ed1d0dd0b'
[plugin:crio] collecting output of 'crictl inspecti 90bf71031466153a92fd9dc71e064f6aac63e9fce27a2636f036a09ca7ff26ea'
[plugin:crio] collecting output of 'crictl inspecti dbd125e25c8ade62f2938361146b23740fd7874c9e33714a29646b12cd3e017f'
[plugin:crio] collecting output of 'crictl inspecti 140253c0c1a51e61bcb113e1b837496e8958b7b90c027fec8f4b5ddb08e060fc'
[plugin:crio] collecting output of 'crictl inspecti b9f516edc1b156b0504465b6075adc9814f447debec401890318f8ac31054dfe'
[plugin:crio] collecting output of 'crictl inspecti 95534a9fb872628fbeda0f52f8d06462042ca8dd838b0ebefecc9c56b92218eb'
[plugin:crio] collecting output of 'crictl inspecti e2d8c35ce60c1e429e8f6160507bbde2f7a17635e138192f9cf731eda2b849a4'
[plugin:crio] collecting output of 'crictl inspecti 154226eaa544c7cd8fc91996d77fac472b1f149378399529553c62d7630b2158'
[plugin:crio] collecting output of 'crictl inspecti 24ff3011f60ea1ade69afd61b3a8e3eb2ee6b1fe2c342aaf0f9076a3bf710ca3'
[plugin:crio] collecting output of 'crictl inspecti 182e2d06a92b449a358f734fbd0aa94e9968f5e9dcbacea334a3ff64d2da54b3'
[plugin:crio] collecting output of 'crictl inspecti 21162c320f9459d85ce6bd7070433ffa31d62358ed40afe074b790ccddcab782'
[plugin:crio] collecting output of 'crictl inspecti 8f13cacfce95f5b2a0c2b50df05869e3236b07dfc9a0c5102b41b04d07b8fea0'
[plugin:crio] collecting output of 'crictl inspecti bdb981388da5ffedc0e8b0069e0309b4d3978550ef9ee46609bb4f58880e527e'
[plugin:crio] collecting output of 'crictl inspecti 4675029e660ab72a42c14e8032de66f14fafb663b7b42560b79e837c56b9389f'
[plugin:crio] collecting output of 'crictl inspecti bdf8e0b08db5ed5f436f9bed5fb6f4981326e428cfb6769e450360db64a29e75'
[plugin:crio] collecting output of 'crictl inspecti e666790521c3d46bc09d46da58d71c8fbe1187a47dcb7c10fffa64a9124b8c7b'
[plugin:crio] collecting output of 'crictl inspecti 6202cfe10d67716d76162dd34a924c4f43335036cbb1d2d9a13f2fe894cb0da2'
[plugin:crio] collecting output of 'crictl inspecti 2129063b76c3703ca046cf31a4b59b231dae9253971ad65e2517f8c78c16f86a'
[plugin:crio] collecting output of 'crictl inspecti f0e053c66e8bc4afc5f6b73a71447cc54cd399191a8aaaf62dc2cbc6900a4c21'
[plugin:crio] collecting output of 'crictl inspecti ffb8ac0907237aac782d9c1e796f717f8ce3824f76bbdd8277c43b24ab9ec907'
[plugin:crio] collecting output of 'crictl inspecti 89e8127deda47f6a0157c0794916e371eea4b4980d01c4ba17e929f95226976f'
[plugin:crio] collecting output of 'crictl inspectp b08884374aa3cc9e11e697919688eefebfabea4dce9cb9e61876b052d65ec942'
[plugin:crio] collecting output of 'crictl inspectp c01ed02be1a3cd9f8d87c18f155912ea4857622e7c99ce4f6798f799b177dc9d'
[plugin:crio] collecting output of 'crictl inspectp e429a27ca6cd2c3ef6d1dc79df4e4f1cf1dc188c6450ded8ce0209ebff668b66'
[plugin:crio] collecting output of 'crictl inspectp b25c44221a233cb233ddcd25a6eda445fce540a85b18e752868acaf67bcb50c6'
[plugin:crio] collecting output of 'crictl inspectp 25ff819c050b6fb0d0cd830d0cf41f3a7a4113bfeb93d16c38ac2ab775362df3'
[plugin:crio] collecting output of 'crictl inspectp fb3c4ec8ddc91e9607aae99e11566addb4d988660561525281ce03db5b4d69ff'
[plugin:crio] collecting output of 'crictl inspectp 703f481bd38d187730b049cd5327d1fcd11bbc4bc73f1fe8984dc4830dd95e7d'
[plugin:crio] collecting output of 'crictl inspectp 0ebe623d23a413d1aff6eafd8aa093580389c6be80c21dbd9eb50238045a2b32'
[plugin:crio] collecting output of 'crictl inspectp ca280a7fba950ee532527bb427e079fd91a30c85bc92bf629b71f9f593e091e5'
[plugin:crio] collecting output of 'crictl inspectp b007b18daf6293b4d8b5d185d5ec72442bf1b0125bd80b52914bf0b80caecb9f'
[plugin:crio] collecting output of 'crictl inspectp e1be83b1b9b62b46964ddadb4d5b7e3406a1ea550bae9278ab500ed6d9d06c39'
[plugin:crio] collecting output of 'crictl inspectp eb8a798ff18e9e3586abd8941555e8153f02bb46fc3178a1ff499152adb672cb'
[plugin:crio] collecting output of 'crictl inspectp 878357fa4458cfc9ea6f13342d00639315171bcd4fe6eb814829601c632d64c2'
[plugin:crio] collecting output of 'crictl inspectp 877f831a6eb9ebda8722e841360c90611ea3827ee1820e3edc94976ee601514e'
[plugin:crio] collecting output of 'crictl inspectp 95acbc095f712d127d36dfa5b4458af29c194ad103b44130753f256f76c9b4bb'
[plugin:crio] collecting output of 'crictl inspectp 3f7cc61df55d554f260aee849bab1cb15324d99a300663b8ea2bc5e85074f100'
[plugin:crio] collecting output of 'crictl inspectp c9f5c132cb0ee9f416d00684ed30e58d8a386c93b7a533e4ca4271b1ad31936e'
[plugin:crio] collecting output of 'crictl inspectp 87eb87538debf0642a7a7f362aa9608ee54b716de62611e71eb58d99412a2c0c'
[plugin:crio] collecting output of 'crictl inspectp aa7b71c520c46212d1d5e6ac8e51bea69abea5f88545e61c430f95c351d9b675'
[plugin:crio] collecting output of 'crictl inspectp f98490a20b6cba2c331b76182b91d94cdfc8be9797602a80ad065882aee48a5b'
[plugin:crio] collecting output of 'crictl inspectp ebd8e69aa0f6576adf4acf9eb0627ebbb49bc0427457df74e286f033222954a8'
[plugin:crio] collecting output of 'crictl inspectp 51acae7314c39947aab6c63efcc16b04809dc1f503e3749c99ec718e5232a539'
[plugin:crio] collecting output of 'crictl inspectp 650ae38f26a8efd2e91769eec1b79b5b01a3f62ca70d82dfed41d6fb7ca459bb'
[plugin:crio] collecting output of 'crictl inspectp fe4bd0644eff21f6fa8ca7925d426e0c97c02debd9f18b33b23ac6b05357a03b'
[plugin:crio] collecting output of 'crictl inspectp 5f6e7bd39d2896521f0eb0b4d27a93bd352210ad8fe813ba65a7299669132f7b'
[plugin:crio] collecting output of 'crictl inspectp 2789070a6c91f35fca6c6024a5ea4b0742dbda084b15f86e105032468296e00b'
[plugin:crio] collecting output of 'crictl inspectp 4f456db3b8a9705c1c4a859af8ba8046a403027b87c291eac161199723270b6a'
[plugin:crio] collecting output of 'crictl inspectp 56ec9d460947cbdee8b7b1b3ae0c7e891bdf5ae0e3b067eb26d3c4345868b79c'
[plugin:crio] collecting output of 'crictl inspectp 0b5ce1bd4c72933e06cf50ed14dd4d243828d94208b645234d3a4502d0fc9f83'
[plugin:crio] collecting output of 'crictl inspectp 97e49d48a2383b787c06ae8a701ed67883e8e760dc932c03f0b4f7e7c32f4537'
[plugin:crio] collecting output of 'crictl inspectp a398bbe05cc39a9f795d331c9556c287a8b91b3eabdd4d9d7ca103f0b5873fcb'
[plugin:crio] collecting output of 'crictl inspectp 06ce271f732d5db79a841777ab7204ead8ff27b63d808f3a4b16ae46c05fb232'
[plugin:crio] collecting output of 'crictl inspectp a45b9b7166ac32056975b8fb384e8b77aabd2175ee41edb10f97c0ba7d93903e'
[plugin:crio] collecting output of 'crictl inspectp d6a70055acf3718eb4623a333dfb5231f5b9411adeb3439d3285022097375e75'
[plugin:crio] collecting output of 'crictl inspectp a0ff09e1d825352c327fd7278c96b5b80c619404d7f31b838e28f420648ae0fe'
[plugin:crio] collecting output of 'crictl inspectp d752e107bb2d04d8a666c5ff8980658e6dfd4d29c55d644d2f567e5f3af7e31b'
[plugin:crio] collecting output of 'crictl inspectp 7d277f768f783bfb2a51248831252573c8ced5d8736ad6105d427e7d8b7cd599'
[plugin:crio] collecting output of 'crictl inspectp 02479e5b22b15a832b2c62d7d37790b3727bc30746f72793a31b239f09304122'
[plugin:crio] collecting output of 'crictl inspectp c1fdf28d00c1684847101d3c52525d78d51b6ce4f44ce76729d4eabbd6e3f9cd'
[plugin:crio] collecting output of 'crictl inspectp 946c0a5a45cf8eb481e623d30773852fe36d32a20e0ae9c762d974da3ed9f64c'
[plugin:crio] collecting output of 'crictl inspectp 39d5554bd633166e3fd0dca627ce887a225f4c43b73fd423148b2e6a03880529'
[plugin:crio] collecting output of 'crictl inspectp daf57ff6f914a8111b843093f3afa59b084e1d79ceaf2ed3b5f80222996b2853'
[plugin:crio] collecting output of 'crictl inspectp 0ce17bc1dd4dcdba7521c9d4f3da3e565e9698057b807a4110ef24485e906419'
[plugin:crio] collecting output of 'crictl inspectp e8f6f151d6085e02b37237c99d8c7c460ddcb2eadc52c02df69da94f5fd5e83a'
[plugin:crio] collecting output of 'crictl inspectp f03020fb7ee043d529be5382b5722c53327c74c8b72cfd7e76c2ec45f91b4b4e'
[plugin:crio] collecting output of 'crictl inspectp 3e7b37019427bb1407f0cc91e067d24c4f7aa28ec1b170384337eb6e8cc36dc2'
[plugin:crio] collecting output of 'crictl inspectp b2d93108e7d406567571bac8554e2af4a16f66866228b031d9e667fc14be300b'
[plugin:crio] collecting output of 'crictl inspectp 63cebf61e3759c6071083a836df7023e9a33baefa7e636d1047bae335ac7ceb2'
[plugin:crio] collecting output of 'crictl inspectp 6305b17a1d13b00841c11fe3dd327200a797e95c1a956668fd2f2a0734d240ee'
[plugin:crio] collecting output of 'crictl inspectp 3ec2aaa8580c092acbde97961c8e0948d454a385f0337924fe6c2732646aa750'
[plugin:crio] collecting output of 'crictl inspectp ec476f107f853fe6a9dc65bb1060c13784670fb507fcd32501a87d6edbbf9871'
[plugin:crio] collecting output of 'crictl inspectp 3170b07475934d343dd7ea20baff3b99e27a126c4887efbd5d540e5b6d7f2ab2'
[plugin:crio] collecting output of 'crictl inspectp edbbc3d48bf10d4c2481b5dc8fca5d4ca5a6ac47f18990eff73659efcea787e1'
[plugin:crio] collecting output of 'crictl inspectp 23151de9a086f3f4d9bc62166761b83b5e08e502df74a0125c1821b31e50e4b7'
[plugin:crio] collecting output of 'rpm -V cri-o cri-tools'

  Finished running plugins

Creating compressed archive...
[archive:TarFileArchive] finalizing archive '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs' using method 'auto'
[archive:TarFileArchive] built archive at '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs.tar' (size=5099520)

Your sosreport has been generated and saved in:
	/host/var/tmp/sosreport-dell-r430-08-2020-12-09-ovdmsrs.tar.xz

 Size	208.55KiB
 Owner	root
 md5	dc877c716bd894080a8461ebe7260339

Please send this file to your support representative.




Error is:

[plugin:crio] collecting path '/host/etc/crio/crio.conf.d/00-default'
[archive:TarFileArchive] File not collected: '[Errno 2] No such file or directory: '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs/etc/crio/crio.conf.d/00-default''

Comment 12 Pavel Moravec 2020-12-10 11:56:53 UTC

Thanks for the testing. The failing command is simple

shutil.copy(src, dest)

So why that command fails for /host/etc/crio/crio.conf.d/00-default , but succeeds for e.g. /host/etc/sysconfig/crio-network or /host/etc/crio/seccomp.json , per:

[plugin:crio] collecting path '/host/etc/sysconfig/crio-network'
[plugin:crio] collecting path '/host/etc/crio/crio.conf.d/00-default'
[archive:TarFileArchive] File not collected: '[Errno 2] No such file or directory: '/host/var/tmp/sos.t3dxp7xm/sosreport-dell-r430-08-2020-12-09-ovdmsrs/etc/crio/crio.conf.d/00-default''
[plugin:crio] collecting path '/host/etc/crio/seccomp.json'

?


"No such file or directory" - is that file accessible from within the container? Can you copy it?

Comment 13 Derrick Ornelas 2020-12-10 15:40:55 UTC

Sorry, I should have included that information.  Yes, it is accessible from within the container:

# toolbox 
Container 'toolbox-root' already exists. Trying to start...
(To remove the container and start with a fresh toolbox, run: sudo podman rm 'toolbox-root')
toolbox-root
Container started successfully. To exit, type 'exit'.


[root@dell-r430-08 /]# ls -la /host/etc/crio/crio.conf.d/00-default /host/etc/sysconfig/crio-network  /host/etc/crio/seccomp.json
-rw-r--r--. 1 root root   868 Nov 17 22:47 /host/etc/crio/crio.conf.d/00-default
-rw-r--r--. 1 root root 11937 Nov 17 22:50 /host/etc/crio/seccomp.json
-rw-r--r--. 1 root root    53 Nov 17 22:50 /host/etc/sysconfig/crio-network


[root@dell-r430-08 /]# cp -vr /host/etc/crio/crio.conf.d/ /host/var/tmp/.
'/host/etc/crio/crio.conf.d/' -> '/host/var/tmp/./crio.conf.d'
'/host/etc/crio/crio.conf.d/00-default' -> '/host/var/tmp/./crio.conf.d/00-default'


[root@dell-r430-08 /]# mktemp -d /host/var/tmp/test.XXXXXXX
/host/var/tmp/test.RmjdNuK


[root@dell-r430-08 /]# cp -vr /host/etc/crio/crio.conf.d/ /host/var/tmp/test.RmjdNuK/
'/host/etc/crio/crio.conf.d/' -> '/host/var/tmp/test.RmjdNuK/crio.conf.d'
'/host/etc/crio/crio.conf.d/00-default' -> '/host/var/tmp/test.RmjdNuK/crio.conf.d/00-default'


root@dell-r430-08 /]# ls -lR /host/var/tmp/test.RmjdNuK/
/host/var/tmp/test.RmjdNuK/:
total 0
drwxr-xr-x. 2 root root 24 Dec 10 15:36 crio.conf.d

/host/var/tmp/test.RmjdNuK/crio.conf.d:
total 4
-rw-r--r--. 1 root root 868 Dec 10 15:36 00-default

Comment 14 Pavel Moravec 2020-12-14 08:45:04 UTC

(In reply to Derrick Ornelas from comment #13)
> Sorry, I should have included that information.  Yes, it is accessible from
> within the container:
> 

Could you please share the reproducer with me? (how to login to the host system, run the container and how to allow me some changes within the container).

Thanks in advance.

Comment 16 Pavel Moravec 2020-12-15 13:26:27 UTC

This is a tricky bug, shame on it:

sed -i "s/self.opts.sysroot,/self.sysroot,/g" /usr/lib/python3.6/site-packages/sos/component.py

is the fix.

https://github.com/sosreport/sos/pull/2346 raised, will be in the next re-spin.

Thanks for a good eye on this!

Comment 17 Pavel Moravec 2020-12-15 13:44:21 UTC

Jake,
the https://github.com/sosreport/sos/pull/2346 should have some trivial reproducer (e.g. using support tools container) - are you able to come up with some, to help QE to verify it as well?

Thanks in advance.

Comment 18 Pavel Moravec 2021-01-04 16:16:10 UTC

Hello,
could you please re-test the fix on the below build? Thanks in advance.


A yum repository for the build of sos-4.0-4.el8 (task 34038848) is available at:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/4.el8/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/4.el8/sos-4.0-4.el8.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/4.el8/noarch/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/4.el8/noarch/sos-4.0-4.el8.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/4.el8/noarch/sos-4.0-4.el8.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/4.el8/noarch/sos-audit-4.0-4.el8.noarch.rpm

The repository will be available for the next 60 days. Scratch build output will be deleted
earlier, based on the Brew scratch build retention policy.

Comment 25 errata-xmlrpc 2021-05-18 14:48:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sos bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1604