Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
sosreport is stuck after "Finished running plugins" is displayed.
I asked the customer to run sosreport with --debug option.
As the customer said, sosreport was not responding after "Finished running plugins" is displayed.
-- cfdip110_20191125(18БF24БF58).log --
[Mon Nov 25 18:26:45.325 2019] (overcloud) [root@server ~]# sosreport --debug -v -n networking,logs,filesys
...snip..
...
[Mon Nov 25 18:32:06.545 2019] [sos.sosreport:setup] executing 'sosreport --debug -v -n networking,logs,filesys'
[Mon Nov 25 18:32:06.545 2019] [sos.sosreport:setup] using 'rhosp' preset defaults (--all-logs --plugopts process.lsof=off --threads 4)
[Mon Nov 25 18:32:06.545 2019] [sos.sosreport:setup] effective options now: --all-logs --debug --noplugins networking,logs,filesys --plugopts process.lsof=off --threads 4 -v
...
[Mon Nov 25 18:36:51.545 2019] [plugin:docker] collecting output of 'docker inspect 6e4ee0c1c89a'
[Mon Nov 25 18:36:51.608 2019] [plugin:docker] collecting output of 'docker inspect 542704fcfd2a'
[Mon Nov 25 18:36:51.686 2019] Finishing plugins [Running: pacemaker]
[Mon Nov 25 18:37:18.329 2019]
[Mon Nov 25 18:37:18.329 2019] Finished running plugins
[Mon Nov 25 18:37:18.377 2019]
[Mon Nov 25 18:48:05.389 2019] ^C
[Mon Nov 25 18:48:05.405 2019] Exiting on user cancel
[Mon Nov 25 18:48:10.624 2019]
[Mon Nov 25 18:48:16.796 2019] (overcloud) [root@cfocp020 ~]#
At that time, only sosreport had high CPU usage by monitoring with ps or top even though it didn't yet create an archive in the background.
-- cfdip110_20191125(18БF28БF16).log --
[Mon Nov 25 18:30:15.804 2019] [root@cfocp020 ~]# ps -aux
...
[Mon Nov 25 18:37:35.207 2019] root 743520 74.4 0.6 1806608 820904 pts/0 R+ 18:31 4:08 /usr/bin/python /sbin/sosreport -
~~~
[Mon Nov 25 18:47:51.685 2019] root 743520 91.1 0.6 1864920 860616 pts/0 R+ 18:31 14:24 /usr/bin/python /sbin/sosreport -
-- cfdip110_20191125(18БF26БF56).log --
[Mon Nov 25 18:29:51.162 2019] [root@cfocp020 ~]# top
...
[Mon Nov 25 18:37:19.158 2019] 743520 root 20 0 1806604 820892 8380 R 23.9 0.6 3:56.92 sosreport
[Mon Nov 25 18:37:22.267 2019] 743520 root 20 0 1806608 820904 8388 R 100.0 0.6 4:00.04 sosreport
[Mon Nov 25 18:37:25.376 2019] 743520 root 20 0 1806608 820904 8388 R 100.0 0.6 4:03.15 sosreport
~~~
[Mon Nov 25 18:47:51.967 2019] 743520 root 20 0 1865088 853308 8388 R 100.0 0.6 14:29.71 sosreport
For the above logs, please see the files in the attachment "20191125_cfocp020_sosreport_debug.zip".
How reproducible:
- Always in the customer system
Steps to Reproduce:
1. Run sosreport as follows:
# sosreport -n networking,logs,filesys
Actual results:
sosreport is stuck.
Expected results:
sosreport completes.
Additional info:
- kernel-3.10.0-957.el7.x86_64
- sos-3.7-10.el7_7
- Red Hat OpenStack Platform 13.0
- The customer says they will run director's RC file for overcloud before running sosreport.
- I have seen some similar cases that sosreport has a problem in OpenShift. So I guess it's issue specific to container host.
I bet this is dup of 1704957 : trivial workaround: run sosreport with --no-report. If I am right (will double-check with the sos data), then this is fixed in upcoming 7.8.