RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1784234 - sosreport is stuck after "Finished running plugins" is displayed.
Summary: sosreport is stuck after "Finished running plugins" is displayed.
Keywords:
Status: CLOSED DUPLICATE of bug 1704957
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sos
Version: 7.7
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Jan Jansky
QA Contact: Upgrades and Supportability
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-17 03:07 UTC by kyoneyama
Modified: 2023-09-18 00:19 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-27 08:20:24 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description kyoneyama 2019-12-17 03:07:08 UTC 
sosreport is stuck after "Finished running plugins" is displayed.

I asked the customer to run sosreport with --debug option.
As the customer said, sosreport was not responding  after "Finished running plugins" is displayed.

  -- cfdip110_20191125(18БF24БF58).log --
  [Mon Nov 25 18:26:45.325 2019] (overcloud) [root@server ~]# sosreport --debug -v -n networking,logs,filesys
  ...snip..
  ...
  [Mon Nov 25 18:32:06.545 2019] [sos.sosreport:setup] executing 'sosreport --debug -v -n networking,logs,filesys'
  [Mon Nov 25 18:32:06.545 2019] [sos.sosreport:setup] using 'rhosp' preset defaults (--all-logs --plugopts process.lsof=off --threads 4)
  [Mon Nov 25 18:32:06.545 2019] [sos.sosreport:setup] effective options now: --all-logs --debug --noplugins networking,logs,filesys --plugopts process.lsof=off --threads 4 -v
   ...
  [Mon Nov 25 18:36:51.545 2019] [plugin:docker] collecting output of 'docker inspect 6e4ee0c1c89a'
  [Mon Nov 25 18:36:51.608 2019] [plugin:docker] collecting output of 'docker inspect 542704fcfd2a'
  [Mon Nov 25 18:36:51.686 2019]   Finishing plugins              [Running: pacemaker]
  [Mon Nov 25 18:37:18.329 2019] 
  [Mon Nov 25 18:37:18.329 2019]   Finished running plugins
  [Mon Nov 25 18:37:18.377 2019] 
  [Mon Nov 25 18:48:05.389 2019] ^C
  [Mon Nov 25 18:48:05.405 2019] Exiting on user cancel
  [Mon Nov 25 18:48:10.624 2019] 
  [Mon Nov 25 18:48:16.796 2019] (overcloud) [root@cfocp020 ~]# 



At that time, only sosreport had high CPU usage by monitoring with ps or top even though it didn't yet create an archive in the background.

  -- cfdip110_20191125(18БF28БF16).log --
  [Mon Nov 25 18:30:15.804 2019] [root@cfocp020 ~]# ps -aux
  ...
  [Mon Nov 25 18:37:35.207 2019] root      743520 74.4  0.6 1806608 820904 pts/0  R+   18:31   4:08 /usr/bin/python /sbin/sosreport -
  ~~~
  [Mon Nov 25 18:47:51.685 2019] root      743520 91.1  0.6 1864920 860616 pts/0  R+   18:31  14:24 /usr/bin/python /sbin/sosreport -


  -- cfdip110_20191125(18БF26БF56).log --
  [Mon Nov 25 18:29:51.162 2019] [root@cfocp020 ~]# top
  ...
  [Mon Nov 25 18:37:19.158 2019]  743520 root      20   0 1806604 820892   8380 R  23.9  0.6   3:56.92 sosreport
  [Mon Nov 25 18:37:22.267 2019]  743520 root      20   0 1806608 820904   8388 R 100.0  0.6   4:00.04 sosreport    
  [Mon Nov 25 18:37:25.376 2019]  743520 root      20   0 1806608 820904   8388 R 100.0  0.6   4:03.15 sosreport 
  ~~~
  [Mon Nov 25 18:47:51.967 2019]  743520 root      20   0 1865088 853308   8388 R 100.0  0.6  14:29.71 sosreport  

For the above logs, please see the files in the attachment "20191125_cfocp020_sosreport_debug.zip".


How reproducible:

- Always in the customer system

Steps to Reproduce:

1. Run sosreport as follows:

  # sosreport -n networking,logs,filesys


Actual results:

sosreport is stuck.


Expected results:

sosreport completes.


Additional info:

- kernel-3.10.0-957.el7.x86_64
- sos-3.7-10.el7_7

- Red Hat OpenStack Platform 13.0
- The customer says they will run director's RC file for overcloud before running sosreport.
- I have seen some similar cases that sosreport has a problem in OpenShift. So I guess it's issue specific to container host.
Comment 3 Pavel Moravec 2019-12-20 12:55:33 UTC 
I bet this is dup of 1704957 : trivial workaround: run sosreport with --no-report. If I am right (will double-check with the sos data), then this is fixed in upcoming 7.8.
Comment 4 Pavel Moravec 2019-12-20 13:10:51 UTC 
I asked via both the cases to confirm the expected duplicity of 1704957 - please get that data from customer and provide answer here.
Comment 6 Pavel Moravec 2019-12-27 08:20:24 UTC 
Thaks for confirmation of duplicate of 1704957.

*** This bug has been marked as a duplicate of bug 1704957 ***
Comment 7 Red Hat Bugzilla 2023-09-18 00:19:17 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days
Note You need to log in before you can comment on or make changes to this bug.