Bug 821319

Summary:	RHUI Diagnostics Tooling: rhui-debug script for gathering debug data (similar to katello-debug/spacewalk-debug)
Product:	Red Hat Update Infrastructure for Cloud Providers	Reporter:	Xixi <xdmoon>
Component:	Tools	Assignee:	James Slagle <jslagle>
Status:	CLOSED ERRATA	QA Contact:	Martin Kočí <mkoci>
Severity:	medium	Docs Contact:
Priority:	high
Version:	2.1	CC:	bmr, jslagle, juwu, kbidarka, mkoci, sghai, tsanders, vkuznets, xdmoon
Target Milestone:	---	Keywords:	Reopened
Target Release:	2.1.1
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	A rhui-debug script was requested for gathering debug data. rhui-debug.py is now installed on Red Hat Update Appliance (RHUA) at rhui-debug.py. Run 'python /usr/share/rh-rhua/rhui-debug.py' to collect debug data; pathways to the collected data will also be returned.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-02-27 16:59:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Xixi 2012-05-14 06:00:35 UTC

Description of problem:
Debug data-gathering script for RHUI similar to katello-debug/spacewalk-debug, which gathers config and log files and other information that includes everything needed for debugging in most cases.  

This will also integrate with sosreport who'll call the script to include the data in sosreport tar.

Version-Release number of selected component (if applicable):
current/newest

How reproducible:
See above.

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 3 James Slagle 2012-06-11 17:00:20 UTC

Sayli, reassigning this one to you.  Let me know if you need any assistance.

Comment 4 James Slagle 2012-07-31 18:04:56 UTC

rhui-debug.py installed at /usr/share/rh-rhua/rhui-debug.py on the RHUA

Comment 5 James Slagle 2012-08-01 12:11:02 UTC

Verify that the script is installed at the right location and that when you run 'python /usr/share/rh-rhua/rhui-debug.py' it does a data collection and outputs the path to the collected files.

Comment 6 Martin Kočí 2012-08-03 15:30:15 UTC

Verified existence of the script /usr/share/rh-rhua/rhui-debug.py as well as random debug files are readable etc..further info see:

[root@ip-10-110-241-123 ~]# ll /usr/share/rh-rhua/rhui-debug.py
-rw-r--r--. 1 root root 6008 Aug  1 13:44 /usr/share/rh-rhua/rhui-debug.py
[root@ip-10-110-241-123 ~]# python /usr/share/rh-rhua/rhui-debug.py
Successfully created directory [/tmp/rhui-debug-2012-08-03-T11-26-40]
Copying contents of directory [/root/.rhui/] to [/tmp/rhui-debug-2012-08-03-T11-26-40/root/.rhui/] 
Copying file [/etc/rhui/rhui-tools.conf] to [/tmp/rhui-debug-2012-08-03-T11-26-40/etc/rhui]
Copying contents of directory [/etc/pulp/] to [/tmp/rhui-debug-2012-08-03-T11-26-40/etc/pulp/] 
Copying contents of directory [/etc/gofer/plugins/] to [/tmp/rhui-debug-2012-08-03-T11-26-40/etc/gofer/plugins/] 
Copying contents of directory [/var/log/pulp/] to [/tmp/rhui-debug-2012-08-03-T11-26-40/var/log/pulp/] 
Copying contents of directory [/var/log/httpd/] to [/tmp/rhui-debug-2012-08-03-T11-26-40/var/log/httpd/] 
Copying file [/etc/httpd/conf.d/pulp.conf] to [/tmp/rhui-debug-2012-08-03-T11-26-40/etc/httpd/conf.d]
Copying file [/etc/yum/pluginconf.d/pulp-profile-update.conf] to [/tmp/rhui-debug-2012-08-03-T11-26-40/etc/yum/pluginconf.d]
Copying file [/etc/pki/pulp/content/pulp-protected-repos] to [/tmp/rhui-debug-2012-08-03-T11-26-40/etc/pki/pulp/content]
Creating commands directory [/tmp/rhui-debug-2012-08-03-T11-26-40/commands]
Storing output of command [/tmp/rhui-debug-2012-08-03-T11-26-40/commands] in [/tmp/rhui-debug-2012-08-03-T11-26-40/commands/ls_-lR_var.lib.pulp]

Checked against /root/RHEL-6.3-RHUI-2.1-20120801.0-Server-x86_64-DVD1.iso
Moving bug to VERIFIED.

Comment 7 Julie 2012-08-15 17:37:08 UTC

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
For Red Hat Update Infrastructure (RHUI), a rhui-debug script was requested for gathering debug data. rhui-debug.py is now installed on Red Hat Update Appliance (RHUA) at /usr/share/rh-rhua/rhui-debug.py. Run 'python /usr/share/rh-rhua/rhui-debug.py' to collect debug data; pathways to the collected data will also be returned.

Comment 9 errata-xmlrpc 2012-08-24 11:54:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-1205.html

Comment 10 Bryn M. Reeves 2012-10-17 11:31:32 UTC

Would it be possible to change this in a future release to take an output path on the command line?

This is how all the other *-debug scripts we run from sos operate and it avoids the overhead of double-copying and cleaning up (we try to avoid unlinking things outside our own directories).

I also note that the script is pulling in the whole of /var/log/http - that directory is already handled by sos and can grow very large under some configs. We have existing size-limiting code to handle this in our httpd module. With the current script we will end up including this twice (or will need a hack in sos to delete it..).

Comment 18 James Slagle 2012-11-29 21:38:00 UTC

I've packaged the script into a seperate package, and will add a step to our install_CDS.sh script to make sure it gets installed on CDS instances.

commit 87b844ec092177bd8303d523e44324b8ebfdf1d2

Comment 19 Vitaly Kuznetsov 2013-02-04 13:12:04 UTC

[root@rhua ~]# python /usr/share/rh-rhua/rhui-debug.py
Successfully created directory [/tmp/rhui-debug-2013-02-04-T08-04-36]
Copying contents of directory [/root/.rhui/] to [/tmp/rhui-debug-2013-02-04-T08-04-36/root/.rhui/] 
Copying file [/etc/rhui/rhui-tools.conf] to [/tmp/rhui-debug-2013-02-04-T08-04-36/etc/rhui]
Copying contents of directory [/etc/pulp/] to [/tmp/rhui-debug-2013-02-04-T08-04-36/etc/pulp/] 
Copying contents of directory [/etc/gofer/plugins/] to [/tmp/rhui-debug-2013-02-04-T08-04-36/etc/gofer/plugins/] 
Copying contents of directory [/var/log/pulp/] to [/tmp/rhui-debug-2013-02-04-T08-04-36/var/log/pulp/] 
Copying file [/etc/httpd/conf.d/pulp.conf] to [/tmp/rhui-debug-2013-02-04-T08-04-36/etc/httpd/conf.d]
Copying file [/etc/yum/pluginconf.d/pulp-profile-update.conf] to [/tmp/rhui-debug-2013-02-04-T08-04-36/etc/yum/pluginconf.d]
Copying file [/etc/pki/pulp/content/pulp-protected-repos] to [/tmp/rhui-debug-2013-02-04-T08-04-36/etc/pki/pulp/content]
Creating commands directory [/tmp/rhui-debug-2013-02-04-T08-04-36/commands]
Storing output of command [/tmp/rhui-debug-2013-02-04-T08-04-36/commands] in [/tmp/rhui-debug-2013-02-04-T08-04-36/commands/ls_-lR_var.lib.pulp]
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/root/.rhui/ && echo SUCCESS
rhua.example.com
rhui.log
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/etc/pulp/ && echo SUCCESS
admin
cds.conf
consumer
logging
pulp.conf
repo_auth.conf
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/etc/gofer/plugins/ && echo SUCCESS
builtin.conf
consumer.conf
package.conf
pulpplugin.conf
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/var/log/pulp/ && echo SUCCESS
client.log
db.log
events.log
grinder.log
pulp.log
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/etc/rhui/rhui-tools.conf && echo SUCCESS
/tmp/rhui-debug-2013-02-04-T08-04-36/etc/rhui/rhui-tools.conf
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/etc/httpd/conf.d/pulp.conf && echo SUCCESS
/tmp/rhui-debug-2013-02-04-T08-04-36/etc/httpd/conf.d/pulp.conf
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/etc/yum/pluginconf.d/pulp-profile-update.conf && echo SUCCESS
/tmp/rhui-debug-2013-02-04-T08-04-36/etc/yum/pluginconf.d/pulp-profile-update.conf
SUCCESS
[root@rhua ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-36/etc/pki/pulp/content/pulp-protected-repos && echo SUCCESS
/tmp/rhui-debug-2013-02-04-T08-04-36/etc/pki/pulp/content/pulp-protected-repos
SUCCESS
[root@rhua ~]# ls -lR /var/lib/pulp | diff -u /tmp/rhui-debug-2013-02-04-T08-04-36/commands/ls_-lR_var.lib.pulp - && echo SUCCESS
SUCCESS
[root@rhua ~]# Last login: Mon Feb  4 08:04:15 2013 from 10.48.162.207
[root@cds1 ~]# python /usr/share/rh-rhua/rhui-debug.py --cds
Successfully created directory [/tmp/rhui-debug-2013-02-04-T08-04-45]
Copying contents of directory [/etc/pulp/] to [/tmp/rhui-debug-2013-02-04-T08-04-45/etc/pulp/] 
Copying contents of directory [/etc/gofer/plugins/] to [/tmp/rhui-debug-2013-02-04-T08-04-45/etc/gofer/plugins/] 
Copying contents of directory [/var/log/pulp-cds/] to [/tmp/rhui-debug-2013-02-04-T08-04-45/var/log/pulp-cds/] 
Copying file [/etc/httpd/conf.d/pulp-cds.conf] to [/tmp/rhui-debug-2013-02-04-T08-04-45/etc/httpd/conf.d]
Copying file [/var/log/gofer/agent.log] to [/tmp/rhui-debug-2013-02-04-T08-04-45/var/log/gofer]
Creating commands directory [/tmp/rhui-debug-2013-02-04-T08-04-45/commands]
Storing output of command [/tmp/rhui-debug-2013-02-04-T08-04-45/commands] in [/tmp/rhui-debug-2013-02-04-T08-04-45/commands/ls_-lR_var.lib.pulp-cds]
[root@cds1 ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-45/etc/pulp/ && echo SUCCESS
cds.conf
repo_auth.conf
SUCCESS
[root@cds1 ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-45/etc/gofer/plugins/ && echo SUCCESS
builtin.conf
cdsplugin.conf
package.conf
SUCCESS
[root@cds1 ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-45/var/log/pulp-cds/ && echo SUCCESS
gofer.log
SUCCESS
[root@cds1 ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-45/etc/httpd/conf.d/pulp-cds.conf && echo SUCCESS
/tmp/rhui-debug-2013-02-04-T08-04-45/etc/httpd/conf.d/pulp-cds.conf
SUCCESS
[root@cds1 ~]# ls -1 /tmp/rhui-debug-2013-02-04-T08-04-45/var/log/gofer/agent.log && echo SUCCESS
/tmp/rhui-debug-2013-02-04-T08-04-45/var/log/gofer/agent.log
SUCCESS
[root@cds1 ~]# ls -lR /var/lib/pulp-cds | diff -u /tmp/rhui-debug-2013-02-04-T08-04-45/commands/ls_-lR_var.lib.pulp-cds - && echo SUCCESS
SUCCESS

Comment 21 errata-xmlrpc 2013-02-27 16:59:08 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0571.html