Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 913616 (nova_debug_dump_5.0)

Summary: [RFE][nova]: Provide for a live debug dump of system state
Product: Red Hat OpenStack Reporter: Daniel Berrangé <berrange>
Component: openstack-novaAssignee: Solly Ross <sross>
Status: CLOSED UPSTREAM QA Contact: Omri Hochman <ohochman>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0CC: dallan, ndipanov, sgordon
Target Milestone: Upstream M3Keywords: FutureFeature, MoveUpstream, Triaged
Target Release: 5.0 (RHEL 7)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
: cinder_debug_dump_5.0 glance_debug_dump_5.0 keystone_debug_dump_5.0 neutron_debug_dump_5.0 swift_debug_dump_5.0 nova_debug_dump_4.0.z nova_debug_dump_3.0.z (view as bug list) Environment:
Last Closed: 2014-04-08 21:28:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1007591, 1008621, 1008625, 1008627, 1008630, 1071469, 1071470    

Description Daniel Berrangé 2013-02-21 15:55:43 UTC 
Description of problem:
Add the ability to obtain a debug dump of the system state for an OpenStack service based process.  eg upon sending SIGUSR1 trigger a dump of all native threads, green threads, live config and any other relevant info

See this upstream proposal

https://wiki.openstack.org/wiki/GuruMeditationReport

(Filed against Nova, but applicable to all)
Comment 3 Solly Ross 2013-06-06 20:30:37 UTC 
@Daniel Berrange: were you thinking something like this: http://pastebin.test.redhat.com/146078 (`git apply` to nova)

(apply this: http://pastebin.test.redhat.com/146081 to python-novaclient to easily access without having to manually curl TENANT_ID/guru)
Comment 4 Daniel Berrangé 2013-06-07 09:50:18 UTC 
No, this is not at all what I was considering for design. Integration into the RPC service, nova client, etc is absolutely not something that is intended. This needs to be a capability that is ultra reliable in the face of broken arbitrarily OpenStack infrastructure. As such it must not depends on things like the RPC system, database, or REST interfaces.

I wrote a proof of concept here

  https://github.com/berrange/nova/commits/gurumed2

however, I did not submit this upstream because I was intending to generalize it into a python package that is broadly independent of openstack code. Further alot of the same infrastructure is required for the related system tracing work in https://bugzilla.redhat.com/show_bug.cgi?id=913619
Comment 5 Solly Ross 2013-06-07 14:50:59 UTC 
Oh, OK.  That wasn't exactly clear from the description and wiki page.  I'll take a look at the proof of concept and see if I can generalize it/integrate it into some of the framework that I wrote.
Comment 6 Solly Ross 2013-06-10 18:46:40 UTC 
Patch submitted upstream to oslo-config to integrate oslo.report into Oslo
Comment 7 Solly Ross 2013-09-12 20:19:31 UTC 
patch to Oslo merged upstream under 'openstack.common.report'.  Now to push functionality into nova...
Comment 8 Solly Ross 2014-01-24 23:08:51 UTC 
Patch for integrating functionality into nova posted upstream.
Comment 9 Solly Ross 2014-04-08 21:28:38 UTC 
Since this is targeting 5.0 and the change made it into Icehouse, I'm closing this "UPSTREAM"