Bug 832328 - beaker-provision does not reliably shut down
Summary: beaker-provision does not reliably shut down
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Retired
Component: lab controller
Version: 0.9
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dan Callaghan
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-06-15 07:52 UTC by Dan Callaghan
Modified: 2019-05-22 13:42 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-06-26 06:41:23 UTC
Embargoed:


Attachments (Terms of Use)

Description Dan Callaghan 2012-06-15 07:52:47 UTC
If beaker-provision is sent SIGTERM (which is how the init script attempts to shut it down) while under heavy load, the signal handler could raise ShutdownException in the wrong greenlet:

2012-06-15 01:50:53 [ERROR   ] {19685} bkr.labcontroller.provision:85 Error processing command 42442
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/provision.py", line 78, in handle
    handle_configure_netboot(command)
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/provision.py", line 126, in handle_configure_netboot
    command['netboot']['initrd_url'], command['fqdn'])
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/netboot.py", line 153, in fetch_images
    siphon(urllib2.urlopen(initrd_url), dest)
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/netboot.py", line 45, in siphon
    dest.write(chunk)
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/provision.py", line 182, in shutdown_handler
    raise ShutdownException() # gevent prints this to stderr, just ignore it
ShutdownException

Normally it is expected to be raised in the main greenlet, where it is handled correctly. If it raises in the wrong greenlet then the daemon will never shut down cleanly.

The fix is to use an event to signal shutdown, rather than raising an exception.

Comment 1 Dan Callaghan 2012-06-15 07:55:56 UTC
On Gerrit: http://gerrit.beaker-project.org/1142

Comment 3 Dan Callaghan 2012-06-26 06:41:23 UTC
Beaker 0.9.0 has been released.


Note You need to log in before you can comment on or make changes to this bug.