Bug 1067243

Summary: if auth config in labcontroller.conf is wrong, lab controller daemons start but fail to do anything useful
Product: [Retired] Beaker Reporter: Dan Callaghan <dcallagh>
Component: lab controllerAssignee: Dan Callaghan <dcallagh>
Status: CLOSED CURRENTRELEASE QA Contact: tools-bugs <tools-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.15CC: aigao, asaha, dcallagh, jingwang, llim, rmancy
Target Milestone: 0.16   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-03-17 03:01:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dan Callaghan 2014-02-20 01:40:37 UTC
Description of problem:
If the authentication settings in /etc/beaker/labcontroller.conf are wrong (for example mistyped password or wrong user acocunt) the lab controller daemons start successfully but don't work properly.

The root of the problem is that kobo does not raise an exception if it fails to authenticate to the hub.

Version-Release number of selected component (if applicable):
0.15.4

How reproducible:
always

Steps to Reproduce:
1. Edit /etc/beaker/labcontroller.conf to have wrong settings in AUTH_METHOD, USERNAME, and/or PASSWORD
2. Restart the daemon (bug applies to beaker-proxy, beaker-provision, beaker-watchdog, beaker-transfer)

Actual results:
The daemon starts up, logs a message about failing to authenticate, and then keeps running but fails to do anything useful. For example, beaker-watchdog will loop failing to fetch watchdogs:

kobo.client.HubProxy INFO Creating new session...
kobo.client.HubProxy DEBUG Failed to create new session: <Fault 1: <class 'bkr.server.authentication.LoginException'>:Invalid username or password>
bkr.labcontroller.watchdog ERROR Traceback (most recent call last):
   File "/usr/lib/python2.6/site-packages/bkr/labcontroller/watchdog.py", line 42, in main_loop
     expired_watchdogs = watchdog.hub.recipes.tasks.watchdogs ('expired')
   File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
     return self.__send(self.__name, args)
   File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
     verbose=self.__verbose
   File "/usr/lib/python2.6/site-packages/bkr/labcontroller/proxy.py", line 55, in request
     result = transport_class.request(self, *args, **kwargs)
   File "/usr/lib/python2.6/site-packages/kobo/xmlrpc.py", line 432, in _request
     return self._parse_response(h.getfile(), sock)
   File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
     return u.close()
   File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
     raise Fault(**self._stack[0])
 Fault: <Fault 1: <class 'bkr.common.bexceptions.BX'>:No lab controller passed in and not currently logged in>

Whereas beaker-proxy will start but then fail every request it receives.

Expected results:
Daemon should refuse to start with a suitable descriptive error message indicating what the problem is, so the admin knows that the configuration is wrong.

Additional info:
This problem has existed in Beaker for a very long time. I first tried to fix it back in 2011:

http://gerrit.beaker-project.org/824

but the fix was blocked by the Kobo issue noted above (filed as bug 753006, as yet unfixed). However now that we are no longer using Kobo we are in a position to fix this.

Comment 2 Dan Callaghan 2014-02-20 04:50:22 UTC
(In reply to Dan Callaghan from comment #0)
> However now that we are no longer using Kobo we are in a
> position to fix this.

Scratch that, I forgot that we are still using kobo-client which is where the hub login code is.

Comment 3 Dan Callaghan 2014-02-20 04:59:10 UTC
Ignore comment 2... we are *not* using kobo-client anymore and we *can* fix this in Beaker.

Comment 4 Dan Callaghan 2014-02-26 05:15:16 UTC
On Gerrit: http://gerrit.beaker-project.org/2849

Comment 7 Dan Callaghan 2014-03-05 02:51:37 UTC
(In reply to wangjing from comment #6)
> Actual results:
> others are same with the expected results, the failed part is:
> step 2.1/2.2 when running 'service beaker-transfer restart', return the
> error message for 'initializing *Watchdog*', rather than 'beaker-transfer':

That's an existing error message, not new in this bug. It says "Error initializing Watchdog" for beaker-transfer because it's talking about a Watchdog object, which is what beaker-transfer uses internally. Admittedly it looks a bit confusing, since it makes it look like it's an error from beaker-watchdog instead. But that's not really relevant to this bug -- the important part is that the authentication failure is actually reported, and the daemon exits with an error, so that the initscript reports "FAILED".

Comment 8 wangjing 2014-03-06 06:24:40 UTC
based on comment7, file bug 1073266 for tracking this issue.
so verified this bug.
thanks!

Comment 9 Dan Callaghan 2014-03-17 03:01:52 UTC
Beaker 0.16.0 has been released.