Bug 894159

Summary: rhts-sync-block spamming console with rhts-sync-block: <Fault 8002: 'error'> messages
Product: [Retired] Beaker Reporter: Jan Stancek <jstancek>
Component: beahAssignee: Dan Callaghan <dcallagh>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.10CC: aigao, asaha, bpeck, dcallagh, jburke, llim, mcsontos, pbunyan, qwan, rmancy, xjia
Target Milestone: 0.14.4   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: Misc
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1032887 (view as bug list) Environment:
Last Closed: 2013-12-19 05:09:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1032887    

Description Jan Stancek 2013-01-10 21:16:11 UTC
Description of problem:
In multihost testing, server side does:
    rhts-sync-set -s READY
    rhts-sync-block -s DONE $CLIENTS

and for the whole test run rhts generates following messages on console and in logs: 

(RHEL7)
rhts-sync-block: <Fault 8002: 'error'> 
2013-01-10 15:57:11,250 rhts_task.twisted emit: ERROR Unhandled Error 
Traceback (most recent call last): 
Failure: twisted.python.failure.DefaultException: Timeout
...

(RHEL6)
rhts-sync-block: <Fault 8002: 'error'>
...

Version-Release number of selected component (if applicable):
beah-0.6.42-2.el7.noarch
rhts-test-env-4.53-1.el7.noarch
rhts-python-4.53-1.el7.noarch

How reproducible:
100%

Steps to Reproduce:
1. run multihost test, for example connectathon-mh

Actual results:
error messages

Expected results:
no error messages

Additional info:

Comment 4 Dan Callaghan 2013-09-30 01:28:36 UTC
The spam appeared because we stopped suppressing stderr from rhts-sync-block, because that was making it impossible to diagnose/debug real failures.

The error here just means "state not yet reached" so we can probably do a more targetted suppression of that message in particular, without suppressing all errors. It won't be pretty though...

Comment 6 Nick Coghlan 2013-11-06 07:15:12 UTC
*** Bug 692535 has been marked as a duplicate of this bug. ***

Comment 7 Nick Coghlan 2013-11-11 06:08:26 UTC
Fixing this (and testing that fix) turned out to be more complicated than expected - it has been moved back to targeting 0.15.2.

Comment 8 Nick Coghlan 2013-11-12 06:10:12 UTC
I started on a patch that changes the beah XML-RPC API to allow rhts to pass the timeout through to the beah XML-RPC server rather than having to poll locally:

Draft rhts change: http://gerrit.beaker-project.org/#/c/2455/
Draft beah XML-RPC API change: http://gerrit.beaker-project.org/#/c/2454/

Comment 9 Dan Callaghan 2013-11-13 04:46:22 UTC
(In reply to Nick Coghlan from comment #8)
> I started on a patch that changes the beah XML-RPC API to allow rhts to pass
> the timeout through to the beah XML-RPC server rather than having to poll
> locally:
> 
> Draft rhts change: http://gerrit.beaker-project.org/#/c/2455/
> Draft beah XML-RPC API change: http://gerrit.beaker-project.org/#/c/2454/

There are a few additional complexities to this approach (moving the timeout to beah's fake RHTS XMLRPC server) which means it may not be worthwhile.

This patch just suppresses all XMLRPC faults in rhts-sync-block, under the assumption that they are timeouts:

http://gerrit.beaker-project.org/2485

but this still leaves the "Timeout" errors being spewed on the RHTS side, like this:

Traceback (most recent call last): 
Failure: twisted.python.failure.DefaultException: Timeout

Comment 12 Nick Coghlan 2013-11-22 03:41:09 UTC
The cloned bug 1032887 covers the fact that the beah log files still get spammed with spurious errors even after this fix, but the change at least eliminates the noise from the console log.

Comment 13 xjia 2013-11-22 05:45:35 UTC
According comment10, 11 and 12. 
Verify this bug.