Bug 1292278 - goferd stops logging when executing a task and qdrouterd restarted
goferd stops logging when executing a task and qdrouterd restarted
Product: Red Hat Satellite 6
Classification: Red Hat
Component: katello-agent (Show other bugs)
x86_64 Linux
high Severity medium (vote)
: Unspecified
: --
Assigned To: satellite6-bugs
Katello QA List
: Triaged
Depends On:
  Show dependency treegraph
Reported: 2015-12-16 16:41 EST by Pavel Moravec
Modified: 2017-08-01 16:02 EDT (History)
7 users (show)

See Also:
Fixed In Version: qpid-proton-0.9-12
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-08-01 16:02:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Pavel Moravec 2015-12-16 16:41:13 EST
Description of problem:
When goferd is executing a task (i.e. package install, capsule sync), if one restarts qdrouterd where the goferd is connected to, goferd stops logging. Completely. It stops logging (dis)connections, gofer.rmi.dispatcher logs, everything.

This is quite serious problem for troubleshooting.

Version-Release number of selected component (if applicable):
Sat 6.1.5

How reproducible:

Steps to Reproduce:
0. on the client machine:
tail -f /var/log/messages

1. on Satellite:
hammer -u admin -p PASSWORD content-host package install --content-host-id UUID --organization-id 1 --packages sos

2. On the client machine, wait in the /var/log/messages until:
"gofer.rmi.dispatcher:600 - call: Content.install() .."

log appears.
3. _Before_ logging "gofer.agent.rmi:129 - .. processed in:", restart qdrouterd on Satellite/Capsule - where the goferd is connected to.

4. Do whatever action to be logged by goferd. Install/remove a package, stop qdrouterd,..

Actual results:
neither action in 4. is logged by goferd

Expected results:
goferd continues in logging

Additional info:
Comment 2 Pavel Moravec 2015-12-16 16:51:12 EST
To little bit clarify:

Step 4 can be followed any time (but let be safe, wait 20 seconds to ensure goferd is reconnected). The latest log goferd prints out is the

"gofer.rmi.dispatcher:600 - call: Content.install() .."

I.e. it does not even log the Disconnected event from step 3 (!)
Comment 3 Jeff Ortel 2016-01-26 10:33:15 EST
I suspect the root cause is https://issues.apache.org/jira/browse/PROTON-1090.  The resolution has been requested to be back ported to proton 0.9 for satellite.  I will try this with proton 0.12.
Comment 5 Bryan Kearney 2016-07-26 11:25:24 EDT
Moving 6.2 bugs out to sat-backlog.
Comment 6 Bryan Kearney 2016-07-26 11:37:54 EDT
Moving 6.2 bugs out to sat-backlog.
Comment 8 Chris Duryee 2016-09-26 12:53:03 EDT
From 'rpm -q --changelog qpid-proton-c-0.9-16.el7.x86_64' on recent 6.2 machine:

* Thu Jan 21 2016 Mike Cressman <mcressman@redhat.com> - 0.9-12
- Added PROTON-1090 (bz1295957)

per #3, marking as on_qa
Comment 9 Peter Ondrejka 2016-11-29 09:30:20 EST
Using steps from problem description I'm still able to reproduce this on Satellite 6.3 snap 6. After restarting qdrouterd on Sat server, goferd stops logging to /var/log/messages on host. 

There is one exception, when trying to remove a package that is not present on a host, on server:

~]# hammer host package remove --host-id 34 --packages cat

creates the following log entry:

Nov 29 14:21:06 carl-modic goferd: No Match for argument: cat

Not sure why this exception occurs.
Comment 10 Brad Buckingham 2017-05-31 16:45:50 EDT
Can we get a re-test on latest 6.3. snap?
Comment 11 Bryan Kearney 2017-05-31 16:46:26 EDT
Moving this to POST since the builds contain qpid-proton-c-0.9-16.el7.x86_64
Comment 12 Bryan Kearney 2017-08-01 16:02:09 EDT
The fix to this bug will be delivered with release 6.3 of Satellite.

Note You need to log in before you can comment on or make changes to this bug.