Bug 1252571 - daemon and/or listener logs disappear
daemon and/or listener logs disappear
Product: Red Hat Certification Program
Classification: Red Hat
Component: redhat-certification (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Greg Nichols
Brian Brock
Depends On:
  Show dependency treegraph
Reported: 2015-08-11 13:56 EDT by Brian Brock
Modified: 2015-11-19 11:35 EST (History)
1 user (show)

See Also:
Fixed In Version: redhat-certification-1.0-20150813
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-11-19 11:35:24 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
RedHatCertDaemon.log captured with tail (9.02 KB, text/plain)
2015-08-11 15:27 EDT, Brian Brock
no flags Details

  None (edit)
Description Brian Brock 2015-08-11 13:56:51 EDT
After starting the daemon & listener, and then running a test, the log files /var/log/rhcert/RedHatCertificationListener.log and RedHatCertDaemon.log are no longer present.


How reproducible:
no reproducible case yet, but occurs regularly in testing.

Steps to Reproduce:
1. Start the daemon with `rhcert-backend server start`
2. verify that the servers (daemon & listener) started.  retry, if needed.
3. run any tests
4. check for the log files in /var/log/rhcert

Actual results:
/var/log/RedHatCertDaemon.log and /var/log/RedHatCertificationListener.log do not exist.  Before the most recent test run, with no server restart, those log files existed.

Expected results:
persistent log files

Additional info:
seen while running solely the cd-rw test, during test execution (debug option not selected).  LTS is the SUT (server is also client).  The cd-rw test completed and passed.  /var/log/rhcert/runs/ appears complete.  There's no obvious effect from the web ui.  /var/log/messages doesn't show anything related.

The daemon and listener are still running. /proc/$daemonpid/fd shows that stdout and stderr are pointed at "/var/log/rhcert/RedHatCertDaemon.log (deleted)" or "/var/log/rhcert/RedHatCertificationListener.log (deleted)"

dmesg has some non-fatal errors (the burn completed successfully) over sr0
Comment 1 Brian Brock 2015-08-11 13:59:29 EDT
I've previously (recently) seen similar behavior when running other tests, but have so far been unable to reproduce the failure at will.
Comment 2 Brian Brock 2015-08-11 15:21:38 EDT
Saw this on a fresh install on combined LTS / SUT system.

Triggered by attaching a host to a certification, immediately after registering that host to the LTS.

The directory /var/log/rhcert/runs is created after the log files disappear.

All of /var/log/rhcert disappears.  The listener log seems to disappear first, but that could be an artifact of how I'm monitoring the logs.  A tail watching either log file continues to log new information.

The message "Checking for continuation" is printed to RedHatCertDaemon.log at roughly the same time as the log files disappear.  I'll paste the log in another message.
Comment 3 Brian Brock 2015-08-11 15:27:45 EDT
Created attachment 1061658 [details]
RedHatCertDaemon.log captured with tail

captured via tail.  output includes output after the file is unlinked from /var/log/rhcert
Comment 4 Greg Nichols 2015-08-11 16:59:46 EDT
(In reply to Brian Brock from comment #2)
> Saw this on a fresh install on combined LTS / SUT system.
> Triggered by attaching a host to a certification, immediately after
> registering that host to the LTS.

In this case, "clean" is called to remove any existing test results, including /var/log/rhcert.   It should probably just remove /var/log/rhcert/runs (the test run logging directories) so that the process logs are preserved.

Could this also explain the Summary case?  Did step 3) include associating the SUT with a certification?
Comment 7 Brian Brock 2015-09-16 17:18:25 EDT
The original case occurred when a fresh system started the daemon, stopped, and then started again.  Associating the SUT with a cert happened (normally) without any problems.
Comment 8 Brian Brock 2015-10-01 00:59:57 EDT
verified in:

Logs are no longer disappearing.
Comment 10 errata-xmlrpc 2015-11-19 11:35:24 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.