Red Hat Bugzilla – Bug 1252571
daemon and/or listener logs disappear
Last modified: 2015-11-19 11:35:24 EST
After starting the daemon & listener, and then running a test, the log files /var/log/rhcert/RedHatCertificationListener.log and RedHatCertDaemon.log are no longer present.
no reproducible case yet, but occurs regularly in testing.
Steps to Reproduce:
1. Start the daemon with `rhcert-backend server start`
2. verify that the servers (daemon & listener) started. retry, if needed.
3. run any tests
4. check for the log files in /var/log/rhcert
/var/log/RedHatCertDaemon.log and /var/log/RedHatCertificationListener.log do not exist. Before the most recent test run, with no server restart, those log files existed.
persistent log files
seen while running solely the cd-rw test, during test execution (debug option not selected). LTS is the SUT (server is also client). The cd-rw test completed and passed. /var/log/rhcert/runs/ appears complete. There's no obvious effect from the web ui. /var/log/messages doesn't show anything related.
The daemon and listener are still running. /proc/$daemonpid/fd shows that stdout and stderr are pointed at "/var/log/rhcert/RedHatCertDaemon.log (deleted)" or "/var/log/rhcert/RedHatCertificationListener.log (deleted)"
dmesg has some non-fatal errors (the burn completed successfully) over sr0
I've previously (recently) seen similar behavior when running other tests, but have so far been unable to reproduce the failure at will.
Saw this on a fresh install on combined LTS / SUT system.
Triggered by attaching a host to a certification, immediately after registering that host to the LTS.
The directory /var/log/rhcert/runs is created after the log files disappear.
All of /var/log/rhcert disappears. The listener log seems to disappear first, but that could be an artifact of how I'm monitoring the logs. A tail watching either log file continues to log new information.
The message "Checking for continuation" is printed to RedHatCertDaemon.log at roughly the same time as the log files disappear. I'll paste the log in another message.
Created attachment 1061658 [details]
RedHatCertDaemon.log captured with tail
captured via tail. output includes output after the file is unlinked from /var/log/rhcert
(In reply to Brian Brock from comment #2)
> Saw this on a fresh install on combined LTS / SUT system.
> Triggered by attaching a host to a certification, immediately after
> registering that host to the LTS.
In this case, "clean" is called to remove any existing test results, including /var/log/rhcert. It should probably just remove /var/log/rhcert/runs (the test run logging directories) so that the process logs are preserved.
Could this also explain the Summary case? Did step 3) include associating the SUT with a certification?
The original case occurred when a fresh system started the daemon, stopped, and then started again. Associating the SUT with a cert happened (normally) without any problems.
Logs are no longer disappearing.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.