Bug 486480 - [RFE] Master should send obituary from .old logs if necessary
Summary: [RFE] Master should send obituary from .old logs if necessary
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: condor
Version: 1.1
Hardware: All
OS: Linux
low
medium
Target Milestone: 2.3
: ---
Assignee: Timothy St. Clair
QA Contact: Lubos Trilety
URL:
Whiteboard: FutureFeature
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-02-19 22:47 UTC by Matthew Farrellee
Modified: 2013-03-06 18:38 UTC (History)
5 users (show)

Fixed In Version: condor-7.8.2-0.1
Doc Type: Enhancement
Doc Text:
C: When the condor master daemon sends an obituary during a log rollover event of the failed daemon. C: The obituary will not be sent. F: Update logic to check for rollover log, and send. R: The master should send an obituary email when a daemon fails during a log rollover.
Clone Of:
Environment:
Last Closed: 2013-03-06 18:38:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 486462 0 low CLOSED query workers during log rotation crash schedd and collector 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHSA-2013:0564 0 normal SHIPPED_LIVE Low: Red Hat Enterprise MRG Grid 2.3 security update 2013-03-06 23:37:09 UTC

Description Matthew Farrellee 2009-02-19 22:47:54 UTC
condor-7.2.2-0.1.el5 and related to BZ486462

From /var/log/condor/MasterLog:

2/19 11:00:08 Sending obituary for "/usr/sbin/condor_schedd"
2/19 11:00:08 Forking Mailer process...
2/19 11:00:08 Failed to email /var/log/condor/SchedLog: cannot open file

The Schedd failed in the middle of a log rotation. The Master was not able to email an obituary because the new SchedLog had not been created. In such a case the Master should attempt to mail part of the SchedLog.old instead.

Comment 1 Timothy St. Clair 2011-05-18 18:47:46 UTC
Is there an easy repro condition?

Comment 2 Matthew Farrellee 2011-05-18 18:58:38 UTC
I would expect setting MAX_SCHEDD_LOG to a small number then sending SIGKILL to the condor_schedd would assist in reproducing.

Comment 3 Luigi Toscano 2012-03-07 19:51:46 UTC
(In reply to comment #2)
> I would expect setting MAX_SCHEDD_LOG to a small number then sending SIGKILL to
> the condor_schedd would assist in reproducing.

Is this still the suggested way to reproduce it?

Comment 4 Timothy St. Clair 2012-03-14 16:48:29 UTC
-------------------------------------------------------
To repro: 

1.) Navigate to your LOG locations && `rm -f SchedLog*`

2.) set SCHEDD = /some/path/to/a/script/like/the/old/below in your config 

3.) drop a script which ~= the one below

#!/bin/sh 
echo "MASSIVE FAIL OBIT TEST" >> /your/LOG/loc/SchedLog.old
exit 1

4.) Start condor.

before fix you'll see a fail to open like in comment #1

after fix you'll see it fork the daemon and email.
-------------------------------------------------------

Tracking changes upstream.

Comment 6 Timothy St. Clair 2012-03-19 16:23:48 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
C: When the condor master daemon sends an obituary during a log rollover event of the failed daemon. 
C: The obituary will not be sent.
F: Update logic to check for rollover log, and send.
R: The master should send an obituary email when a daemon fails during a log rollover.

Comment 10 Martin Kudlej 2013-02-06 09:00:40 UTC
Tested on RHEL 5.9/6.4 x i386/x86_64 with condor-7.8.8-0.4.1 and it works. -->VERIFIED

Comment 12 errata-xmlrpc 2013-03-06 18:38:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0564.html


Note You need to log in before you can comment on or make changes to this bug.