Bug 1051091 - retrace-server-cleanup job crashes if retrace_log does not exist
Summary: retrace-server-cleanup job crashes if retrace_log does not exist
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: retrace-server
Version: el6
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
Assignee: Michal Toman
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-09 16:48 UTC by Dave Wysochanski
Modified: 2015-03-23 00:42 UTC (History)
4 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2014-08-15 18:58:09 UTC


Attachments (Terms of Use)

Description Dave Wysochanski 2014-01-09 16:48:31 UTC
Description of problem:

If retrace_log is not created, the cleanup job crashes and does not run properly.  This affects cleaning up all other jobs since the cleanup task crashes if any task directory does not contain retrace_log.

Traceback (most recent call last):
  File "/usr/bin/retrace-server-cleanup", line 71, in <module>
    task.set_log("Killed by garbage collector\n", True)
  File "/usr/lib/python2.6/site-packages/retrace/retrace.py", line 1431, in set_log
    self.set_atomic(RetraceTask.LOG_FILE, log, mode=mode)
  File "/usr/lib/python2.6/site-packages/retrace/retrace.py", line 1334, in set_atomic
    shutil.copyfile(filename, tmpfilename)
  File "/usr/lib64/python2.6/shutil.py", line 50, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/cores/retrace/tasks/396908399/retrace_log'
Traceback (most recent call last):
  File "/usr/bin/retrace-server-cleanup", line 71, in <module>
    task.set_log("Killed by garbage collector\n", True)
  File "/usr/lib/python2.6/site-packages/retrace/retrace.py", line 1431, in set_log
    self.set_atomic(RetraceTask.LOG_FILE, log, mode=mode)
  File "/usr/lib/python2.6/site-packages/retrace/retrace.py", line 1334, in set_atomic
    shutil.copyfile(filename, tmpfilename)
  File "/usr/lib64/python2.6/shutil.py", line 50, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/cores/retrace/tasks/396908399/retrace_log'
Traceback (most recent call last):
/var/log/retrace-server/cleanup_error.log


Version-Release number of selected component (if applicable):
retrace-server-1.10-1.el6.noarch


How reproducible:
Every time if any directory does not contain retrace_log


Steps to Reproduce:
1. Some task does not complete properly such that retrace_log is not created
2.
3.

Actual results:
retrace cleanup job stops running and no tasks are cleaned up after encountering the task which does not have retrace_log

Expected results:
cleanup job should continue to run even if some directory does not contain this file

Additional info:
I opened a ticket for this and the admins manually moved / removed the tasks missing retrace_log, but we need a longer term fix as this may happen again.

Comment 1 Michal Toman 2014-02-26 09:15:10 UTC
Fixed in upstream

commit 2771d17190129f2e85b7613809eeaa52a2d23d78
Author: Michal Toman <mtoman@redhat.com>
Date:   Tue Feb 25 17:02:42 2014 +0100

    set_atomic: allow to append to non-existing file
    
    Signed-off-by: Michal Toman <mtoman@redhat.com>

Comment 2 Fedora Update System 2014-02-27 13:46:55 UTC
retrace-server-1.11-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/retrace-server-1.11-1.el6

Comment 3 Fedora Update System 2014-03-01 07:11:54 UTC
Package retrace-server-1.11-1.el6:
* should fix your issue,
* was pushed to the Fedora EPEL 6 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing retrace-server-1.11-1.el6'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2014-0687/retrace-server-1.11-1.el6
then log in and leave karma (feedback).

Comment 5 Dave Wysochanski 2014-03-07 21:16:39 UTC
Brian Rhatigan tested this one and we think it is fixed.

Comment 6 Fedora Update System 2014-07-31 11:52:50 UTC
retrace-server-1.12-2.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/retrace-server-1.12-2.el6

Comment 7 Fedora Update System 2014-08-15 18:58:09 UTC
retrace-server-1.12-2.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.