Bug 739219 - Aviary does not handle job output filenames that do not contain explicit paths
Summary: Aviary does not handle job output filenames that do not contain explicit paths
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: condor-aviary
Version: Development
Hardware: All
OS: Linux
low
medium
Target Milestone: 2.3
: ---
Assignee: Pete MacKinnon
QA Contact: Martin Kudlej
URL:
Whiteboard:
Depends On: 731065
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-09-16 20:48 UTC by Trevor McKay
Modified: 2013-03-06 18:39 UTC (History)
6 users (show)

Fixed In Version: condor-7.8.2-0.1
Doc Type: Bug Fix
Doc Text:
Cause: Query for a job output file using the relative filename which may have been defined in a submit file. Consequence: File could not be found by Aviary getJobData operation. Fix: Implementation will try the supplied name first and if that fails it will also try prepending the IWD value to the first value and retry a stat of the file. Result: Job output files that have been submitted with relative paths can be implicitly resolved in getJobData operation.
Clone Of: 731065
Environment:
Last Closed: 2013-03-06 18:39:04 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2013:0564 0 normal SHIPPED_LIVE Low: Red Hat Enterprise MRG Grid 2.3 security update 2013-03-06 23:37:09 UTC

Description Trevor McKay 2011-09-16 20:48:28 UTC
+++ This bug was initially created as a clone of Bug #731065 +++

Description of problem:

It is possible to submit jobs to condor with or without explicit paths in the names of error, output, and log files.  If filenames do not contain explicit paths, the files are created in the working directory.  However, the Cumin job output screen does not correctly handle filenames without explicit paths.

Version-Release number of selected component (if applicable):


How reproducible:

100%

Steps to Reproduce:
1.  Submit a job using condor_submit or Cumin with error, output and log filenames devoid of path information.
2.  Drill down into the job and go to the "Output" tab
3.  Try to view the files with the output, error, and userlog radio buttons
  
Actual results:

Cumin will display "Failed to open <filename>"  (this is the result returned from QMF.

Expected results:

Cumin should handle default behavior for files that do not begin with absolute paths.

Additional info:

Note, condor_submit will actually expand the path information for the log file correctly but will not expand output or error.  So cumin will actually be able to display the log file for jobs from condor_submit.

--- Additional comment from tmckay on 2011-08-16 12:01:07 EDT ---

Tested a solution.  When Cumin attempts to fetch output file contents based on a job classad, prepend the value of the Iwd classad attribute to the output file name if the output file name does not already begin with a "/".

--- Additional comment from tmckay on 2011-08-16 12:27:47 EDT ---

Fixed in revision 4933.

--- Additional comment from tmckay on 2011-08-30 11:41:22 EDT ---

Needs more work.  The above solution works with QMF, but Aviary does not take a path argument when retrieving an output file.  Rather, it takes an enumeration value indicating the type of the file and retrieves the file path from the job ad.

So, for this to work with Aviary, cumin needs to prepend the Iwd value to the output file arguments before submission if the fields do not contain paths.

--- Additional comment from tmckay on 2011-09-16 16:24:13 EDT ---

Fixed in revision 4981.  Should work for both QMF and Aviary interfaces.

Note, related to BZ#739205 when working with Aviary.

--- Additional comment from tmckay on 2011-09-16 16:32:57 EDT ---

Note, there may still be an additional hole here.  If a job is submitted from the command line without explicit paths on files, and the job is read from Cumin through Aviary, the path information will not be there.

I'll verify, this might lead to a BZ against Aviary since there would be nothing that cumin could do in this case.

--- Additional comment from tmckay on 2011-09-16 16:47:01 EDT ---

Yes, the above comment is true.  IMHO, either Aviary has to use the working directory to extend the filenames in the case of no path, or the call needs to take a path argument instead of or in addition to the enumeration value.

I think this should be duped and opened against Aviary, cumin has been fixed as much as possible in this case.

Here's an example submission file that will illustrate this:

executable = /bin/sleep
args = 3d
requirements = 1
universe = vanilla
RequestMemory = 1
RequestDisk = 1
DiskUsage = 1
ImageSize = 1
should_transfer_files = Never
concurrency_limits = sleep
initialdir = /home/tmckay
output = test_submit.output
error = test_submit.error
log = test_submit.log
queue 1

Comment 2 Pete MacKinnon 2011-10-11 18:50:19 UTC
Trevor & I will work out something in kind for this.

Comment 4 Pete MacKinnon 2012-05-04 14:47:52 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause: Query for a job output file using the relative filename which may have been defined in a submit file.
Consequence: File could not be found by Aviary getJobData operation.
Fix: Implementation will try the supplied name first and if that fails it will also try prepending the IWD value to the first value and retry a stat of the file.
Result: Job output files that have been submitted with relative paths can be implicitly resolved in getJobData operation.

Comment 7 Martin Kudlej 2013-01-11 12:47:08 UTC
Tested on RHEL 5.9/6.4 x i386/x86_64 with
condor-7.8.8-0.3
condor-aviary-7.8.8-0.3
condor-classads-7.8.8-0.3
condor-wallaby-base-db-1.25-1
condor-wallaby-client-5.0.5-1
condor-wallaby-tools-5.0.5-1
python-condorutils-1.5-6
python-qpid-0.18-4
python-qpid-qmf-0.18-13
python-wallabyclient-5.0.5-1
qpid-cpp-client-0.18-13
qpid-cpp-server-0.18-13
qpid-qmf-0.18-13
qpid-tools-0.18-7
ruby-condor-wallaby-5.0.5-1
ruby-qpid-qmf-0.18-13
ruby-wallaby-0.16.3-1
wallaby-0.16.3-1
wallaby-utils-0.16.3-1

and it works. -->VERIFIED

Comment 9 errata-xmlrpc 2013-03-06 18:39:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0564.html


Note You need to log in before you can comment on or make changes to this bug.