Bug 731065

Tested a solution.  When Cumin attempts to fetch output file contents based on a job classad, prepend the value of the Iwd classad attribute to the output file name if the output file name does not already begin with a "/".

Fixed in revision 4933.

Needs more work.  The above solution works with QMF, but Aviary does not take a path argument when retrieving an output file.  Rather, it takes an enumeration value indicating the type of the file and retrieves the file path from the job ad.

So, for this to work with Aviary, cumin needs to prepend the Iwd value to the output file arguments before submission if the fields do not contain paths.

Fixed in revision 4981.  Should work for both QMF and Aviary interfaces.

Note, related to BZ#739205 when working with Aviary.

Note, there may still be an additional hole here.  If a job is submitted from the command line without explicit paths on files, and the job is read from Cumin through Aviary, the path information will not be there.

I'll verify, this might lead to a BZ against Aviary since there would be nothing that cumin could do in this case.

Yes, the above comment is true.  IMHO, either Aviary has to use the working directory to extend the filenames in the case of no path, or the call needs to take a path argument instead of or in addition to the enumeration value.

I think this should be duped and opened against Aviary, cumin has been fixed as much as possible in this case.

Here's an example submission file that will illustrate this:

executable = /bin/sleep
args = 3d
requirements = 1
universe = vanilla
RequestMemory = 1
RequestDisk = 1
DiskUsage = 1
ImageSize = 1
should_transfer_files = Never
concurrency_limits = sleep
initialdir = /home/tmckay
output = test_submit.output
error = test_submit.error
log = test_submit.log
queue 1

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause
    Cumin did not handle cases where job ads contained output, error, or log file names without full paths.

Consequence
    Cumin failed to retrieve file contents under the Output tab for a job.

Fix
    Cumin attempts to append the working directory from the job ad to the names of output files if the file does not contain a full path.  When Cumin submits a job, it prepends the working directory to the output file names before submission.

Result
    In most cases Cumin will handle the output file names correctly.  However, if jobs are submitted through another mechanism like condor_submit and output files are not given full paths and Aviary is used by Cummin for data retrieval, Cumin still will not be able to retrieve the output files.

Prepare 2 jobs and executable
bz731065.txt - without absolute paths
bz731065_abs.txt - with absolute paths
job.sh - executable displaying "123 test"

Reproducing:

Install condor+cumin from RHN on rhel 5-i386 + 5-x86_64
condor-7.6.3-0.3.el5
condor-classads-7.6.3-0.3.el5
condor-qmf-7.6.3-0.3.el5
cumin-0.1.4916-1.el5
qpid-cpp-client-0.10-8.el5
qpid-cpp-server-0.10-8.el5
package python-wallaby is not installed

Install condor+cumin from RHN on rhel 6-i386 + 6-x86_64
condor-7.6.3-0.3.el6
condor-classads-7.6.3-0.3.el6
condor-qmf-7.6.3-0.3.el6
cumin-0.1.4916-1.el6
qpid-cpp-client-0.10-6.el6
qpid-cpp-server-0.10-6.el6
package python-wallaby is not installed

Submit job:
su -c 'condor_submit /tmp/bz731065.txt' test
Verify results:
Failed to open test_submit.output
Failed to open test_submit.error
test_submit.log OK

Submit job:
su -c 'condor_submit /tmp/bz731065_abs.txt' test
Verify results:
test_submit.output OK - "123 test" is displayed
test_submit.error OK
test_submit.log OK

Update to new version 5-i386 + 5-x86_64:
condor-7.6.4-0.7.el5
condor-classads-7.6.4-0.7.el5
condor-qmf-7.6.4-0.7.el5
cumin-0.1.5068-1.el5
qpid-cpp-client-0.10-9.el5
qpid-cpp-server-0.10-9.el5
python-wallaby-0.11.0-2.el5

Update to new version 6-i386 + 6-x86_64
condor-7.6.4-0.7.el6
condor-classads-7.6.4-0.7.el6
condor-qmf-7.6.4-0.7.el6
cumin-0.1.5068-1.el6
qpid-cpp-client-0.10-8.el6_1
qpid-cpp-server-0.10-8.el6_1
python-wallaby-0.11.0-2.el6

run 'cumin-admin upgrade-schema'

start cumin

Submit job:
su -c 'condor_submit /tmp/bz731065.txt' test
Verify result:
test_submit.output OK - "123 test" is displayed
test_submit.error OK
test_submit.log OK

Submit job:
su -c 'condor_submit /tmp/bz731065_abs.txt' test
Verify result:
test_submit.output OK - "123 test" is displayed
test_submit.error OK
test_submit.log OK

---> VERIFIED

Created attachment 528171 [details]
bz731065.txt

Created attachment 528172 [details]
bz731065_abs.txt

Created attachment 528173 [details]
job.sh

    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,11 +1 @@
-Cause
+Prior to this update, when a job returned output, an error message, or a log entry containing a file name without full path, the Cumin web console failed to retrieve the file contents under the Output tab for a job. With this update, Cumin attempts to append the working directory from the job ad to the names of output files if the file does not contain a full path. When Cumin submits a job, it prepends the working directory to the output file names before submission. Now, Cumin handles the output file names correctly in most cases. However, if a job is submitted through another mechanism, such as the condor_submit utility, the output files are not given full paths, and the Aviary web service is used by Cumin for data retrieval, Cumin still will not be able to retrieve the output files.-    Cumin did not handle cases where job ads contained output, error, or log file names without full paths.
-
-Consequence
-    Cumin failed to retrieve file contents under the Output tab for a job.
-
-Fix
-    Cumin attempts to append the working directory from the job ad to the names of output files if the file does not contain a full path.  When Cumin submits a job, it prepends the working directory to the output file names before submission.
-
-Result
-    In most cases Cumin will handle the output file names correctly.  However, if jobs are submitted through another mechanism like condor_submit and output files are not given full paths and Aviary is used by Cummin for data retrieval, Cumin still will not be able to retrieve the output files.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-0045.html