Bug 614993 - Using Memory or RequestMemory in job requirements drops both default RequestMemory and Memory requirements
Summary: Using Memory or RequestMemory in job requirements drops both default RequestM...
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: condor
Version: 1.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: 1.3
: ---
Assignee: Matthew Farrellee
QA Contact: Tomas Rusnak
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-07-15 18:13 UTC by Jon Thomas
Modified: 2018-10-27 13:33 UTC (History)
3 users (show)

(edit)
Previously, condor_submit keyed automatic inclusion of Memory and RequestMemory requirements off the same value. Specifying Memory or RequestMemory in job Requirements would mean the default expression for the other was not automatically included. With this update, automatic inclusion for Memory is keyed off absence of Memory in Requirements and RequestMemory is keyed off RequestMemory. It is now possible to add Memory or RequestMemory to Requirements and get the default expression for the other.
Clone Of:
(edit)
Last Closed: 2010-10-14 16:13:38 UTC


Attachments (Terms of Use)
patch that breaks apart these two requirements (1.30 KB, patch)
2010-07-15 18:13 UTC, Jon Thomas
no flags Details | Diff


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2010:0773 normal SHIPPED_LIVE Moderate: Red Hat Enterprise MRG Messaging and Grid Version 1.3 2010-10-14 15:56:44 UTC

Description Jon Thomas 2010-07-15 18:13:59 UTC
Created attachment 432186 [details]
patch that breaks apart these two requirements

The default Memory and RequestMemory requirements are

( ( (Memory * 1024) >= ImageSize ) && ( ( RequestMemory * 1024 ) >= ImageSize ) )";

When a user defines any RequestMemory or Memory requirement ( but not virtualmemory), the system drops both ( (Memory * 1024) >= ImageSize ) and ( ( RequestMemory * 1024 ) >= ImageSize ) ).

This means, if a user were to submit a job with (RequestMemory =!= UNDEFINED), they would also have to add ( (Memory * 1024) >= ImageSize ) if they wanted this as a requirement.

To duplicate submit a job with no requirements to get a baseline and examine the jobs's classad. Then submit a job with requirements = (RequestMemory =!= UNDEFINED) and examine requirements in classad.

Comment 1 Matthew Farrellee 2010-07-16 00:05:22 UTC
Bad, pre-patch...

$ echo "cmd=/bin/true\nqueue" | condor_submit -dump out ; grep Requirements out 
Requirements = (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (((Memory * 1024) >= ImageSize) && ((RequestMemory * 1024) >= ImageSize)) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

$ echo "cmd=/bin/true\nrequirements=Memory=!=UNDEFINED\nqueue" | condor_submit -dump out ; grep Requirements out 
Requirements = (Memory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

$ echo "cmd=/bin/true\nrequirements=RequestMemory=!=UNDEFINED\nqueue" | condor_submit -dump out ; grep Requirements out 
Requirements = (RequestMemory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

$ echo "cmd=/bin/true\nrequirements=RequestMemory=!=UNDEFINED && Memory =!= UNDEFINED\nqueue" | condor_submit -dump out ; grep Requirements out 
Requirements = (RequestMemory =!= UNDEFINED && Memory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)


Good, post-patch...

$ echo "cmd=/bin/true\nqueue" | ./condor_submit -dump out ; grep Requirements out  
Requirements = (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && ((Memory * 1024) >= ImageSize) && ((RequestMemory * 1024) >= ImageSize) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

$ echo "cmd=/bin/true\nrequirements=Memory=!=UNDEFINED\nqueue" | ./condor_submit -dump out ; grep Requirements out
Requirements = (Memory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && ((RequestMemory * 1024) >= ImageSize) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

$ echo "cmd=/bin/true\nrequirements=RequestMemory=!=UNDEFINED\nqueue" | ./condor_submit -dump out ; grep Requirements out
Requirements = (RequestMemory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && ((Memory * 1024) >= ImageSize) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

$ echo "cmd=/bin/true\nrequirements=RequestMemory=!=UNDEFINED && Memory =!= UNDEFINED\nqueue" | ./condor_submit -dump out ; grep Requirements out
Requirements = (RequestMemory =!= UNDEFINED && Memory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)


Difference is when explicitly placing RequestMemory or Memory in the Requirements expression, they remove only the default RequestMemory or Memory expressions provided by condor_submit.

The expression is grouped differently, but jobs still run.

$ echo "cmd=/bin/true\nrequirements=RequestMemory=!=UNDEFINED && Memory =!= UNDEFINED\nqueue" | condor_submit                                  
Submitting job(s).
1 job(s) submitted to cluster 520.
$ echo "cmd=/bin/true\nrequirements=RequestMemory=!=UNDEFINED\nqueue" | condor_submit 
Submitting job(s).
1 job(s) submitted to cluster 521.
$ echo "cmd=/bin/true\nqueue" | condor_submit 
Submitting job(s).
1 job(s) submitted to cluster 522.

$ condor_history 520
 ID      OWNER            SUBMITTED     RUN_TIME ST   COMPLETED CMD            
 520.0   matt            7/15 16:59   0+00:00:01 C   7/15 16:59 /bin/true
$ condor_history 521
 ID      OWNER            SUBMITTED     RUN_TIME ST   COMPLETED CMD            
 521.0   matt            7/15 16:59   0+00:00:00 C   7/15 16:59 /bin/true
$ condor_history 522
 ID      OWNER            SUBMITTED     RUN_TIME ST   COMPLETED CMD            
 522.0   matt            7/15 16:59   0+00:00:01 C   7/15 17:00 /bin/true

Comment 2 Matthew Farrellee 2010-07-16 00:23:14 UTC
Built in 7.4.4-0.5

Comment 3 Tomas Rusnak 2010-08-09 12:44:15 UTC
Reproduced on:

$CondorVersion: 7.4.4 Jun 30 2010 BuildID: RH-7.4.4-0.4.el5 PRE-RELEASE $
$CondorPlatform: I386-LINUX_RHEL5 $

Results:
=======TEST1=======
Requirements = (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (((Memory * 1024) >= ImageSize) && ((RequestMemory * 1024) >= ImageSize)) && (TARGET.FileSystemDomain == MY.FileSystemDomain)
=======TEST2=======
Requirements = (Memory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)
=======TEST3=======
Requirements = (RequestMemory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)
=======TEST4=======
Requirements = (RequestMemory =!= UNDEFINED && Memory =!= UNDEFINED) && (Arch == "INTEL") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

Comment 4 Tomas Rusnak 2010-08-09 12:49:12 UTC
Tested on all combination of RHEL4/5 and x86/x86_64 architecture using:

condor-7.4.4-0.8

Results:
=======TEST1=======
Requirements = (Arch == "X86_64") && (OpSys == "LINUX") && (Disk >= DiskUsage) && ((Memory * 1024) >= ImageSize) && ((RequestMemory * 1024) >= ImageSize) && (TARGET.FileSystemDomain == MY.FileSystemDomain)
=======TEST2=======
Requirements = (Memory =!= UNDEFINED) && (Arch == "X86_64") && (OpSys == "LINUX") && (Disk >= DiskUsage) && ((RequestMemory * 1024) >= ImageSize) && (TARGET.FileSystemDomain == MY.FileSystemDomain)
=======TEST3=======
Requirements = (RequestMemory =!= UNDEFINED) && (Arch == "X86_64") && (OpSys == "LINUX") && (Disk >= DiskUsage) && ((Memory * 1024) >= ImageSize) && (TARGET.FileSystemDomain == MY.FileSystemDomain)
=======TEST4=======
Requirements = (RequestMemory =!= UNDEFINED && Memory =!= UNDEFINED) && (Arch == "X86_64") && (OpSys == "LINUX") && (Disk >= DiskUsage) && (TARGET.FileSystemDomain == MY.FileSystemDomain)

>>> VERIFIED

Comment 5 Florian Nadge 2010-10-08 10:05:35 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, the default Memory and RequestMemory requirements were combined.  If users were to submit a job with RequestMemory, they would also have to add Memory. This update separates these two requirements.

Comment 6 Matthew Farrellee 2010-10-08 11:43:41 UTC
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1,4 @@
-Previously, the default Memory and RequestMemory requirements were combined.  If users were to submit a job with RequestMemory, they would also have to add Memory. This update separates these two requirements.+C: condor_submit keyed automatic inclusion of Memory and RequestMemory requirements off the same value.
+C: Specifying Memory or RequestMemory in job Requirements would mean the default expression for the other was not automatically included.
+F: Automatic inclusion for Memory is keyed off absence of Memory in Requirements and RequestMemory is keyed off RequestMemory.
+R: It is possible to add either to Requirements and get the default expression for the other.

Comment 7 Martin Prpič 2010-10-10 11:27:21 UTC
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,4 +1 @@
-C: condor_submit keyed automatic inclusion of Memory and RequestMemory requirements off the same value.
+Previously, condor_submit keyed automatic inclusion of Memory and RequestMemory requirements off the same value. Specifying Memory or RequestMemory in job Requirements would mean the default expression for the other was not automatically included. With this update, automatic inclusion for Memory is keyed off absence of Memory in Requirements and RequestMemory is keyed off RequestMemory. It is now possible to add Memory or RequestMemory to Requirements and get the default expression for the other.-C: Specifying Memory or RequestMemory in job Requirements would mean the default expression for the other was not automatically included.
-F: Automatic inclusion for Memory is keyed off absence of Memory in Requirements and RequestMemory is keyed off RequestMemory.
-R: It is possible to add either to Requirements and get the default expression for the other.

Comment 9 errata-xmlrpc 2010-10-14 16:13:38 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0773.html


Note You need to log in before you can comment on or make changes to this bug.