Bug 700774 - [RFE] aviary doesn't insert default values into job classads
Summary: [RFE] aviary doesn't insert default values into job classads
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: condor
Version: 2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: 2.0.1
: ---
Assignee: Pete MacKinnon
QA Contact: Martin Kudlej
URL:
Whiteboard:
Depends On:
Blocks: 723887
TreeView+ depends on / blocked
 
Reported: 2011-04-29 11:09 UTC by Martin Kudlej
Modified: 2012-02-07 10:04 UTC (History)
5 users (show)

Fixed In Version: condor-7.6.3-0.1
Doc Type: Enhancement
Doc Text:
Cause: Aviary-submitted jobs included a minimal set of job attributes necessary to run in most universes. However, it was not a complete set for all slot configurations. Consequence: Aviary-submitted jobs would not match to startds configured for dynamic slot provisioning unless a user explicitly added the necessary attributes. Change: Aviary submissions now implicitly include the required attributes necessary for correct matching to dynamic provisioning slots. Result: Aviary-submitted jobs match and run on dynamic provisioning slots without additional user intervention.
Clone Of:
Environment:
Last Closed: 2011-09-07 16:43:19 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:1249 0 normal SHIPPED_LIVE Moderate: Red Hat Enterprise MRG Grid 2.0 security, bug fix and enhancement update 2011-09-07 16:40:45 UTC

Description Martin Kudlej 2011-04-29 11:09:26 UTC
Description of problem:
I've set Dynamic slots feature and it has modified slots requirements from:
Requirements = ( START ) && ( IsValidCheckpointPlatform )
to:
Requirements = ( START ) && ( IsValidCheckpointPlatform ) && ( WithinResourceLimits )

and WithinResourceLimits contains:
WithinResourceLimits = ( ifThenElse(TARGET.RequestCpus =!= undefined,MY.Cpus > 0 && TARGET.RequestCpus <= MY.Cpus,1 <= MY.Cpus) && ifThenElse(TARGET.RequestMemory =!= undefined,MY.Memory > 0 && TARGET.RequestMemory <= MY.Memory,false) && ifThenElse(TARGET.RequestDisk =!= undefined,MY.Disk > 0 && TARGET.RequestDisk <= MY.Disk,false) )

So if job doesn't contain classads RequestDisk and RequestMemory it cannot be matched with any slots. 
Condor_submit adds these classads automatically but aviary doesn't. So they should be added explicitly to "extra" classads list.

The same situation is with QMF submit without extra "Request*" classads.

Version-Release number of selected component (if applicable):
condor-7.6.1-0.4.el6.i686

How reproducible:
100%

Steps to Reproduce:
1. install condor pool, aviary
2. set up Dynamic slots feature
3. try submit simple job via aviary and same job via condor_submit
  
Actual results:
Aviary scheduler and QMF scheduler don't add required classads into job classads as condor_submit does. For example "Request*" classads.

Expected results:
Aviary scheduler and QMF scheduler will add required classads into job classads as condor_submit does.

Comment 2 Pete MacKinnon 2011-07-05 18:57:48 UTC
Fixed upstream at UW V7.6_branch b9a8b412a2

condor_submit-like defaults added in for:

- DiskUsage=1
- ImageSize=0
- RequestCpus=1
- RequestDisk=DiskUsage
- RequestMemory=ceiling(ifThenElse(JobVMMemory =!= undefined,JobVMMemory,ImageSize / 1024.000000))

Comment 3 Pete MacKinnon 2011-07-06 14:47:15 UTC
Trevor, can you comment on the support from cumin for this? The QMF plugins don't augment the inbound submission across the board (e.g., Requirements are sent from the cumin client).

Comment 4 Trevor McKay 2011-07-06 15:15:58 UTC
Looks like we have defaults for these:

- DiskUsage=0
- RequestDisk, configurable default, out of the box is 1GB

- RequestMemory
for non-VM jobs, configurable default, out of the box is 512MB
for VM jobs

            "ceiling(ifThenElse(JobVMMemory =!= undefined," + \
                               "JobVMMemory, " + \
                               "ImageSize / 1024.000000))"

I don't see any defaults for ImageSize or RequestCpus set in cumin.

Comment 5 Pete MacKinnon 2011-07-08 14:57:14 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause: Aviary-submitted jobs included a minimal set of job attributes necessary to run in most universes. However, it was not a complete set for all slot configurations.
Consequence: Aviary-submitted jobs would not match to startds configured for dynamic slot provisioning unless a user explicitly added the necessary attributes.
Change: Aviary submissions now implicitly include the required attributes necessary for correct matching to dynamic provisioning slots.
Result: Aviary-submitted jobs match and run on dynamic provisioning slots without additional user intervention.

Comment 7 Martin Kudlej 2011-08-09 11:10:54 UTC
Tested with 
condor-7.6.3-0.3
condor-aviary-7.6.3-0.3
on RHEL 5.7/6.1 x i386/x86_64 and it works. -> VERIFIED

Comment 8 errata-xmlrpc 2011-09-07 16:43:19 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-1249.html


Note You need to log in before you can comment on or make changes to this bug.