Bug 722996 - Document advanced extra attribute usage in Aviary submits
Summary: Document advanced extra attribute usage in Aviary submits
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Deadline: 2011-09-10
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: Grid_Developer_Guide
Version: 2.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: 2.1
: ---
Assignee: Alison Young
QA Contact: Tomas Rusnak
URL:
Whiteboard:
Depends On: 733205 733209
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-07-18 17:51 UTC by Pete MacKinnon
Modified: 2012-01-26 19:15 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-01-26 19:15:31 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 754451 0 high CLOSED 2.1 engineering review and feedback 2021-02-22 00:41:40 UTC

Internal Links: 754451

Description Pete MacKinnon 2011-07-18 17:51:13 UTC
We need documentation to describe how users can interact with advanced features, like dynamic slots. 

Demonstrate various sets of extra attributes through Aviary submitJob to achieve advanced feature:

0) Document ability to override
1) Document constrained set of standard features accessible only via 
override
2) Drive set of (1) to zero
...
n) Go whole hog with all sub file features

Comment 1 Alison Young 2011-08-25 13:00:35 UTC
Hi Pete,

Can you provide me with some source content or hand off the needinfo to somebody who can. Have set the content deadline for Sept 9.

Thanks,
Alison

Comment 2 Pete MacKinnon 2011-08-29 19:59:45 UTC
"
Extra Attributes
----------------

The extra attributes option is an advanced feature that is provided so that a submitter can apply more fine-grained details to their job advertisement. These details are expressed in the ClassAd language of Condor <insert ref to that doc>. The Aviary SOAP API exposes these in the Attribute XSD type in the following form:
* name (string)
* type (enumeration)
* value (string)

The values of the type enumeration that can be set are:
* INTEGER
* FLOAT
* STRING
* EXPRESSION
* BOOLEAN

For example, let's consider the following ClassAd attributes expressed in their native syntax:

Output = /tmp/myjob.out

In Aviary, this same details would be expressed as:

         <extra>
            <name>Output</name>
            <type>STRING</type>
            <value>/tmp/myjob.out</value>
         </extra>

A user can override any of the basic job attributes if they choose by setting the allowOverrides XML attribute on the SubmitJob element to "true".
The required attributes are:
* Cmd
* Owner
* Iwd
An additional job attribute type for "Requirements" can be specified but will be given a default value if none is specified. 

If you submit a different value for a basic attribute in the extra list with the allowOverrides attribute set to "false", then it will be ignored and the value specified in the equivalent SubmitJob basic attribute field will be used.
For example:

		<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:job="http://job.aviary.grid.redhat.com">
		   <soapenv:Header/>
		   <soapenv:Body>
			  <job:SubmitJob allowOverrides="true">
				 <cmd>/bin/sleep</cmd>
				 <args>40</args>
				 <owner>condor</owner>
				 <iwd>/tmp</iwd>
				 <requirements>
					<type>OS</type>
					<value>LINUX</value>
				 </requirements>
				 <extra>
					<name>Owner</name>
					<type>STRING</type>
					<value>somebody</value>
				 </extra>
			  </job:SubmitJob>
		   </soapenv:Body>
		</soapenv:Envelope>

In this case the value of "somebody" will be used instead of "condor" because we have explicitly allowed an override of that basic attribute.

Since extra attributes are designed to allow tuning for job submission, let's look at a slightly more complex use case. Let's say we have a job that can make use of all the CPUs on a machine where condor is configured for dynamic slots <insert ref>.
The basic requirements field is simplified for common use cases so we need to add a more detailed requirements value for matching like the following example:

		<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:job="http://job.aviary.grid.redhat.com">
		   <soapenv:Header/>
		   <soapenv:Body>
			  <job:SubmitJob allowOverrides="true">
				 <cmd>/bin/sleep</cmd>
				 <args>40</args>
				 <owner>condor</owner>
				 <iwd>/tmp</iwd>
				 <extra>
					<name>Requirements</name>
					<type>EXPRESSION</type>
					<value>&quot;(TARGET.TotalCpus =!= UNDEFINED) &amp;&amp; ((Target.PartitionableSlot =?= TRUE) || (TARGET.DynamicSlot =?= TRUE)&quot;</value>
				 </extra>
				 <extra>
					<name>RequestCpus</name>
					<type>EXPRESSION</type>
					<value>TotalCpus</value>
				 </extra>
			  </job:SubmitJob>
		   </soapenv:Body>
		</soapenv:Envelope>
		
In this case if we had not specified allowOverrides="true", the Requirements field would have been given a default value of "True".

"

Comment 3 Pete MacKinnon 2011-08-30 19:51:15 UTC
Matt rightly pointed out that the above snippet should be:

<extra>
 <name>Out</name>
  <type>STRING</type>
 <value>/tmp/myjob.out</value>
</extra>

"Out" instead of "Output".

Comment 4 Pete MacKinnon 2011-09-29 20:42:35 UTC
Additionally...

"
Security
--------

Aviary supports submissions and queries over a secure SSL connection with mutual authentication. In order to enable this advanced feature, use Remote Configuration to add the SSLEnabledAviaryScheduler and SSLEnabledQueryServer features.

"

Comment 12 Robert Rati 2011-11-09 15:40:10 UTC
Those are the correct feature names.

Comment 14 Tomas Rusnak 2011-11-16 14:26:16 UTC
The "&amp;" and "quot;" directives are not transformed into readable form in example on the bottom of 5.1. submitJob Extra Attributes chapter:

<extra>
   <name>Requirements</name>
   <type>EXPRESSION</type>
   <value>&quot;(TARGET.TotalCpus =!= UNDEFINED)
   &amp;&amp; ((Target.PartitionableSlot =?= TRUE) ||
   (TARGET.DynamicSlot =?= TRUE)&amp;quot;</value>
</extra>

Please, could you correct it?

Comment 15 Pete MacKinnon 2011-11-16 14:34:15 UTC
Tomas, we decided that this should represent a literal XML fragment thus showing the correct encoding of " and &. Please see previous comments.

Comment 16 Tomas Rusnak 2011-11-16 14:39:22 UTC
Ah, I read that, but it was my misunderstanding. Thank you for you hint.

Changes in all chapters confirmed.

Revision 1-7	Fri Nov 11 2011	Alison Young
BZ#722996 - updated example code
Revision 1-5	Thu Oct 20 2011	Alison Young
BZ#722996 - updated security chapter
Revision 1-4	Wed Oct 19 2011	Alison Young
BZ#722996 - Advanced extra attribute usage in Aviary submits

Revision info updated according to changes.

>>> VERIFIED

Comment 17 Pete MacKinnon 2011-11-16 14:46:39 UTC
Alison, oops!

That last part:

&amp;quot;</value>

should be just:

quot;</value>

Comment 18 Pete MacKinnon 2011-11-16 22:51:34 UTC
&quot;</value>

that is :-)

Comment 20 Tomas Rusnak 2011-11-21 11:04:49 UTC
Confirmed changes according Pete comment:

<name>Requirements</name>
<type>EXPRESSION</type>
<value>&quot;(TARGET.TotalCpus =!= UNDEFINED)
&amp;&amp; ((Target.PartitionableSlot =?= TRUE) ||
(TARGET.DynamicSlot =?= TRUE)&quot;</value>

>>> VERIFIED

Comment 21 Lana Brindley 2012-01-26 19:15:31 UTC
This book is now available on redhat.com/docs. Please raise a new bug if you spot any issues.

Thanks,
LKB


Note You need to log in before you can comment on or make changes to this bug.