Bug 698664

Summary: [RFE] submit more than one processes in one cluster
Product: Red Hat Enterprise MRG Reporter: Martin Kudlej <mkudlej>
Component: condor-aviaryAssignee: grid-maint-list <grid-maint-list>
Status: CLOSED WONTFIX QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: DevelopmentCC: iboverma, matt, tstclair
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-26 20:22:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Martin Kudlej 2011-04-21 13:42:05 UTC
Description of problem:
I think it will be fine to have possibility to submit more than one process in one cluster as it is possible via condor_submit command by "queue N".

Now:
<xs:element name="SubmitJob">
  <xs:complexType>
    <xs:sequence> 
      <xs:element name="cmd" type="xs:string"/>
        <xs:element name="args" type="xs:string" minOccurs="0"/>
        <xs:element name="owner" type="xs:string"/>
        <xs:element name="iwd" type="xs:string"/>
        <xs:element name="submission_name" type="xs:string" minOccurs="0"/>
        <xs:element name="requirements" type="mrg:ResourceConstraint"  minOccurs="0" maxOccurs="unbounded"/>
        <xs:element name="extra" type="mrg:Attribute" minOccurs="0" maxOccurs="unbounded"/>
    </xs:sequence> 
    <xs:attribute name="allowOverrides" type="xs:boolean" default="false"/>
  </xs:complexType>   
</xs:element>

Future:
<xs:element name="SubmitJob">
  <xs:complexType>
    <xs:sequence> 
      <xs:element name="cmd" type="xs:string"/>
        <xs:element name="args" type="xs:string" minOccurs="0"/>
        <xs:element name="owner" type="xs:string"/>
        <xs:element name="iwd" type="xs:string"/>
        <xs:element name="submission_name" type="xs:string" minOccurs="0"/>
        <xs:element name="requirements" type="mrg:ResourceConstraint"  minOccurs="0" maxOccurs="unbounded"/>
        <xs:element name="extra" type="mrg:Attribute" minOccurs="0" maxOccurs="unbounded"/>

        <xs:element name="processes" type="xs:int" default="1"/>

    </xs:sequence> 
    <xs:attribute name="allowOverrides" type="xs:boolean" default="false"/>
  </xs:complexType>   
</xs:element>

Comment 2 Pete MacKinnon 2012-02-25 01:07:01 UTC
Proposal for modified Aviary XSD submitJob operation:

<xs:element name="SubmitJob">
<xs:complexType>
<xs:sequence>
<xs:element name="cmd" type="xs:string"/>
<xs:element name="args" type="xs:string" minOccurs="0"/>
<xs:element name="owner" type="xs:string"/>
<xs:element name="iwd" type="xs:string"/>
<xs:element name="submission_name" type="xs:string" minOccurs="0"/>
<xs:element name="requirements" type="mrg:ResourceConstraint"
minOccurs="0" maxOccurs="unbounded"/>
<xs:element name="extra" type="mrg:Attribute" minOccurs="0"
maxOccurs="unbounded"/>
<!-- START NEW -->
<xs:element name="queue" type="xs:int" minOccurs="0"/>
<!-- END NEW -->
</xs:sequence>
<xs:attribute name="allowOverrides" type="xs:boolean" default="false"/>
</xs:complexType>
</xs:element>

User will be able to optionally specify the number of jobs to create in
this new submission. Within the implementation, we will loop and call
submit() on the SchedulerObject to the value of "queue", returning a
partially opaque JobID for each new cluster created. We mean
"partially", since each JobID will have a cluster.proc string
incorporated within the ID (same as current impl). This portion can be
extracted and used for the other scheduler operations such as hold,
release, etc. which require a c.proc for executing the state-related
operation. This could be an expensive return value for large queue values (say 1000) so may need to consider whether it is simply the submission ID that is returned.

A token (e.g., $N$) will be used in managing log file naming patterns in the
submit attributes and represents nothing more than the int sequence of
the expansion and has no direct relationship to a cluster or proc value. Creating multiple clusters or procs in a single cluster should not be 
part of the interface contract. Users should not rely on the behavior.
As for where the token is valid, the key places are going to be Args, 
Out, Err, Log, maybe Iwd and Cmd.

Note that this proposal differs from the condor_submit behavior of
queue=N which constructs a cluster and N procs. Instead, Aviary will be
creating N clusters associated with 1 globally unique submission string.

Some thought still needs to be given as to whether the submission name is also a token used in the contexts noted above.

Comment 3 Anne-Louise Tangring 2016-05-26 20:22:54 UTC
MRG-Grid is in maintenance and only customer escalations will be considered. This issue can be reopened if a customer escalation associated with it occurs.