This chapter gives an overview of job description, but does not explain all elements that can be put into a job description. For detailed information about all elements and their ordering please see Job Description Schema documentation.

2. Single-Job Description

The general form of a job description used to start a single job (meant for creating a Managed Executable Job Resource instance) is as follows:

<job>
    <!--put additional elements here-->
    <executable><!--put executable here--></executable>
    <!--put additional elements here-->
</job>

Here is a basic example of a job description for a single-job:

<job>
    <executable>bin/echo</executable>
    <argument>Testing</argument>
    <argument>1...2...3</argument>
    <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
    <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
</job>

3. Multi-Job Description

The general form of a job description used to start a multi-job (meant for creating a Managed Multi Job Resource instance) is as follows:

<multiJob>
    <!--Put subjob default elements here.-->
    <job>
        <factoryEndpoint
                xmlns:gram="http://www.globus.org/namespaces/2008/03/gram/job"
                xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <wsa:Address>
                <!--put ManagedJobFactoryService address here-->
            </wsa:Address>
            <wsa:ReferenceParameters>
                <gram:ResourceID><!--put scheduler type here--></gram:ResourceID>
            </wsa:ReferenceParameters>
        </factoryEndpoint>
        <executable><!--put executable path here--></executable>
    </job>
    <!--put additional job elements here-->
</multiJob>

Here is a basic example of a job description for a multi-job:

<multiJob>
    <executable>/bin/echo</executable>
    <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
    <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
    <job>
        <factoryEndpoint
                xmlns:gram="http://www.globus.org/namespaces/2008/03/gram/job"
                xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <wsa:Address>
                https://mymachine.mydomain.com:8443/wsrf/services/ManagedJobFactoryService
            </wsa:Address>
            <wsa:ReferenceParameters>
                <gram:ResourceID>PBS</gram:ResourceID>
            </wsa:ReferenceParameters>
        </factoryEndpoint>
        <argument>Testing</argument>
        <argument>1...2...3</argument>
    <job>
    <job>
        <factoryEndpoint
                xmlns:gram="http://www.globus.org/namespaces/2008/03/gram/job"
                xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <wsa:Address>
                https://myothermachine.myotherdomain.org:8443/wsrf/services/ManagedJobFactoryService
            </wsa:Address>
            <wsa:ReferenceParameters>
                <gram:ResourceID>Pbs</gram:ResourceID>
            </wsa:ReferenceParameters>
        </factoryEndpoint>
        <argument>Hi There!</argument>
        <argument>Dear John!</argument>
    </job>
</multiJob>

4. Staging Directives

The GRAM4 job description schema imports types from the RFT job description schema for specifying staging directives (i.e. fileStageIn, fileStageOut, and fileCleanUp). See RFT transfer request for details on these imported types.

Since fileStageIn and fileStageOut are of type TransferRequestType and fileCleanUp is of type DeleteRequestType, mentally replace "transferRequest" with "fileStageIn" or "fileStageOut", and "deleteRequest" with "fileCleanUp" in the RFT transfer request documentation. The 'Request Options' section is of particular usefulness.

5. Substitution Variables

Job description variables are special strings in a job description that are replaced by the GRAM service with values that the client-side does not a priori know.

The set of variables is fixed in the gram service implementation. This is different from previous implementations of RSL substitutions in GT2 and GT3, where a user could define a new variable for use inside a job description document. This was done to preserve the simplicity of the job description XML schema (relatively to the GT3.2 RSL schema), which does not require a specialized XML parser to serialize a job description document.

Valid variables and their description:

GLOBUS_USER_HOME	The path to the home directory for the local account/user
GLOBUS_USER_NAME	The local account the job is running under
GLOBUS_JOB_ID	UUID of the job, created on the server-side
GLOBUS_SCRATCH_DIR	An alternative directory where a file system is shared with the compute nodes that a user might want to use. Typically it will provide more space than the users default HOME dir. This directory's value may contain ${GLOBUS_USER_HOME}, which will be replaced with value of that substitution.
GLOBUS_LOCATION	Path to the Globus Toolkit installation

Variable substitutions may not occur in all job description attributes. Their use is restricted to those which contain arbitrary string data and which may be used to access the local resource associated with a job. The list of attributes which may contain variables is:

executable
directory
argument
environment
stdin
stdout
stderr
libraryPath
fileStageIn
fileStageOut
fileCleanUp

6. Extensions

To allow adding features to GRAM4 while avoiding breaking compatibility between versions, an extensibility point was included in the job description schema. This appears as the <extensions> element at the bottom of a job description document. Starting with version 4.2.0 of the Globus Toolkit, GRAM4 will support both a number of specific extenions as well as generic constructs that can be used for passing custom values to the resource manager/scheduler adapter Perl modules.

6.1. Supported Extensions

The following are specific supported extensions to the GRAM4 job description schema. They do not require any modification of the resource manager/scheduler adapter Perl modules.

6.1.1. Condor specific parameters

If a user submits a job to Gram4 specifying Condor as local resource manager a condor-specific job description will be created from the XML job description which will be used when the job is submitted to Condor. A user can influence the creation of the condor-specific job description by adding condorsubmit elements to the extensions element:

<job>
    ...
    <extensions>
        <condorsubmit name="nameOfAnElement">valueOfTheElement</condorsubmit>
    </extensions>
</job>

More than one condorsubmit element can be placed in the extensions element.

The following example shows how to set a different Requirements element than is added by default. By default Gram4 adds a Requirement element and sets the parameter OpSys and Arch to values that fit the head-node where Gram4 is running. If e.g. the operating system on the head-node is Linux and the architecture is X86_64, the Requirements element in a Condor job description will look like

Requirements=OpSys == "LINUX" && Arch == "X86_64"

If this is not what is needed, requirements can be added as follows:

<job>
  <executable>/bin/date</executable>
  <extensions>
    <condorsubmit name="Requirements">OpSys == "LINUX" &amp;&amp; (Arch == "X86_64" || Arch == "INTEL")</condorsubmit>
  </extensions>
</job>

Note that the special char & must be coded as &amp.

6.1.2. PBS Node Selection Parameters

Node selection constraints in PBS can be specified in two ways, generally using a construct intended to eventually apply to all resource managers which support node selection, or explicitly by specifying a simple string element. The former will be more portable, but the later will appeal to those familiar with specifying node constraints for PBS jobs.

6.1.2.1. Using the nodes extensions element

To specify PBS node selection constraints explicitly, one can simply construct a single, simple string extension element named nodes with a value that conforms to the #PBS -l nodes=... PBS job description directive. The Globus::GRAM::ExtensionsHandler module will make this available to the PBS adapter script by invoking $description->{nodes}. The updated PBS adapter package checks for this value and will create a directive in the PBS job description using this value.

For example the following nodes extensions element

...
<extensions>
  <nodes>activemural:ppn=10+5:ia64-compute:ppn=2</nodes>
</extensions>
...

will result in the following directive in the PBS job description:

#PBS -l nodes=activemural:ppn=10+5:ia64-compute:ppn=2

6.1.2.2. Using the resourceAllocationGroup extensions element

To use the generic construct for specifying node selection constraints, use the resourceAllocationGroup element:

<extensions>
    <resourceAllocationGroup>
        <!-- Optionally select hosts by type and number... -->
        <hostType>...</hostType>
        <hostCount>...</hostCount>

        <!-- *OR* by host names -->

        <hostName>...</hostName>
        <hostName>...</hostName>
        . . .


        <!-- With a total CPU count for this group... -->
        <cpuCount>...</cpuCount>

        <!-- *OR* an explicit number of CPUs per node... -->
        <cpusPerNode>...</cpusPerNode>
        . . .


        <!-- And a total process count for this group... -->
        <processCount>...</processCount>

        <!-- *OR* an explicit number of processes per node... -->
        <processesPerNode>...</processesPerNode>
    </resourceAllocationGroup>
</extensions>

Extension elements specified according to the above pseudo-schema will be converted to an appropriate nodes parameter which will be treated as if an explicit nodes extension element were specified. Multiple resourceAllocationGroup elements may be specified. This will simply append the constraints to the nodes paramater with a '+' separator. Note that one cannot specify both hostType/hostCount and hostName elements. Similarly, one cannot specify both processCount and processesPerNode elements.

Here are some examples of using resourceAllocationGroup:

<!-- #PBS -l nodes=1:ppn=10 -->
<!-- 10 processes -->
<extensions>
    <resourceAllocationGroup>
        <cpuCount>10</cpuCount>
        <processCount>10</processCount>
    </resourceAllocationGroup>
</extensions>

<!-- #PBS -l nodes=activemural:ppn=10+5:ia64-compute:ppn=2 -->
<!-- 1 process (process default) -->
<extensions>
    <resourceAllocationGroup>
        <hostType>activemural</hostType>
        <cpuCount>10</cpuCount>
    </resourceAllocationGroup>
    <resourceAllocationGroup>
        <hostType>ia64-compute</hostType>
        <hostCount>5</hostCount>
        <cpusPerHost>2</cpusPerHost>
    </resourceAllocationGroup>
</extensions>

<!-- #PBS -l nodes=vis001:ppn=5+vis002:ppn=5+comp014:ppn=2+comp015:ppn=2 -->
<!-- 15 total processes -->
<extensions>
    <resourceAllocationGroup>
        <hostName>vis001</hostName>
        <hostName>vis002</hostName>
        <cpuCount>10</cpuCount>
        <processesPerHost>5</processesPerHost>
    </resourceAllocationGroup>
    <resourceAllocationGroup>
        <hostName>comp014</hostName>
        <hostName>comp015</hostName>
        <cpusPerHost>2</cpusPerHost>
        <processCount>5</processCount>
    </resourceAllocationGroup>
</extensions>

6.2. Additional Extension Constructs

The following are general constructs that are supported by the ExtensionsHandler.pm Perl module. Although no modifications to ExtensionsHandler.pm are required, you will need to edit the appropriate resource manager/scheduler adapter Perl module as neccessary to affect the submission of jobs to the local resource manager/batch scheduler.

The GRAM4 job description schema includes a section for extending the job description with custom elements. To make sense of this in the resource manager adapter Perl scripts, a Perl module named Globus::GRAM::ExtensionsHandler is provided to turn these custom elements into paramters that the adapter scripts can understand.

It should be noted that although non-GRAM XML elements only are allowed in the <extensions> element of the job description, the extensions handler makes no distinction based on namespace. Thus, <foo:myparam> and <bar:myparam> will both be treated as just <myparam>.

Familiarity with the adapter scripts is assumed in the following subsections.

6.2.1. Simple String Parameters

Simple string extension elements are converted into single-element arrays with the name of the unqualified tag name of the extension element as the array's key name in the Perl job description hash. Simple string extension elements can be considered a special case of the string array construct in the next section.

For example, adding the following element to the <extensions> element of the job description:

<extensions>
    <myparam>yahoo!</myparam>
</extensions>

will cause the $description->myparam() to return the following value:

'yahoo!'

6.2.2. String Array Parameters

String arrays are a simple iteration of the simple string element construct. If you specify more than one simple string element in the job description, these will be assembled into a multi-element array with the unqualified tag name of the extension elements as the array's key name in the Perl job description hash.

For example:

<extensions>
    <myparams>Hello</myparams>
    <myparams>World!</myparams>
</extensions>

will cause the $description->myparams() to return the following value:

            [ 'Hello', 'World!' ]

6.2.3. Name/Value Parameters

Name/value extension elements can be thought of as string arrays with an XML attribute 'name'. This will cause the creation of a two-dimensional array with the unqualified extension element tag name as the name of the array in the Perl job description hash.

For example:

<extensions>
    <myvars name="pi">3.14159</myvars>
    <myvars name="mole">6.022 x 10^23</myvars>
</extensions>

will cause the $description->myvars() to return the following value:

[ [ 'pi', '3.14159'], ['mole', '6.022 x 10^23'] ]

6.3. Supporting custom extensions in the Perl adapter modules.

See the System Administrator's Guide section on Configuring GRAM4 for information on how to customize the resource manager/scheduler adapter Perl modules

Prev		Next
Using GRAM4	Home	Submitting jobs