Input files (if staged to the remote host) are always placed in the remote experiment directory. However you can specify the name used for the file in the remote directory with the destination name (DST above). This feature is very useful when your executable always expect a fixed input filename, and you want to process different input files (as is common in parametric computations).

	Important
	The destination names for input files MUST be a single name, do not use absolute paths or URLs.

You can specify the input files using:

	Note
	The executable is treated as an input file, so the same remarks are applicable for the EXECUTABLE Job Template parameter.

Output files are always copied FROM the remote experiment directory. However you can specify the destination of the output files of your applications.

	Important
	The source names for output files MUST be a single name, do not use absolute paths or URLs.

You can specify the destination of output files using:

Standard streams includes the standard input for your executable (STDIN_FILE) and its standard output and error (STDOUT_FILE and STDERR_FILE).

The STDIN_FILE can be defined using any of the methods described above for the input files. However you can not specified a destination name for the standard input stream, as is internally handled by the system. Note also that only one standard input file can be specified.

Example:

STDIN_FILE = In/input.${JOB_ID} #Will use input from directory In

The STDOUT_FILE and STDERR_FILE parameters can be defined using any of the methods described above for the output files. However you can not specified a source name (only destination). Note also that only one standard output and error files can be specified.

Example:

STDOUT_FILE = Out/ofile #Will place stdout in Out directory with name ofile

Restart files are periodically copied to the job directory ($GW_LOCATION/var/$JOB_ID/). Restart files are only specified with its name. Note also that you can defined a checkpointing server with the CHECKPOINT_URL Job Template parameter.

Example:

RESTART_FILES = tmp_file

The syntax of the requirement expressions is defined as:

stmt::= expr';'
expr::= VARIABLE '=' INTEGER
        | VARIABLE '>' INTEGER
        | VARIABLE '<' INTEGER
        | VARIABLE '=' STRING
        | expr '&' expr
        | expr '|' expr
        | '!' expr
        | '(' expr ')'

Each expression is evaluated to 1 (TRUE) or 0 (FALSE). Only those hosts for which the requirement expression is evaluated to TRUE will be considered to execute the job.

Logical operators are as expected ( less '<', greater '>', '&' AND, '|' OR, '!' NOT), '=' means equals with integers. When you use '=' operator with strings, it performs a shell wildcard pattern matching.

Examples:

REQUIREMENTS = LRMS_NAME = "*pbs*"; # Only use pbs like jobmanagers
REQUIREMENTS = HOSTNAME = "*.es"; #Only hosts in Spain
REQUIREMENTS = HOSTNAME = "hydrus.dacya.ucm.es"; #Only use hydrus.dacya.ucm.es

The syntax of the rank expressions is defined as:

stmt::= expr';'
expr::= VARIABLE
        | INTEGER
        | expr '+' expr
        | expr '-' expr
        | expr '*' expr
        | expr '/' expr
        | '-' expr
        | '(' expr ')'

Rank expressions are evaluated using each host information. '+', '-', '*', '/' and '-' are arithmetic operators, so only integer values should be used in rank expressions.

To set the REQUIREMENTS and RANK parameter values the following extended set of variables, provided by the Information Manager, can be used:

Job environment variables can be easily set with the ENVIRONMENT parameter of the Job Template. These environment variables are parsed, so you can use the GridWay variables defined in Section 4, “Variable substitution”, to set the job environment.

	Note
	The variables defined in the ENVIRONMENT are "sourced" in a bash shell. In this way you can take advantage of the bash substitution capabilities and built-in functions. For example: ENVIRONMENT = VAR = "`expr ${JOB_ID} + 3`" # will set VAR to JOB_ID + 3

In addition to those variables set in the ENVIRONMENT parameter, GridWay set the following variables, that can be used by your applications:

If you want to specify your own monitor or wrapper script you can do it using WRAPPER or MONITOR variables in your job template (or modifying the default one located in the etc directory of your GW installation). The file you specify must be a full path name of the script or a relative path if it is located inside $GW_LOCATION. Here you can not use gsiftp or file url protocol prefixes.

GridWay supports Job Submission Description Language (JSDL). This specification is a language for describing the job requirements for submission to resources. The JSDL language specification is based on XML Schema that facilitate the expression of those requirements as a set of XML elements. More info at https://forge.gridforum.org/sf/projects/jsdl-wg

The JSDL document structure is as follows:

<JobDefinition>
|-------<JobDescription>
|---------------<JobIdentification>
|-----------------------<JobName>?
|-----------------------<Description>?
|-----------------------<JobAnnotation>*
|-----------------------<JobProject>*
|-----------------------<xsd:any##other>*
|---------------</JobIdentification>?
|---------------<Application>
|-----------------------<ApplicationName>?
|-----------------------<ApplicationVersion>?
|-----------------------<Description>?
|-----------------------<xsd:any##other>*
|---------------</Application>?
|---------------<Resources>?
|-----------------------<CandidateHosts>
|-------------------------------<HostName>+
|-----------------------</CandidateHosts>?
|-----------------------<FileSystem>
|-------------------------------<Description>?
|-------------------------------<MountPoint>?
|-------------------------------<MountSource>?
|-------------------------------<DiskSpace>?
|-------------------------------<FileSystemType>?
|-------------------------------<xsd:any##other>*
|-----------------------</FileSystem>*
|-----------------------<ExclusiveExecution>?
|-----------------------<OperatingSystem>?
|-------------------------------<OperatingSystemType>
|---------------------------------------<OperatingSystemName>
|---------------------------------------<xsd:any##other>*
|-------------------------------</OperatingSystemType>?
|-------------------------------<OperatingSystemVersion>?
|-------------------------------<Description>?
|-------------------------------<xsd:any##other>*
|-----------------------</OperatingSystem>?
|-----------------------<CPUArchitecture>
|-------------------------------<CPUArchitectureName>
|-------------------------------<xsd:any##other>*
|-----------------------</CPUArchitecture>?
|-----------------------<IndividualCPUSpeed>?
|-----------------------<IndividualCPUTime>?
|-----------------------<IndividualCPUCount>?
|-----------------------<IndividualNetworkBandwidth>?
|-----------------------<IndividualPhysicalMemory>?
|-----------------------<IndividualVirtualMemory>?
|-----------------------<IndividualDiskSpace>?
|-----------------------<TOTALCPUTime>?
|-----------------------<TOTALCPUCount>?
|-----------------------<TOTALPhysicalMemory>?
|-----------------------<TOTALVirtualMemory>?
|-----------------------<TOTALDiskSpace>?
|-----------------------<TOTALResourceCount>?
|-----------------------<xsd:any##other>*
|---------------</Resources>?
|---------------<DataStaging>
|-----------------------<FileName>
|-----------------------<FileSystemName>?
|-----------------------<CreationFlag>
|-----------------------<DeleteOnTermination>?
|-----------------------<Source>
|-------------------------------<URI>?
|-------------------------------<xsd:any##other>*
|-----------------------</Source>?
|-----------------------<Target>
|-------------------------------<URI>?
|-------------------------------<xsd:any##other>*
|-----------------------</Target>?
|-----------------------<xsd:any##other>*
|---------------</DataStaging>*
|---------------<xsd:any##other>*
</JobDefinition>

	Note
	The symbol "?" denotes zero or one occurrences, "*" denotes zero or more occurrences and "+" denotes one or more occurrences.

This schema defines the JSDL specification for describing an application executed on a POSIX compliance system. Due to GridWay Job Template specification, this schema MUST be included in the JSDL file. The JSDL POSIX Application Schema is as follow:

<POSIXApplication name="xsd:NCName"?>
|-------<Executable>?
|-------<Argument>*
|-------<Input>?
|-------<Output>?
|-------<Error>?
|-------<WorkingDirectory>?
|-------<Environment>*
|-------<WallTimeLimit>?
|-------<FileSizeLimit>?
|-------<CoreDumpLimit>?
|-------<DataSegmentLimit>?
|-------<LockedMemoryLimit>?
|-------<MemoryLimit>?
|-------<OpenDescriptorsLimit>?
|-------<PipeSizeLimit>?
|-------<StackSizeLimit>?
|-------<CPUTimeLimit>?
|-------<ProcessCountLimit>?
|-------<VirtualMemoryLimit>?
|-------<ThreadCountLimit>?
|-------<UserName>?
|-------<GroupName>?
</POSIXApplication name="xsd:NCName"?>

More details at https://forge.gridforum.org/sf/projects/jsdl-wg

This schema defines the JSDL specification for describing a simple HPC application that is made up of an executable file running within an operating system process. It shares much in common with the JSDL POSIXApplication. The JSDL HPC Application Schema is as follow:

<HPCProfileApplication name="xsd:NCName"?>
|-------<Executable>?
|-------<Argument>*
|-------<Input>?
|-------<Output>?
|-------<Error>?
|-------<WorkingDirectory>?
|-------<Environment>*
|-------<UserNamet>?
</HPCProfileApplication name="xsd:NCName"?>

More details at https://forge.gridforum.org/sf/projects/jsdl-wg

Next table compares JSDL and GridWay Job Template schema, and the JSDL elements support by the current GridWay version.

This is a well known exercise. For our purposes, we will calculate the integral of the following function:

Being f(x) = 4/(1+x2). So, π will be the integral of f(x) in the interval [0,1].

In order to calculate the whole integral, it's interesting to divide the function in several sections and compute them separately:

As you can see, the more sections you make, the more exact π will be:

So, you have a Grid with some nodes, you have GridWay... Why don't use them to calculate the π number by giving all the nodes a section to compute with only one command?

	Note
	You will find all the files needed to perform this example in the $GW_LOCATION/examples/pi directory.

For this example, we have chosen the C Programming Language. Create a text file called pi.c and copy inside the following lines:

#include <stdio.h>
#include <string.h>

int main (int argc, char** args)
{
  int task_id;
  int total_tasks;
  long long int n;
  long long int i;

  double l_sum, x, h;

  task_id = atoi(args[1]);
  total_tasks = atoi(args[2]);
  n = atoll(args[3]);

  fprintf(stderr, "task_id=%d total_tasks=%d n=%lld\n", task_id, total_tasks, n);

  h = 1.0/n;

  l_sum = 0.0;

  for (i = task_id; i < n; i += total_tasks)
  {
    x = (i + 0.5)*h;
    l_sum += 4.0/(1.0 + x*x);
  }

  l_sum *= h;

  printf("%0.12g\n", l_sum);

  return 0;
}
  

Now it's time to compile it:

$ gcc -O3 pi.c -o pi

after this, you should have an executable called pi. This command receives three parameters:

For making GridWay work with your program, you must create a Job Template. In this case, we will call it pi.jt. Copy the following lines inside:

EXECUTABLE  = pi
ARGUMENTS   = ${TASK_ID} ${TOTAL_TASKS} 100000
STDOUT_FILE = stdout_file.${TASK_ID}
STDERR_FILE = stderr_file.${TASK_ID}
RANK        = CPU_MHZ

This time, we will submit an array of jobs. This is done by issuing the following command:

$ gwsubmit -v -t pi.jt -n 4
ARRAY ID: 0

TASK JOB
0    0
1    1
2    2
3    3

In order to wait for the jobs to complete, you can use the gwwait command.

The argument passed to gwwait is the array identifier given by gwsubmit when executed with the -v option. It could be also obtained through gwps

This command will block and return when all jobs have been executed:

$ gwwait -v -A 0
0   : 0
1   : 0
2   : 0
3   : 0

This command, when issued with option -v shows the exit codes for each job in the array (usually, 0 means success).

The execution of these jobs has returned some output files with the result of each execution:

stdout_file.0
stdout_file.1
stdout_file.2
stdout_file.3

Now, we will need something to sum the results inside each file. For this, you can use an awk script like the following:

$ awk 'BEGIN {sum=0} {sum+=$1} END {printf "Pi is %0.12g\n", sum}' stdout_file.*
Pi is 3.1415926536

Well, not much precision, right? You could try it again, but this time with a much higher number of intervals (e.g. 10,000,000,000). Would you increment also the number of tasks? Which would be the best compromise?

Do you imagine how easy would be to implement these steps in a shell script in order to perform them unattendedly? Here you are the prove:

#!/bin/sh

AID=`gwsubmit -v -t pi.jt -n 4 | head -1 | awk '{print $3}'`

if [ $? -ne 0 ]
then
    echo "Submission failed!"
    exit 1
fi

gwwait -v -A $AID

if [ $? -eq 0 ]
then
    awk 'BEGIN {sum=0} {sum+=$1} END {printf "Pi is %0.12g\n", sum}' stdout_file.*
else
    echo "Some tasks failed!"
fi
    

A DAG-based workflow consists of a temporal relationship between tasks, where the input, output or execution of one ore more tasks depends on one or more other tasks. For this example we have chosen a simple workflow.

Figure 4.1. Workflow example.

In this example, job A generates a random number, jobs B and C add 1 to that number and, finally job D adds the result of these jobs. This is the final result is two times the number generated by A, plus two. In our case, the numbers are passed between jobs using the standard output files.

Job Template for job A (A.jt):

EXECUTABLE=/bin/echo
ARGUMENTS="$RANDOM"
STDOUT_FILE=out.A

Job Template for jobs B and C (B.jt and C.jt):

EXECUTABLE=/usr/bin/expr
ARGUMENTS="`cat out.A`" + 1
INPUT_FILES=out.A
STDOUT_FILE=out.B #out.C for job C

Job Template for job D (D.jt):

EXECUTABLE=/usr/bin/expr
ARGUMENTS="`cat out.B`" + "`cat out.C`"
INPUT_FILES=out.B, out.C
STDOUT_FILE=out.workflow

Once you have set up the previous Job Templates, the workflow can be easily submitted with the following commands:

$ gwsubmit -v -t A.jt
JOB ID: 5

$ gwsubmit -v -t B.jt -d "5"
JOB ID: 6

$ gwsubmit -v -t C.jt -d "5"
JOB ID: 7

$ gwsubmit -t C.jt -d "6 7"

	Note
	In the previous example, jobs B and C can be submitted as an array job using just one template with output, OUTPUT_FILES = out.${TASK_ID}. Therefore, input of job D will be INPUT_FILES = out.0, out.1.

The above steps can be easily implemented in a shell script.

#!/bin/sh

A_ID=`gwsubmit -v -t A.jt | cut -f2 -d':' | cut -f2 -d' '`
B_ID=`gwsubmit -v -t B.jt -d "$A_ID" | cut -f2 -d':' | cut -f2 -d' '`
C_ID=`gwsubmit -v -t C.jt -d "$A_ID" | cut -f2 -d':' | cut -f2 -d' '`
D_ID=`gwsubmit -v -t D.jt -d "$B_ID $C_ID"| cut -f2 -d':' | cut -f2 -d' '`

#Sync with last job of the workflow
gwwait $D_ID

echo "Random number `cat out.A`"
echo "Workflow computation `cat out.workflow`"
  

Note that when input and output files vary depending on the iteration or job id number, you should generate Job Templates dynamically before submitting each job. This can be done programmatically by using the DRMAA API, or via shell scripting.

4.1.1. Using gwdag tool

Alternatively you can describe DAG workflows using a file similar to the one used by Condor DAGMAN. In this case the dependencies are not managed by GW but by the gwdag tool. You only have to substitute Condor job descriptions with GridWay job templates. Here is a file describing the same DAG as the previous example:

JOB  A  A.jt
JOB  B  B.jt
JOB  C  C.jt
JOB  D  D.jt
PARENT A CHILD B C
PARENT B C CHILD D

To submit this job you only have to specify the file describing this DAG to gwdag tool:

$ gwdag <name of the file>

You can also get a DOT file for a DAG description that you can use later to generate a graph showing the flow using -d flag:

$ gwdag -d <name of the file> > <name of the dot file>

Figure 4.2. Dag graph generated by the gwdag tool.