GT 4.2.1 User's Guide

The Java WS Core container is the web services hosting environment based on Java on which the GT higher-level Java web services (such as RFT and CAS) are based.

To start the Java WS Core container in any default installation of GT, run globus-start-container:

$GLOBUS_LOCATION bin/globus-start-container

If you want to run without transport-level security, use the -nosec option:

$GLOBUS_LOCATION bin/globus-start-container -nosec

To stop the container, run:

$GLOBUS_LOCATION bin/globus-stop-container

The following GT components are higher-level web services based on Java WS Core

You can use the wsrf-query command to query any WSRF resource property document. For example, you can use the following command to query the WS MDS Index Service for all the resource properties collected by the default Index Service on your local host:

$GLOBUS_LOCATION/bin/wsrf-query -s http://localhost:8443/wsrf/services/DefaultIndexService '/*'

The globus-wsc-container command is an implementation of a Web Service container for hosting services written in C. By default, the container will run in the foreground and process SOAP requests until terminated by a signal. See globus-wsc-container documentation for a complete list of command-line options.

% globus-wsc-container

Contact: https://grid.example.org:8443/

There is no special command for stopping a C container. If the command is in the foreground (default), then sending the TERM signal (typically ctrl-C).

% globus-wsc-container

Contact: https://grid.example.org:8443

^C

Execution cancelled, cleaning up.

% 

If the container is in the background, it can be terminated with the POSIX-standard kill command. If the container was started with the -pidfile command-line option, that fill can be read to determine which process to kill. For example:

% globus-wsc-container -bg -pidfile $GLOBUS_LOCATION/var/wsc.pid

Contact: https://grid.example.org:8443

% cat $GLOBUS_LOCATION/var/wsc.pid

19773

% kill 19773

% 

The container will automatically remove the PID file ($GLOBUS_LOCATION/var/wsc.pid in this example).

WSRF services share information on resource state through resource properties. C WS Core provides several tools for inspecting these properties. A list of the properties provided by Globus Toolkit services is available in the developer's guide.

The globus-wsrf-get-property and globus-wsrf-get-properties commands provide two options for getting the value of a single resource property or multiple resource properties, respectively. For this example, we'll explore some of the properties provided by the GRAM4 service.

First, we'll check the version information of a GRAM4 service using globus-wsrf-get-property:

% globus-wsrf-get-property -s https://grid.example.org:8443/wsrf/services/ManagedJobFactoryService \
    '{http://mds.globus.org/metadata/2005/02}ServiceMetaDataInfo'

<ns1:ServiceMetaDataInfo xmlns:ns1="http://mds.globus.org/metadata/2005/02">
    <ns1:startTime>2008-06-19T16:34:31.248Z</ns1:startTime>
    <ns1:version>4.1.0</ns1:version>
    <ns1:serviceTypeName>ManagedJobFactoryService</ns1:serviceTypeName>
</ns1:ServiceMetaDataInfo>

% 

Now, we'll check for some system-specific information using globus-wsrf-get-properties:

% globus-wsrf-get-properties -s https://grid.example.org:8443/wsrf/services/ManagedJobFactoryService \
    '{http://www.globus.org/namespaces/2004/10/gram/job}hostCPUType' \
    '{http://www.globus.org/namespaces/2004/10/gram/job}hostOSName'


<ns1:hostCPUType xmlns:ns1="http://www.globus.org/namespaces/2008/03/gram/job">i686</ns1:hostCPUType>
<ns2:hostOSName xmlns:ns2="http://www.globus.org/namespaces/2008/03/gram/job">Linux</ns2:hostOSName>


% 

The globus-wsrf-query-propery program can be used to perform more sophisticated queries of resource properties using XPath expressions. We can check for the number of local resource managers supported by this installation:

% globus-wsrf-query \
    -s https://grid.example.org:8443/wsrf/services/ManagedJobFactoryService \
    'count(//*[local-name() = "availableLocalResourceManagers"])'

2
% 

We can then get the names of the local resource managers:

% globus-wsrf-query \
    -s https://grid.example.org:8443/wsrf/services/ManagedJobFactoryService \
    '//*[local-name() = "availableLocalResourceManagers"]/*[1]/text()'

Fork

% globus-wsrf-query \
    -s https://grid.example.org:8443/wsrf/services/ManagedJobFactoryService \
    '//*[local-name() = "availableLocalResourceManagers"]/*[2]/text()'

Multi

% 

Before using many of the tools in GT, a user must generate a valid user proxy. Use grid-proxy-init. The following is an example:

% $GLOBUS_LOCATION/bin/grid-proxy-init
Your identity: /O=Grid/OU=GlobusTest/OU=simpleCA.mymachine/OU=mymachine/CN=John Doe
Enter GRID pass phrase for this identity:
Creating proxy ................................. Done
Your proxy is valid until: Tue Oct 26 01:33:42 2004

If you just want the "rules of thumb" on getting started (without all the details), the following options using globus-url-copy will normally give acceptable performance:

globus-url-copy -vb -tcp-bs 2097152 -p 4 source_url destination_url

where:

The source/destination URLs will normally be one of the following:

1.1.1. Putting files

One of the most basic tasks in GridFTP is to "put" files, i.e., moving a file from your file system to the server. So for example, if you want to move the file /tmp/foo from a file system accessible to the host on which you are running your client to a file name /tmp/bar on a host named remote.machine.my.edu running a GridFTP server, you would use this command:

globus-url-copy -vb -tcp-bs 2097152 -p 4 file:///tmp/foo gsiftp://remote.machine.my.edu/tmp/bar

	Note
	In theory, remote.machine.my.edu could be the same host as the one on which you are running your client, but that is normally only done in testing situations.

1.1.2. Getting files

A get, i.e, moving a file from a server to your file system, would just reverse the source and destination URLs:

	Tip
	Remember file: always refers to your file system.

globus-url-copy -vb -tcp-bs 2097152 -p 4 gsiftp://remote.machine.my.edu/tmp/bar file:///tmp/foo

globus-url-copy -vb -tcp-bs 2097152 -p 4 gsiftp://other.machine.my.edu/tmp/foo gsiftp://remote.machine.my.edu/tmp/bar

Pipelining allows the client to have many outstanding, unacknowledged transfer commands at once. Instead of being forced to wait for the "Finished response" message, the client is free to send transfer commands at any time.

Pipelining is enabled by using the -pp option:

globus-url-copy -pp

GridFTP Where There Is FTP (GWTFTP) is an intermediate program that acts as a proxy between existing FTP clients and GridFTP servers. Users can connect to GWFTP with their favorite standard FTP client, and GWFTP will then connect to a GridFTP server on the client’s behalf. To clients, GWFTP looks much like an FTP proxy server. When wishing to contact a GridFTP server, FTP clients instead contact GWTFTP.

Clients tell GWFTP their ultimate destination via the FTP USER <username> command. Instead of entering their username, client users send the following:

USER <GWTFTP username>::<GridFTP server URL>

This command tells GWTFTP the GridFTP endpoint with which the client wants to communicate. For example:

USER bresnaha::gsiftp://wiggum.mcs.anl.gov:2811/

	Note
	Requires GSI C security.

To transfer a single file to many destinations in a multicast/broadcast, use the new -mc option.

	Note
	To use this option, the admin must enable multicasting. Click here for more information.

globus-url-copy -vb -tcp-bs 2097152 -p 4 -mc filename source_url

The filename must contain a line-separated list of destination urls. For example:

gsiftp://localhost:5000/home/user/tst1
gsiftp://localhost:5000/home/user/tst3
gsiftp://localhost:5000/home/user/tst4
 

For more flexibility, you can also specify a single destination url on the command line in addition to the urls in the file. Examples are:

globus-url-copy -MC multicast.file gsiftp://localhost/home/user/src_file

or

globus-url-copy -MC multicast.file gsiftp://localhost/home/user/src_file gsiftp://localhost/home/user/dest_file1

gsiftp://dst1.domain.com:5000/home/user/tst1?cc=1;tcpbs=10M;P=4

Beginning with 4.2.0, RFT also offers a new C client, globus-crft.

    gsiftp://localhost:2811/etc/group  gsiftp://localhost:2811/tmp/test_crft
    gsiftp://ftp.globus.org:2811/pub/README gsiftp://myhost.here/home/user/file

    % globus-crft -c -s -m -vb -f <transfer file> -e <container contact string>

2.1.2. Non-blocking Transfer

The client can do non-blocking RFT submission. It can submit an RFT request and then terminate, returning later to monitor the status of the request. To accomplish this the client saves the EPR of the newly created RFT service to disk.

% globus-crft -c -s -f <transfer file> -e <container contact string> \
                -ef <epr output file>

At some point later the client uses this same file to monitor the state of the transfer:

% globus-crft -ef <epr input file> --getOverallStatus

	Note
	Note that in both cases the option -ef is used. In the first case, since the -c option is used, we are creating a new service and the -ef option is a pointer to an output file. In all cases where -c is not used, the -ef switch is a pointer to an input file.

% globus-crft -ef <epr input file> --destroy

globus-crft --help

To check whether your server is active you may use the globus-rls-admin(1) ping command.

% $GLOBUS_LOCATION/bin/globus-rls-admin -p rls://localhost
ping rls://localhost: 0 seconds
        

When the RLS server is first installed its database of replica location information will be empty, as expected. To create a replica location mapping, use the globus-rls-cli(1) create command. Replica information in RLS is represented as mappings from logical names to target names. Typically, the logical name will be a unique identifier for a given replicated data set and the target name will be a URL identifying a particular replica of the data set.

% $GLOBUS_LOCATION/bin/globus-rls-cli create my-logical-name-1 url-for-target-name-1 rls://localhost
        

	Note
	The create command is intended for creating the initial replica mapping entry for a given logical name. If the user attempts to create another entry using an existing logical name, RLS will report a user error. To map additional target names to an existing logical name, see Section 4, “Adding replica location mappings”.

To map additional target names to a logical name created by the previously described create command, use the globus-rls-cli(1) add command.

% $GLOBUS_LOCATION/bin/globus-rls-cli add my-logical-name-1 url-for-target-name-2 rls://localhost
        

Once your RLS server is populated with replica location mappings, you can query the server for useful information using the globus-rls-cli(1) query command.

% $GLOBUS_LOCATION/bin/globus-rls-cli query lrc lfn my-logical-name-1 rls://localhost
my-logical-name-1: url-for-target-name-1
my-logical-name-1: url-for-target-name-2
        

To remove unwanted replica location mappings from your RLS server, use the globus-rls-cli(1) delete command. The delete operation works directly on the mapping and indirectly on the logical and target names. When the delete operation is performed by the RLS server the association between the specified logical name and the specified target name is eliminated. However, there may still be other target names associated with the logical name, and there could still be other logical names associated with the target name, though the latter scenario is less likely. Only when all mapping associations for a given logical name (or a given target name) are eliminated (i.e., the specified logical name has no target names associated with it) will the logical (or target) name be deleted from the RLS server.

% $GLOBUS_LOCATION/bin/globus-rls-cli delete my-logical-name-1 url-for-target-name-1 rls://localhost
% $GLOBUS_LOCATION/bin/globus-rls-cli query lrc lfn my-logical-name-1 rls://localhost
my-logical-name-1: url-for-target-name-2
% $GLOBUS_LOCATION/bin/globus-rls-cli delete my-logical-name-1 url-for-target-name-2 rls://localhost
% $GLOBUS_LOCATION/bin/globus-rls-cli query lrc lfn my-logical-name-1 rls://localhost
globus_rls_client: LFN doesn't exist: my-logical-name-1
        

The globus-rls-cli(1) supports a variety of bulk operations that enhance productivity for users and reduce network connection overhead from making multiple, separate invocations of the client. The general pattern for bulk operation support as implemented by the client is a parameter list consisting of bulk command-name [command-modifiers] param-1 param-2 param-N, such as bulk query lrc lfn my-logical-name-1 my-logical-name-2 my-logical-name-3.

% $GLOBUS_LOCATION/bin/globus-rls-cli bulk create my-logical-name-1 url-for-target-name-1-1 my-logical-name-2 url-for-target-name-2-1 rls://localhost
% $GLOBUS_LOCATION/bin/globus-rls-cli bulk add my-logical-name-1 url-for-target-name-1-2 my-logical-name-2 url-for-target-name-2-2 rls://localhost
% $GLOBUS_LOCATION/bin/globus-rls-cli bulk query lrc lfn my-logical-name-1 my-logical-name-2 my-logical-name-3 rls://localhost
my-logical-name-3: LFN doesn't exist
my-logical-name-2: url-for-target-name-2-1
my-logical-name-2: url-for-target-name-2-2
my-logical-name-1: url-for-target-name-1-1
my-logical-name-1: url-for-target-name-1-2
        

The globus-rls-cli(1) supports an interactive mode in addition to the general command-line mode. To enter the interactive mode, simply invoke the client without any command.

% $GLOBUS_LOCATION/bin/globus-rls-cli rls://localhost
rls> query lrc lfn my-logical-name-2
my-logical-name-2: url-for-target-name-2-1
my-logical-name-2: url-for-target-name-2-2
rls> query lrc lfn my-logical-name-1
my-logical-name-1: url-for-target-name-1-1
my-logical-name-1: url-for-target-name-1-2
rls> bulk delete my-logical-name-1 url-for-target-name-1-1 my-logical-name-1 
url-for-target-name-1-2 my-logical-name-2 url-for-target-name-2-1 
my-logical-name-2 url-for-target-name-2-2
rls> bulk query lrc lfn my-logical-name-2 my-logical-name-1
my-logical-name-1: LFN doesn't exist
my-logical-name-2: LFN doesn't exist
rls> exit
        

Use the globus-replicalocation-createmappings(1) tool to create mappings.

% $GLOBUS_LOCATION/bin/globus-replicalocation-createmappings \
  -s https://localhost:8443/wsrf/services/ReplicaLocationCatalogService \
  mydata1 gsiftp://path/a/to/mydata1
        

No output is expect from this command when successful.

Use the globus-replicalocation-addmappings(1) tool to add mappings.

% $GLOBUS_LOCATION/bin/globus-replicalocation-addmappings \
  -s https://localhost:8443/wsrf/services/ReplicaLocationCatalogService \
  mydata1 gsiftp://path/b/to/mydata1
        

No output is expect from this command when successful.

Use the globus-replicalocation-defineattributes(1) tool to define attribute definitions.

% $GLOBUS_LOCATION/bin/globus-replicalocation-defineattributes \
  -s https://localhost:8443/wsrf/services/ReplicaLocationCatalogService \
  myattr1 logical string
        

No output is expect from this command when successful.

Use the globus-replicalocation-addattributes(1) tool to add attributes.

% $GLOBUS_LOCATION/bin/globus-replicalocation-addattributes \
  -s https://localhost:8443/wsrf/services/ReplicaLocationCatalogService \
  mydata1 myattr1 logical string attribute-value-goes-here
        

No output is expect from this command when successful.

Use the wsrf-query tool to query mappings.

% $GLOBUS_LOCATION/bin/wsrf-query \
  -s https://localhost:8443/wsrf/services/ReplicaLocationCatalogService \
  "query-target: mydata1" \
  "http://globus.org/replica/location/06/01/QueryDialect"
<ns1:MappingStatusType ns1:logical="mydata1" 
ns1:target="gsiftp://path/a/to/mydata1" 
xmlns:ns1="http://www.globus.org/namespaces/2005/08/replica/location"/>
<ns1:MappingStatusType ns1:logical="mydata1" 
ns1:target="gsiftp://path/b/to/mydata1" 
xmlns:ns1="http://www.globus.org/namespaces/2005/08/replica/location"/>
        

Use the wsrf-query tool to query attributes.

% $GLOBUS_LOCATION/bin/wsrf-query \
  -s https://localhost:8443/wsrf/services/ReplicaLocationCatalogService \
  "query-logical-attributes: mydata1" \
  "http://globus.org/replica/location/06/01/QueryDialect"
<ns1:AttributeStatusType ns1:key="mydata1" ns1:name="myattr1"
 ns1:objtype="logical" ns1:status="attributeExists" ns1:valtype="string"
 xmlns:ns1="http://www.globus.org/namespaces/2005/08/replica/location">
 <_value xmlns="">attribute-value-goes-here</_value>
</ns1:AttributeStatusType>
        

A key parameter for any replication request is the request file. The replication request file is a text file containing CRLF-terminated rows of tab-delimited pairs of Logical Filename (LFN) names and destination (URL) locations. An example of such a file is shown.

% cat testrun.req
testrun-1      gsiftp://myhost:9001/sandbox/files/testrun-1
testrun-2      gsiftp://myhost:9001/sandbox/files/testrun-2
testrun-3      gsiftp://myhost:9001/sandbox/files/testrun-3
testrun-4      gsiftp://myhost:9001/sandbox/files/testrun-4
testrun-5      gsiftp://myhost:9001/sandbox/files/testrun-5
        

Note

The LFNs in the left column of the request file (e.g., testrun-1, testrun-2, and so on shown in the example) must be registered in the RLS catalog and indexed in the RLS index service. This typically involves using the add RLI command (e.g., globus-rls-admin -a rls-receiver-url rls-sender-url) to tell the RLS to send updates to another (or the same) RLS, and then the create command (e.g., globus-rls-cli create testrun-1 gsiftp://sourcehost:9001/path/to/testrun-1 rls-sender-url) to register the LFN at the RLS catalog service. For details see globus-rls-admin(1) and globus-rls-cli(1).

The initial step for any replication is to create the replication resource. Creating the resource depends on the availability of a Batch Replicator service, a delegated credential, and a properly formatted replication request file. The replication request file must be specified by its URL. Currently supported URL schemes for the request file include file, http, and ftp. If the replication client is run local to the service the file scheme is appropriate, whereas if the client is remote than the latter schemes must be used. It is a good practice to specify a filename to save the replication resource's endpoint reference. The endpoint reference is required for all other operations on the resource, such as getting resource properties, starting/stopping, and destroying it. Numerous options are available to influence the behavior of the data replication activities (see globus-replication-create(1)). One option of particular interest is the --start option, which immediately starts the replication activities following creation of the replication resource. An example of using the globus-replication-create(1) tool is shown.

% $GLOBUS_LOCATION/bin/globus-replication-create -s \
 https://myhost:8443/wsrf/services/ReplicationService \
 -C mycredential.epr -V myreplicator.epr file:///scratch/testrun.req
        

This command does not write to stdout when successful unless the --debug option is specified.

Once a replication resource has been create, the replication activities may be started. As mentioned in Create replication resource the replication may be immediately started after it is created. If the immediate start option is not specified, the globus-replication-start(1) tool must be used to start the replication.

% $GLOBUS_LOCATION/bin/globus-replication-start -e myreplicator.epr
        

No output is expect from this command when successful.

Throughout the lifecycle and after the completion of the replication resource, it will be important to lookup its Resource Properties. The standard WS-RF port types are supported and the supplied tools (e.g., wsrf-get-property) may be used with the Batch Replicator and its resources.

% $GLOBUS_LOCATION/bin/wsrf-get-property -e myreplicator.epr \
 "{http://www.globus.org/namespaces/2005/05/replica/replicator}status"
<ns1:status xmlns:ns1="http://www.globus.org/namespaces/2005/05/replica/replicator">
Active</ns1:status>
        

And,

% $GLOBUS_LOCATION/bin/wsrf-get-property -e myreplicator.epr \ 
 "{http://www.globus.org/namespaces/2005/05/replica/replicator}count"
<ns1:count xmlns:ns1="http://www.globus.org/namespaces/2005/05/replica/replicator">
 <ns1:total>10</ns1:total>
 <ns1:finished>0</ns1:finished>
 <ns1:failed>0</ns1:failed>
 <ns1:terminated>0</ns1:terminated>
</ns1:count>
        

Throughout the lifecycle and after the completion of the replication resource, it may be helpful to find individual replication items in the replication resource to inspect the detailed status of the replication activities. The globus-replication-finditems(1) tool is used to find replication items. The following example demonstrates the usage when finding a limited number of items, offset into the lookup results set, for a specified status.

% $GLOBUS_LOCATION/bin/globus-replication-finditems -e myreplicator.epr -S Pending -O 1 -L 2
<ns1:FindItemsResponse xmlns:ns1="http://www.globus.org/namespaces/2005/05/replica/replicator">
 <ns1:items xsi:type="ns1:ReplicationItemType" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ns1:uri xsi:type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema">testrun-2</ns1:uri>
  <ns1:priority xsi:type="xsd:int" xmlns:xsd="http://www.w3.org/2001/XMLSchema">1000</ns1:priority>
  <ns1:status xsi:type="ns1:ReplicationItemStatusEnumerationType">Pending</ns1:status>
  <ns1:destinations xsi:type="ns1:DestinationType">
   <ns1:uri xsi:type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
     gsiftp://myhost:9001/sandbox/files/testrun-2</ns1:uri>
   <ns1:status xsi:type="ns1:DestinationStatusEnumerationType">Pending</ns1:status>
  </ns1:destinations>
 </ns1:items>
 <ns1:items xsi:type="ns1:ReplicationItemType" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ns1:uri xsi:type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema">testrun-3</ns1:uri>
  <ns1:priority xsi:type="xsd:int" xmlns:xsd="http://www.w3.org/2001/XMLSchema">1000</ns1:priority>
  <ns1:status xsi:type="ns1:ReplicationItemStatusEnumerationType">Pending</ns1:status>
  <ns1:destinations xsi:type="ns1:DestinationType">
   <ns1:uri xsi:type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
     gsiftp://myhost:9001/sandbox/files/testrun-3</ns1:uri>
   <ns1:status xsi:type="ns1:DestinationStatusEnumerationType">Pending</ns1:status>
  </ns1:destinations>
 </ns1:items>
</ns1:FindItemsResponse>
        

When the replication is complete, the replication resource may be destroyed. Destroying the replication resource will help to free up system resources (namely, memory), especially in the case that the replication entailed a large amount of individual replication activities (i.e., many files specified in the replication request file). The standard WS-RF port types are supported and the supplied wsrf-destroy tool may be used to destroy the Batch Replicator resource.

% $GLOBUS_LOCATION/bin/wsrf-destroy -e myreplicator.epr
Destroy operation was successful
        

A typical example of using the default Index Service is with the wsrf-query Java WS Core command. For example:

$GLOBUS_LOCATION/bin/wsrf-query -s https://localhost:8443/wsrf/services/DefaultIndexService '/*'

displays all the resource properties collected by the default Index Service on your local host.

You can also use an XPath query to drill down your search as well as other Java WS Core commands such as wsrf-get-property and wsrf-get-properties. For more information, review the User's Guide.

You can check the resource property availableLocalResourceManagers of a GRAM4 factory service to get that information. Replace host and port in the below example to query against other containers:

[martin@osg-test1 ~]$ globus-wsrf-get-property \
  -s https://osg-test1.unl.edu:8443/wsrf/services/ManagedJobFactoryService \
  "{http://www.globus.org/namespaces/2008/03/gram/job}availableLocalResourceManagers"

The result on that machine is (formatted for better readability) shows that the local resource managers Fork, Multi, Condor and PBS are available:

<ns1:availableLocalResourceManagers 
      xmlns:ns1="http://www.globus.org/namespaces/2008/03/gram/job">
  <ns1:localResourceManager>Fork</ns1:localResourceManager>
  <ns1:localResourceManager>Multi</ns1:localResourceManager>
  <ns1:localResourceManager>Condor</ns1:localResourceManager>
  <ns1:localResourceManager>PBS</ns1:localResourceManager>
</ns1:availableLocalResourceManagers>

A more typical result in a production environment is probably Fork, Multi and just one additional LRM like Condor, PBS or LSF.

You can check the resource property defaultLocalResourceManagers of a GRAM4 factory service to get that information. Replace host and port in the below example to query against other containers:

[martin@osg-test1 ~]$ globus-wsrf-get-property \
  -s https://osg-test1.unl.edu:8443/wsrf/services/ManagedJobFactoryService \
  "{http://www.globus.org/namespaces/2008/03/gram/job}localResourceManager"

The result on that machine shows that PBS is the default local resource managers:

<ns1:localResourceManager xmlns:ns1="http://www.globus.org/namespaces/2008/03/gram/job">
    PBS
</ns1:localResourceManager>

Use the globusrun-ws program to submit a simple job without writing a job description document. Use the -c argument, a job description will be generated assuming the first arg is the executable and the remaining are arguments. For example:

% globusrun-ws -submit -c /bin/touch touched_it
Submitting job...Done.
Job ID: uuid:4a92c06c-b371-11d9-9601-0002a5ad41e5
Termination time: 04/23/2005 20:58 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

Confirm on the server-side that the job worked by verifying the file was touched:

% ls -l ~/touched_it 
-rw-r--r--  1 smartin globdev 0 Apr 22 15:59 /home/smartin/touched_it

% date
Fri Apr 22 15:59:20 CDT 2005

Note: You did not tell globusrun-ws where to run your job, so the default of localhost was used.

Also note, that globusrun-ws destroyed the job after it was fully processed.

We call this kind of job interactive, because globusrun-ws does not return after submission. It subscribes for status update notifications of the job and informs the user about a status change as soon as it changes. Once it gets the information the the job has been fully processed it destroys the job, which means that internal state belonging to the job is cleaned up on the server-side.

A user can request that the output of the program is sent back directly to the client as soon as it's available. This is useful if a user does not want to do additional file staging for a quick job. To enable this, specify the -s option.

[martin@osg-test1 ~]$ globusrun-ws -submit \
    -F https://lucky0.mcs.anl.gov:8443/wsrf/services/ManagedJobFactoryService \
    -s -c /bin/echo hello world!
Delegating user credentials...Done.
Submitting job...Done.
Job ID: uuid:1731f602-22fe-11dd-879c-0013d4c3b957
Termination time: 05/16/3008 04:10 GMT
Current job state: Active
Current job state: CleanUp-Hold
hello world!
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
Cleaning up any delegated credentials...Done.

If you want the output of the job to be written to a local file instead of the terminal you'll have to add the -so option:

[martin@osg-test1 ~]$ globusrun-ws -submit \
    -F https://lucky0.mcs.anl.gov:8443/wsrf/services/ManagedJobFactoryService \
    -s -so job.out -c /bin/echo hello world!
Delegating user credentials...Done.
Submitting job...Done.
Job ID: uuid:1731f602-22fe-11dd-879c-0013d4c3b957
Termination time: 05/16/3008 04:10 GMT
Current job state: Active
Current job state: CleanUp-Hold
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
Cleaning up any delegated credentials...Done.

Check the output in the specified file:

[martin@osg-test1 ws-gram]$ cat job.out 
hello, world!

Note that a GridFTP server must be running on the remote machine (lucky0) to enable streaming.

Note that streaming output adds some overhead to the submission and will probably be significantly slower compared to a job without streaming. An alternative to streaming is to use staging to transport the output of the executable back to the client. This however requires that a GridFTP server is running on the client machine.

Use globusrun-ws to submit the same touch job, but this time tell globusrun-ws to run the job on another machine (lucky0.mcs.anl.gov:8443). A GT4 server with GRAM4 installed must run on that machine and listen on port 8443.

% globusrun-ws -submit \
   -F https://lucky0.mcs.anl.gov:8443/wsrf/services/ManagedJobFactoryService \
   -c /bin/touch touched_it
Submitting job...Done.
Job ID: uuid:3050ad64-b375-11d9-be11-0002a5ad41e5
Termination time: 04/23/2005 21:26 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.

Type globusrun-ws -help to learn the details about the contact string.

The specification of a job to submit is to be written by the user in a job description XML file.

Here is an example of a simple job description:

<job>
    <executable>/bin/echo</executable>
    <argument>this is an example_string </argument>
    <argument>Globus was here</argument>
    <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
    <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
</job>

Tell globusrun-ws to read the job description from a file, using the -f argument:

% bin/globusrun-ws -submit -f simple.xml
    Submitting job...Done.
    Job ID: uuid:c51fe35a-4fa3-11d9-9cfc-000874404099
    Termination time: 12/17/2004 20:47 GMT
    Current job state: Active
    Current job state: CleanUp
    Current job state: Done
    Destroying job...Done.
    

Note the usage of the substitution variable ${GLOBUS_USER_HOME} which resolves to the user home directory.

Here is an example with more job description parameters:

<?xml version="1.0" encoding="UTF-8"?>
<job>
    <executable>/bin/echo</executable>
    <directory>/tmp</directory>
    <argument>12</argument>
    <argument>abc</argument>
    <argument>34</argument>
    <argument>this is an example_string </argument>
    <argument>Globus was here</argument>
    <environment>
        <name>PI</name>
        <value>3.141</value>
    </environment>
    <stdin>/dev/null</stdin>
    <stdout>stdout</stdout>
    <stderr>stderr</stderr>
    <count>2</count>
</job>

Note that in this example, a <directory> element specifies the current directory for the execution of the command on the execution machine to be /tmp, and the standard output is specified as the relative path stdout. The output is therefore written to /tmp/stdout:

% cat /tmp/stdout
    12 abc 34 this is an example_string  Globus was here
    

Instead of specifying the contact string on the command-line, you can also put it in the job description:

<job xmlns:wsa="http://www.w3.org/2005/08/addressing">
    <factoryEndpoint>
      <wsa:Address>
          https://osg-test1.unl.edu:8443/wsrf/services/ManagedJobFactoryService
      </wsa:Address>
    </factoryEndpoint>
    <executable>/bin/date</executable>
</job>

Submit the job with the following command (assuming the above description has been stored in the file job.xml):

% bin/globusrun-ws -submit -f job.xml

	Note
	This time you don't have to specify the -F option.

Note that at this point you didn't specify any local resource manager related information. If a user does not specify anything then the job is run by the default local resource manager, that is defined on the server-side. If an admin e.g. configured Condor as default local resource manager, then the jobs submitted so far will be managed by Condor on the server-side.

Check the section Local resource managers interfaced by a GRAM4 installation to find out which local resource managers are available in a GRAM4 installation and which one is configured as the default.

globusrun-ws -submit \
  -F osg-test1.unl.edu:8443/wsrf/services/ManagedJobFactoryService \
  -Ft Condor \
  -c /bin/date

<job xmlns:wsa="http://www.w3.org/2005/08/addressing"
    xmlns:gram="http://www.globus.org/namespaces/2008/03/gram/job">
    <factoryEndpoint>
      <wsa:Address>
          https://osg-test1.unl.edu:8443/wsrf/services/ManagedJobFactoryService
      </wsa:Address>
      <wsa:ReferenceParameters>
        <gram:ResourceID>Condor</gram:ResourceID>
      </wsa:ReferenceParameters>
    </factoryEndpoint>
    <executable>/bin/date</executable>
</job>

globusrun-ws -submit -f myJob.xml

In order to do file staging one must add specific elements to the job description and delegate credentials appropriately (see Section 2, “Delegating credentials”). The file transfer directives follow the RFT syntax, which allows only for third-party transfers. Each file transfer must therefore specify a source URL and a destination URL. URLs are specified as GridFTP URLs (for remote files) or as file URLs (for files local to the service--these are converted internally to full GridFTP URLs by the service).

For instance, in the case of staging a file in, the source URL would be a GridFTP URL (for instance gsiftp://job.submitting.host:2811/tmp/mySourceFile ) resolving to a source document accessible on the file system of the job submission machine (for instance /tmp/mySourceFile ). At run-time the Reliable File Transfer service used by the MEJS on the remote machine would reliably fetch the remote file using the GridFTP protocol and write it to the specified local file (for instance file:///${GLOBUS_USER_HOME}/my_transfered_file, which resolves to ~/my_transfered_file). Here is how the stage-in directive would look like:

<fileStageIn>
    <transfer>
        <sourceUrl>gsiftp://job.submitting.host:2811/tmp/mySourceFile</sourceUrl>
        <destinationUrl>file:///${GLOBUS_USER_HOME}/my_transfered_file</destinationUrl>
    </transfer>
</fileStageIn>

Note: additional RFT-defined quality of service requirements can be specified for each transfer. See the RFT documentation for more information.

Here is an example job description with file stage-in and stage-out:

<job>
    <executable>my_echo</executable>
    <directory>${GLOBUS_USER_HOME}</directory>
    <argument>Hello</argument>
    <argument>World!</argument>
    <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
    <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
    <fileStageIn>
        <transfer>
            <sourceUrl>gsiftp://job.submitting.host:2811/bin/echo</sourceUrl>
            <destinationUrl>file:///${GLOBUS_USER_HOME}/my_echo</destinationUrl>
        </transfer>
    </fileStageIn>
    <fileStageOut>
        <transfer>
            <sourceUrl>file:///${GLOBUS_USER_HOME}/stdout</sourceUrl>
            <destinationUrl>gsiftp://job.submitting.host:2811/tmp/stdout</destinationUrl>
        </transfer>
    </fileStageOut>
    <fileCleanUp>
        <deletion>
            <file>file:///${GLOBUS_USER_HOME}/my_echo</file>
        </deletion>
    </fileCleanUp>
</job>

Note that the job description XML does not need to include a reference to the schema that describes its syntax. As a matter of fact it is possible to omit the namespace in the GRAM job description XML elements as well. The submission of this job to the GRAM services causes the following sequence of actions:

Submit that job (assuming the description is stored in the file myJob.xml):

globusrun-ws -submit -S -f myJob.xml

The flag -S tells globusrun-ws to delegate credentials so that Gram4 can call the file transfer service RFT on behalf of the submitting user, and that RFT can interact with the gridftp servers on behalf of the submitting user.

If you already delegated credentials (see Delegating credentials for how to delegate a credential) and have an endpoint reference of that delegated credentials stored in the file delegCred.epr and want them to be used for the transfers instead of globusrun-ws delegating new credentials, you can tell globusrun-ws to use your credentials:

globusrun-ws -submit -Sf delegCred.epr -Tf delegCred.epr -f myJob.xml

The -Sf flag tells that the specified credential is to be used by Gram4 to call RFT on behalf of the user, and the -Tf flag tells that the specified credential is to be used by RFT to interact with the GridFTP servers.

If a user has more than one user account on a server and the distinguished name (DN) of the user's certificate is mapped to all these user accounts, a user can specify which local account should be used by GRAM4 for the job submission. By default the first local user account that is defined is used for job submission. If this is not the one that should be used the user must explicitly specify the account to be used. The following dummy job description shows how to do this:

<job>
    <localUserId>stu</localUserId>
    <executable>/bin/date</executable>
    <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
    <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
</job>

To allow for customization of values, such as paths, on a per-job basis; a job description substitution variable named "GLOBUS_JOB_ID" can be used.

For example:

<job>
    <executable>/bin/date</executable>
    <stdout>/tmp/stdout.${GLOBUS_JOB_ID}</stdout>
    <stderr>/tmp/stderr.${GLOBUS_JOB_ID}</stderr>
    <fileStageOut>
        <transfer>
            <sourceUrl>file:///tmp/stdout.${GLOBUS_JOB_ID}</sourceUrl>
            <destinationUrl>gsiftp://mymachine.mydomain.com/out.${GLOBUS_JOB_ID}</destinationUrl>
        </transfer>
    </fileStageOut>
</job>

More information about substitution variables can found here.

Basic support is provided for specifying custom extensions to the job description. There are plans to improve the usability of this feature, but at this time it involves a bit of work.

Specifying the actual custom elements in the job description is trivial. Simply add any elements that you need between the beginning and ending extensions tags at the bottom of the job description as in the following basic example:

    <job>
        <executable>/home/user1/myapp</executable>
        <extensions>
            <myData>
                <flag1>on</flag1>
                <flag2>off</flag2>
            </myData>
        </extensions>
    </job>
    

To handle this data, you will have to alter the appropriate Perl scheduler script (i.e. $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/fork.pm for the Fork scheduler, etc...) to parse the data returned from the $description->extensions() sub.

For more information about extensions see the Extensions section.

The job description XML schema allows for specification of a multijob i.e. a job that is itself composed of several executable jobs, which we will refer to as subjobs (note: subjobs cannot be multijobs, so the structure is not recursive). This is useful for instance in order to bundle a group of jobs together and submit them as a whole to a remote GRAM installation.

Note that no relationship can be specified between the subjobs of a multijob. The subjobs are submitted to job factory services in their order of appearance in the multijob description.

Within a multijob description, each subjob description must come along with an endpoint for the factory to submit the subjob to. This enables the at-once submission of several jobs to different hosts. The factory to which the multijob is submitted acts as an intermediary tier between the client and the eventual executable job factories.

Here is an example of a multijob description:

<?xml version="1.0" encoding="UTF-8"?>
<multiJob xmlns:wsa="http://www.w3.org/2005/08/addressing">
    <factoryEndpoint>
       <wsa:Address>
          https://localhost:8443/wsrf/services/ManagedJobFactoryService
      </wsa:Address>
    </factoryEndpoint>
    <directory>${GLOBUS_LOCATION}</directory>
    <count>1</count>

    <job>
       <factoryEndpoint>
         <wsa:Address>https://localhost:8443/wsrf/services/ManagedJobFactoryService</wsa:Address>
       </factoryEndpoint>
       <executable>/bin/date</executable>
       <stdout>${GLOBUS_USER_HOME}/stdout.p1</stdout>
       <stderr>${GLOBUS_USER_HOME}/stderr.p1</stderr>
       <count>2</count>
    </job>

    <job>
       <factoryEndpoint>
         <wsa:Address>https://localhost:8443/wsrf/services/ManagedJobFactoryService</wsa:Address>
       </factoryEndpoint>
       <executable>/bin/echo</executable>
       <argument>Hello World!</argument>        
       <stdout>${GLOBUS_USER_HOME}/stdout.p2</stdout>
       <stderr>${GLOBUS_USER_HOME}/stderr.p2</stderr>
       <count>1</count>
    </job>
</multiJob>

Submit the multi-job with the following command:

% bin/globusrun-ws -submit -f test_multi.xml
    Delegating user credentials...Done.
    Submitting job...Done.
    Job ID: uuid:bd9cd634-4fc0-11d9-9ee1-000874404099
    Termination time: 12/18/2004 00:15 GMT
    Current job state: Active
    Current job state: CleanUp
    Current job state: Done
    Destroying job...Done.
    Cleaning up any delegated credentials...Done.

	Note
	When you submit a multi-job you don't have to specify the local resource manager, you can do so though. The fact that it's a multi-job is detected on the server-side and the right "local resource manager" Multi is used automatically.

	Note
	In this multi-job description the sub-jobs are submitted to the default local resource manager. If you want them to be submitted to a non-default local resource manager you'll have to specify that in an additional ReferenceParameters element in the factoryEndpoint element of each sub-job. See here for more information about this.

A multijob resource is created by the factory and exposes a set of WSRF resource properties different than the resource properties of an executable job. The state machine of a multijob is also different since the multijob represents the overall execution of all the executable jobs it is composed of.