Alas, it's not much of a grid with just one machine. So let's start up on another machine and add it to this little test grid. For a change of pace, I'm going to use the binary installer on this machine. First, though, let's get some prereqs out of the way:

root@cognito:~# adduser globus
root@cognito:~# mkdir /usr/local/globus-4.1.3
root@cognito:~# chown globus:globus /usr/local/globus-4.1.3
root@cognito:/usr/java# ./j2sdk-1_4_2_10-linux-i586.bin 
root@cognito:/usr/local# tar xzf apache-ant-1.6.5-bin.tar.gz 

Then, as user globus:

globus@cognito:~$ tar xzf gt4.1.3-ia32_debian_3.1-binary-installer.tar.gz
globus@cognito:~$ export JAVA_HOME=/usr/java/j2sdk1.4.2_10/
globus@cognito:~$ export ANT_HOME=/usr/local/apache-ant-1.6.5/
globus@cognito:~$ export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$PATH

	Note
	You might notice that I didn't install Postgres on this machine. That's because my grid can actually share the services of the RFT located on my first machine. Even if I weren't planning on that, I could add this new machine to the pg_hba.conf on the first machine and re-use the existing DB server.

Now we can install from binaries:

globus@cognito:~/gt4.1.3-ia32_debian_3.1-binary-installer$ ./configure \
   --prefix=/usr/local/globus-4.1.3
checking for javac... /usr/java/j2sdk1.4.2_10//bin/javac
checking for ant... /usr/local/apache-ant-1.6.5//bin/ant
configure: creating ./config.status
config.status: creating Makefile
globus@cognito:~/gt4.1.3-ia32_debian_3.1-binary-installer$ make
cd gpt-3.2autotools2004 && OBJECT_MODE=32 ./build_gpt
...
Binaries are much faster!  This is done in less than 10 minutes.
...
tar -C /usr/local/globus-4.1.3 -xzf binary-trees/globus_wsrf_rft_test-*/*.tar.gz
tar -C /usr/local/globus-4.1.3 -xzf binary-trees/globus_rendezvous-*/*.tar.gz
echo "Your build completed successfully.  Please run make install."
Your build completed successfully.  Please run make install.
globus@cognito:~/gt4.1.3-ia32_debian_3.1-binary-installer$ make install
ln -s /usr/local/globus-4.1.3/etc/gpt/packages /usr/local/globus-4.1.3/etc/globus_packages
...
config.status: creating fork.pm
..Done

Now let's get security setup on the second machine. We're going to just add trust for the original simpleCA to this new machine, there's no need to create a new one. This is the multiple machines section of the Installing SimpleCA.

globus@cognito:~$ scp choate:.globus/simpleCA/globus_simple_ca_ebb88ce5_setup-0.18.tar.gz .
globus@cognito:~$ export GLOBUS_LOCATION=/usr/local/globus-4.1.3
globus@cognito:~$ $GLOBUS_LOCATION/sbin/gpt-build globus_simple_ca_ebb88ce5_setup-0.18.tar.gz 
gpt-build ====> CHECKING BUILD DEPENDENCIES FOR globus_simple_ca_ebb88ce5_setup
gpt-build ====> Changing to /sandbox/globus/BUILD/globus_simple_ca_ebb88ce5_setup-0.18/
gpt-build ====> BUILDING globus_simple_ca_ebb88ce5_setup
gpt-build ====> Changing to /sandbox/globus/BUILD
gpt-build ====> REMOVING empty package globus_simple_ca_ebb88ce5_setup-noflavor-data
gpt-build ====> REMOVING empty package globus_simple_ca_ebb88ce5_setup-noflavor-dev
gpt-build ====> REMOVING empty package globus_simple_ca_ebb88ce5_setup-noflavor-doc
gpt-build ====> REMOVING empty package globus_simple_ca_ebb88ce5_setup-noflavor-pgm_static
gpt-build ====> REMOVING empty package globus_simple_ca_ebb88ce5_setup-noflavor-rtl
globus@cognito:~$ $GLOBUS_LOCATION/sbin/gpt-postinstall
running /usr/local/globus-4.1.3/setup/globus/./setup-ssl-utils.ebb88ce5..
[ Changing to /usr/local/globus-4.1.3/setup/globus/. ]
...
setup-ssl-utils: Complete

..Done
WARNING: The following packages were not set up correctly:
        globus_simple_ca_ebb88ce5_setup-noflavor-pgm
Check the package documentation or run postinstall -verbose to see what happened

That installed the package, but the warning is letting us know that root still needs to run the setup script:

root@cognito:~# export GLOBUS_LOCATION=/usr/local/globus-4.1.3
root@cognito:~# source $GLOBUS_LOCATION/etc/globus-user-env.sh
root@cognito:~# $GLOBUS_LOCATION/setup/globus_simple_ca_ebb88ce5_setup/setup-gsi -default
setup-gsi: Configuring GSI security
Making /etc/grid-security...
mkdir /etc/grid-security
Making trusted certs directory: /etc/grid-security/certificates/
mkdir /etc/grid-security/certificates/
Installing /etc/grid-security/certificates//grid-security.conf.ebb88ce5...
Running grid-security-config...
nstalling Globus CA certificate into trusted CA certificate directory...
Installing Globus CA signing policy into trusted CA certificate directory...
setup-gsi: Complete

Now our new machine's security directory looks like our other machine:

root@cognito:~# ls /etc/grid-security/
certificates  globus-host-ssl.conf  globus-user-ssl.conf  grid-security.conf
root@cognito:~# ls /etc/grid-security/certificates/
ebb88ce5.0                     globus-user-ssl.conf.ebb88ce5
ebb88ce5.signing_policy        grid-security.conf.ebb88ce5
globus-host-ssl.conf.ebb88ce5

Now we need a hostcert for the new machine:

root@cognito:~# grid-cert-request -host `hostname`
The hostname cognito does not appear to be fully qualified.
Do you wish to continue? [n] n
Aborting
...
If you receive no response, contact Globus Simple CA at bacon@choate
root@cognito:~# hostname
cognito

Uh-oh. Our hostname isn't fully qualified, which is going to cause us trouble down the road. If you have this problem, there are several possible solutions. One is to run the hostname command as root to set your FQDN as your hostname. Another possibility is that your /etc/hosts may have a short name listed for your IP address. Let's see what the problem is on cognito:

root@cognito:~# cat /etc/hosts
127.0.0.1       localhost

# The following lines are desirable for IPv6 capable hosts
# (added automatically by netbase upgrade)

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

That looks okay. On Debian, the hostname is stored in /etc/hostname. Let's see what it says:

root@cognito:~# cat /etc/hostname
cognito

Ah, that's the problem. But this is not so bad, because a reverse-lookup of my IP address should return my FQDN, since it will be looked up in DNS:

root@cognito:~# host 140.221.8.109
109.8.221.140.in-addr.arpa domain name pointer cognito.mcs.anl.gov.

If the problem had been in /etc/hosts, I would have fixed it. Here's what a good /etc/hosts line would look like:

140.221.8.109   cognito.mcs.anl.gov cognito

Since reverse lookups work okay, I will just spell out the FQDN by hand in this cert request:

root@cognito:~# grid-cert-request -host cognito.mcs.anl.gov -force

    /etc/grid-security/hostcert_request.pem already exists
    /etc/grid-security/hostcert.pem already exists
    /etc/grid-security/hostkey.pem already exists
...
Your certificate will be mailed to you within two working days.
If you receive no response, contact Globus Simple CA at bacon@choate

The request already existed for "cognito", but the -force overwrote that request with one for "cognito.mcs.anl.gov". Now I need to copy that back to choate and sign it:

root@cognito:~# cat /etc/grid-security/hostcert_request.pem | mail globus@choate

Now I sign it as globus on choate. Remember, that's where I installed the SimpleCA, so that's where I sign it:

globus@choate:/tmp$ grid-ca-sign -in in.pem -out out.pem

To sign the request
please enter the password for the CA key:

The new signed certificate is at: /home/globus/.globus/simpleCA//newcerts/03.pem
globus@choate:/tmp$  cat /tmp/out.pem | mail root@cognito

Root checks his email, then saves the signed cert:

root@cognito:~# cp out.pem /etc/grid-security/hostcert.pem 
root@cognito:/etc/grid-security# cp hostcert.pem containercert.pem
root@cognito:/etc/grid-security# cp hostkey.pem containerkey.pem
root@cognito:/etc/grid-security# chown globus:globus container*.pem
root@cognito:/etc/grid-security# ls -l *.pem
-rw-r--r--  1 globus globus 2711 2005-11-15 11:14 containercert.pem
-r--------  1 globus globus  887 2005-11-15 11:15 containerkey.pem
-rw-r--r--  1 root   root   2711 2005-11-15 11:14 hostcert.pem
-rw-r--r--  1 root   root   1405 2005-11-15 11:09 hostcert_request.pem
-r--------  1 root   root    887 2005-11-15 11:09 hostkey.pem

There. Now cognito is setup with host and container certs, and it trusts the CA of my grid. The last step for root is to create a grid-mapfile for myself again:

root@cognito:/etc/grid-security# vim grid-mapfile
root@cognito:/etc/grid-security# cat grid-mapfile 
"/O=Grid/OU=GlobusTest/OU=simpleCA-choate.mcs.anl.gov/OU=mcs.anl.gov/CN=Charles Bacon" bacon

Also, user bacon should get a local copy of the usercert:

cognito % scp -r choate:.globus .
Password: 
usercert.pem                                  100%  895     0.9KB/s   00:00    
usercert_request.pem                          100% 1426     1.4KB/s   00:00    
userkey.pem                                   100%  963     0.9KB/s   00:00    

GridFTP setup on the second machine is identical to the first. I'll just list the commands here, see Section 4, “Set up GridFTP” for the file contents, or just copy them from the first machine.

root@cognito:/etc/grid-security# vim /etc/xinetd.d/gridftp
root@cognito:/etc/grid-security# vim /etc/services 
root@cognito:/etc/grid-security# /etc/init.d/xinetd reload
Reloading internet superserver configuration: xinetd.

Now we can test it:

cognito % setenv GLOBUS_LOCATION /usr/local/globus-4.1.3
cognito % source $GLOBUS_LOCATION/etc/globus-user-env.csh
cognito % grid-proxy-init -verify -debug

User Cert File: /home/bacon/.globus/usercert.pem
User Key File: /home/bacon/.globus/userkey.pem

Trusted CA Cert Dir: /etc/grid-security/certificates

Output File: /tmp/x509up_u1817
Your identity: /O=Grid/OU=GlobusTest/OU=simpleCA-choate.mcs.anl.gov/OU=mcs.anl.gov/CN=Charles Bacon
Enter GRID pass phrase for this identity:
Creating proxy ...........++++++++++++
........++++++++++++
 Done
Proxy Verify OK
Your proxy is valid until: Tue Nov 15 23:33:37 2005
cognito % globus-url-copy gsiftp://cognito.mcs.anl.gov/etc/group \
   gsiftp://choate.mcs.anl.gov/tmp/from-cognito

That was a slightly fancier test than I ran on choate. In this case, I did a third-party transfer between two GridFTP servers. It worked, so I have the local and remote security setup correctly.

Setting up the container on the second machine is a lot like the first. I'll list the commands here. See Section 5, “Starting the webservices container”, or you can just copy the files from the first machine. First globus creates the start-stop script:

globus@cognito:~$ vim $GLOBUS_LOCATION/start-stop
globus@cognito:~$ chmod +x $GLOBUS_LOCATION/start-stop

Then root creates an init.d script to call it:

root@cognito:~# vim /etc/init.d/globus-4.1.3
root@cognito:~# chmod +x /etc/init.d/globus-4.1.3
root@cognito:/etc/grid-security# /etc/init.d/globus-4.1.3 start
Starting Globus container. PID: 17269

For a change of pace, we'll setup GRAM first on the second machine, even though we haven't got a working RFT locally. As with last time, we'll need to setup the sudoers. See Section 7, “Setting up GRAM4” for the sudo contents, or copy the sudoers from the first machine.

root@cognito:/etc/grid-security# visudo

Next, however, we'll change the GRAM RFT configuration, using the GRAM docs about setting up "non-default configuration" section of System Administrator's Guide:

globus@cognito:~$ vim $GLOBUS_LOCATION/start-stop

globus@cognito:~$ $GLOBUS_LOCATION/setup/globus/setup-gram-service-common --staging-host=choate.mcs.anl.gov
Running /usr/local/globus-4.1.3/setup/globus/setup-gram-service-common
Determining system information...
...
BUILD SUCCESSFUL
Total time: 21 seconds

Restart the container:

root@cognito:/etc/grid-security# /etc/init.d/globus-4.1.3 restart
Stopping Globus container. PID: 17269
Container stopped
Starting Globus container. PID: 18069

Now we can submit a staging job:

cognito % vim a.rsl
cognito % cat a.rsl
cognito % cat a.rsl

<job>
    <executable>my_echo</executable>
    <directory>${GLOBUS_USER_HOME}</directory>
    <argument>Hello</argument>
    <argument>World!</argument>
    <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
    <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
    <fileStageIn>
        <transfer>
            <sourceUrl>gsiftp://cognito.mcs.anl.gov:2811/bin/echo</sourceUrl>
            <destinationUrl>file:///${GLOBUS_USER_HOME}/my_echo</destinationUrl>
        </transfer>
    </fileStageIn>
    <fileCleanUp>
        <deletion>
            <file>file:///${GLOBUS_USER_HOME}/my_echo</file>
        </deletion>
    </fileCleanUp>
</job>

cognito % globusrun-ws -submit -S -f a.rsl
Delegating user credentials...Done.
Submitting job...Done.
Job ID: uuid:6732f346-5604-11da-9951-0002b3882c16
Termination time: 11/16/2005 18:19 GMT
Current job state: StageIn
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
Cleaning up any delegated credentials...Done.
cognito % cat ~/stdout
Hello World!
cognito % ls ~/my_echo
ls: /home/bacon/my_echo: No such file or directory

This is an example of a staging job. It copies the /bin/echo command from cognito to my home directory and names it my_echo. Then it runs it with some arguments, and captures the stderr/stdout. One of the neat features here is that it used the RFT service on choate to transfer the file via the GridFTP server on cognito. It's starting to look like a Grid!

You can get other examples of GRAM RSL files from GRAM usage scenarios.