Now that we have two machines, we can also setup some information services to monitor them together. Let's have cognito register its index service into choate so we can have an aggregated view of the two machines, as described at Building VOs in the MDS documentation:

globus@cognito:~$ vim /usr/local/globus-4.1.3/etc/globus_wsrf_mds_index/hierarchy.xml 
globus@cognito:~$ grep upstream $GLOBUS_LOCATION/etc/globus_wsrf_mds_index/hierarchy.xml

<!-- <upstream> elements specify remote index services that the local index
    Set an upstream entry for each VO index that you wish to participate in.
    <upstream>https://choate.mcs.anl.gov:8443/wsrf/services/DefaultIndexService</upstream>

root@cognito:~# /etc/init.d/globus-4.1.3 restart
Stopping Globus container. PID: 18069
Container stopped
Starting Globus container. PID: 18405

Now I can run some index service clients and check that the registration worked:

cognito % setenv JAVA_HOME /usr/java/j2sdk1.4.2_10/
cognito % setenv ANT_HOME /usr/local/apache-ant-1.6.5/
cognito % setenv PATH $ANT_HOME/bin:$JAVA_HOME/bin:$PATH
cognito % host cognito
cognito.mcs.anl.gov has address 140.221.8.109
cognito % wsrf-query -s https://choate.mcs.anl.gov:8443/wsrf/services/DefaultIndexService '/*' | grep 140.221.8.109 | wc -l
7

So we've got seven entries in the remote index that reference our machine. That means our upstream registration was processed successfully. But what do those entries look like? Here's an example:

      <ns15:Address xmlns:ns15="http://schemas.xmlsoap.org/ws/2004/03/addressing">
https://140.221.8.109:8443/wsrf/services/ManagedJobFactoryService</ns15:Address>

It's hard to read, isn't it? That's an entry in choate that points to the GRAM4 service running on cognito that we just setup. But our life would be easier if we setup WebMDS to visualize the contents of the Index Service. So let's do that next.

	Note
	Notice that I hadn't setup my java variables yet, but the GRAM client above worked just fine. That's because it's written in C, even though it interacts with the java container. Language neutrality is one of the features of webservices.

WebMDS has a dependency on the Tomcat container, so we'll install that now. The recommended version is 5.0.28, which is available from the Apache Tomcat website. We're following the standard install instructions from the WebMDS Admin Guide.

root@cognito:/usr/local# tar xzf jakarta-tomcat-5.0.28.tar.gz 
root@cognito:/usr/local# chown -R globus:globus jakarta-tomcat-5.0.28

Now the globus user can configure WebMDS:

globus@cognito:~$ vim $GLOBUS_LOCATION/lib/webmds/conf/indexinfo
globus@cognito:~$ grep choate /usr/local/globus-4.1.3/lib/webmds/conf/indexinfo
    <value>https://choate.mcs.anl.gov:8443/wsrf/services/DefaultIndexService</value>
globus@cognito:~$ export CATALINA_HOME=/usr/local/jakarta-tomcat-5.0.28
globus@cognito:~$ $GLOBUS_LOCATION/lib/webmds/bin/webmds-create-context-file \
          $CATALINA_HOME/conf/Catalina/localhost
globus@cognito:~$ $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE:   /usr/local/jakarta-tomcat-5.0.28
Using CATALINA_HOME:   /usr/local/jakarta-tomcat-5.0.28
Using CATALINA_TMPDIR: /usr/local/jakarta-tomcat-5.0.28/temp
Using JAVA_HOME:       /usr/java/j2sdk1.4.2_10/

That started Tomcat on port 8080, so now I can browse to the /webmds directory on that port of my machine (http://cognito.mcs.anl.gov:8080/webmds/ but that's behind a firewall. You can visit your own machine, though). Now I can read the info stored in the index in human-readable format. For instance, I can see this:

RFT	140.221.8.31	0 active transfer resources, transferring 0 files.
26.06 KB transferred in 2 files since start of database.

Those two RFT transfers were the one I ran by hand in the RFT section, then the RFT transfer that happened because of my GRAM job that used file staging. I can also see some information about my GRAM services:

GRAM	140.221.8.109	1 queues, submitting to 0 cluster(s) of 0 host(s).

If I click for details, I get:

ComputingElement:
Name: default
UniqueID: default
Info:
TotalCPUs: 1

This works because the GRAM and RFT services are configured to register into the local service automatically. When we edited the hierarchy.xml file to point to choate, all the information started to be cached centrally.

When we setup our second machine, we copied the usercert over to the new machine because the systems did not share a home directory over NFS. There are other solutions for making proxy credentials available, and we'll use MyProxy to setup another way. First, we'll turn choate into a MyProxy server by following the instructions at configuring MyProxy:

root@choate:~# export GLOBUS_LOCATION=/usr/local/globus-4.1.3/
root@choate:~# cp $GLOBUS_LOCATION/etc/myproxy-server.config /etc
root@choate:~# vim /etc/myproxy-server.config 
root@choate:~# diff /etc/myproxy-server.config $GLOBUS_LOCATION/etc/myproxy-server.config
15,21c15,21
< accepted_credentials  "*"
< authorized_retrievers "*"
< default_retrievers    "*"
< authorized_renewers   "*"
< default_renewers      "none"
< authorized_key_retrievers "*"
< default_key_retrievers "none"
---
> #accepted_credentials  "*"
> #authorized_retrievers "*"
> #default_retrievers    "*"
> #authorized_renewers   "*"
> #default_renewers      "none"
> #authorized_key_retrievers "*"
> #default_key_retrievers "none"
root@choate:~# cat $GLOBUS_LOCATION/share/myproxy/etc.services.modifications >> /etc/services 
root@choate:~# tail /etc/services 
binkp           24554/tcp                       # binkp fidonet protocol
asp             27374/tcp                       # Address Search Protocol
asp             27374/udp
dircproxy       57000/tcp                       # Detachable IRC Proxy
tfido           60177/tcp                       # fidonet EMSI over telnet
fido            60179/tcp                       # fidonet EMSI over TCP

# Local services
gsiftp          2811/tcp
myproxy-server  7512/tcp                        # Myproxy server
root@choate:~# cp $GLOBUS_LOCATION/share/myproxy/etc.xinetd.myproxy /etc/xinetd.d/myproxy
root@choate:~# vim /etc/xinetd.d/myproxy 
root@choate:~# cat /etc/xinetd.d/myproxy 
service myproxy-server
{
  socket_type  = stream
  protocol     = tcp
  wait         = no
  user         = root
  server       = /usr/local/globus-4.1.3/sbin/myproxy-server
  env          = GLOBUS_LOCATION=/usr/local/globus-4.1.3 LD_LIBRARY_PATH=/usr/local/globus-4.1.3/lib 
  disable      = no
}
root@choate:~# /etc/init.d/xinetd reload
Reloading internet superserver configuration: xinetd.
root@choate:~# netstat -an | grep 7512
tcp        0      0 0.0.0.0:7512            0.0.0.0:*               LISTEN     

Again, your system may require a different environment variable than LD_LIBRARY_PATH if you're using MacOS X or IRIX

Now we can check the Myproxy User's Guide to see how to load up a credential and retrieve it remotely:

  bacon@choate:~$ export GLOBUS_LOCATION=/usr/local/globus-4.1.3 
bacon@choate:~$ source $GLOBUS_LOCATION/etc/globus-user-env.sh
bacon@choate:~$ grid-proxy-destroy 
bacon@choate:~$ grid-proxy-info 


ERROR: Couldn't find a valid proxy.
Use -debug for further information.

I destroyed my proxy to keep you from being confused. For the rest of this, I'll be using MyProxy.

bacon@choate:~$ myproxy-init  -s choate
Your identity: /O=Grid/OU=GlobusTest/OU=simpleCA-choate.mcs.anl.gov/OU=mcs.anl.gov/CN=Charles Bacon
Enter GRID pass phrase for this identity:****
Creating proxy .............................................. Done
Proxy Verify OK
Your proxy is valid until: Wed Nov 23 09:48:55 2005
Enter MyProxy pass phrase:******
Verifying - Enter MyProxy pass phrase:******
A proxy valid for 168 hours (7.0 days) for user bacon now exists on choate.
bacon@choate:~$ grid-proxy-info 


ERROR: Couldn't find a valid proxy.
Use -debug for further information.

I got tired of tcsh and switched over to bash.

So what happened? I just loaded a 7 day credential into the MyProxy server on choate. For the next seven days, I'll be able to create proxies from there using the password I supplied as the MyProxy pass phrase. I'll show you what it looks like from cognito:

bacon@cognito:~$ export GLOBUS_LOCATION=/usr/local/globus-4.1.3
bacon@cognito:~$ source $GLOBUS_LOCATION/etc/globus-user-env.sh
bacon@cognito:~$ myproxy-logon -s choate.mcs.anl.gov
Enter MyProxy pass phrase:******
A proxy has been received for user bacon in /tmp/x509up_u1817.
bacon@cognito:~$ grid-proxy-info 
subject  : /O=Grid/OU=GlobusTest/OU=simpleCA-choate.mcs.anl.gov/OU=mcs.anl.gov/CN=Charles Bacon/CN=1390227170/CN=2137426425/CN=87430171
issuer   : /O=Grid/OU=GlobusTest/OU=simpleCA-choate.mcs.anl.gov/OU=mcs.anl.gov/CN=Charles Bacon/CN=1390227170/CN=2137426425
identity : /O=Grid/OU=GlobusTest/OU=simpleCA-choate.mcs.anl.gov/OU=mcs.anl.gov/CN=Charles Bacon
type     : Proxy draft (pre-RFC) compliant impersonation proxy
strength : 512 bits
path     : /tmp/x509up_u1817
timeleft : 11:58:41

And that's how MyProxy works. It turns out that I didn't need to copy my usercert to cognito at all, because I could've stored it in the MyProxy server to begin with.