Status of Current Data Grid Work

We have identified the following items as deliverables for our current work on basic infrastructure to support the Data Grid. As items are completed, we will add hyperlinks to this page.

NOTE: A set of GSI-enabled FTP tools is currently available for those who wish to get started experimenting with them. Versions with a full complement of protocol extensions will be made available later. Answers to frequently-asked questions are also available.

1. Documentation

1.1. White paper explaining our initiative [draft]
1.1b. Answers to frequently-asked questions [available now]
1.2. Draft RFC for GridFTP protocol
     Document the subset of standard FTP features used by globus_ftp
     Document GridFTP protocol extensions for
        Partial file transfer
        Autonegotiation of TCP buffer/window size
        Striped/parallel transfer
1.3. API and program documentation (for all deliverables) using doxygen
     [revised globus_io C API]
     [globus_ftp_control C API]
     [globus_ftp_client C API]
     [globus_gass_copy C API]
     [globus_replica catalog C API]
     [globus_replica management C API]
1.4. Updates to web pages (for all deliverables)
1.5. Updates to tutorials
1.6. Updates to manuals (System Administrator's Guide, Quick Start Guide)
1.7. "Getting Started" Guide for Globus Replica Catalog

2. Production Libraries

2.1. globus_ftp_control (C) (Java, client only): Basic library for managing GridFTP control and data channels, both for clients and servers.
     [status: Alpha testing all features except partial file transfer]

2.2. globus_ftp_client (C)(Java): Easy to use GridFTP client library, supporting:
        Put and get operations, both to/from client memory
        Third party transfer operation
        Fault detection and recovery, using a plugin to define the recovery behavior
     [status: Alpha testing all features except partial file transfer]

2.3. At least one reliability plug-in for globus_ftp_client (C and Java)

2.4. globus_gass_transfer client driver for GridFTP (C): Add FTP/GridFTP support to GASS, in addition to the current HTTP/HTTPS support
     [status: in progress]

2.5. globus_gass_copy (C)(Java): Simple copy operations between any combination of:
        http/https
        ftp/gridftp
        file
        file descriptor (for stdin/stdout copies)
     [status: Alpha testing]

2.6. globus_replica_catalog (C)( Java): LDAP client for manipulating replica catalogs
     [status: Alpha testing]

2.7. globus_replica_manager (C)( Java): Combine globus_replica_catalog with globus_gass_copy to provide end-to-end replication operations
     [status: C API bindings available (see above)]

2.8. (optional) More general (multi-protocol) 3^rd party transfer client library, with custom servers (e.g., DPSS to/from GridFTP) (C)( Java)

2.9. (optional) globus_ftp_mputget (C)(Java): multiple-file put/get to local files

2.10. (optional) globus_gass_transfer server driver for GridFTP (C)

2.11. (optional) SWIG support to all libraries: Enable scripting with tcl, python, perl

2.12. Revised globus_io library with support for new GridFTP code.
[status: Alpha testing]

3. Production Tools

3.1. globus-url-copy (program interface to globus_gass_copy library)
[status: Alpha testing]

3.2. Replica catalog management tools (program interfaces to globus_replica_* libraries)
[status: Alpha testing]

3.3. (optional) globus-ftp-server (simple, user runnable GridFTP server)

4. Third-Party Software with Protocol Extensions

4.1. NCFTP client
    [status: a GSI-enabled version of ncftp is available]
4.2. WUFTPD server
    [status: a GSI-enabled version of wuftpd is available]
4.3. UNITREE ftpd server
    [status: testing]
4.4. HPSS pftpd server
    [status: testing]

5. Prototypes that Demonstrate Functionality

For consideration: Can some of these (e.g., SRB prototypes) include simple authorization capability?

5.1. Custom storage managers
5.1.1. Simple SRB server (C)
5.1.2. Simple HRM server (C)
5.1.3. Simple DPSS servers (C)

5.2. Custom storage client
5.2.1. Simple SRB client (C)

5.3. Fault tolerant GridFTP mget/mput client, which waits for file availability before downloading (Java)