Software Links
Getting Started
- Doc Structure
- A Globus Primer
- Globus Is Modular!
- Quickstart
- Installing GT
- Platform Notes
- Migrating from GT2
- Migrating from GT3
Reference
- PDF version
- Best Practices
- Coding Guidelines
- API docs
- Public Interfaces
- Resource Properties
- Samples
- Glossary
- Performance Studies
Common Runtime
Security
Data Mgt
Information Svcs
Execution Mgt
Table of Contents
The WS MDS Index Service collects information about grid resources and publishes that information as a service group. Client programs use resource property queries or subscription/notification to retrieve information from the index. Information can be added to the index via a number of different mechanisms: since the Index Service is implemented using the Aggregator Framework, any aggregator source can be used to provide information for the index.
This document describes the programmatic interfaces to the Index Service. See also general Globus Toolkit coding guidelines and GT 4.1.1 best practices.
Features new in release 4.1.1
Based on WSRF rather than OGSI
- Aggregated data is published through WS-Resource Properties mechanisms
- The Aggregator Framework module (basis of the index service) collects data from monitored resources using WS-Resource Properties collection mechanisms, including the base WS-Resource Properties poll operations and WS-Notification.
- Persistent configuration of aggregations has been refactored. The service no longer has its own config file for specifying aggregations. Instead a separate client is used to make appropriate registrations. This client may be started alongside the container for the equivalent effect. The client may also be deployed elsewhere to support resource-side configuration (providing similar functionality to the GT3.2 RegistryPublishProvider) or at a third location.
Other Supported Features
- The index appears as a ServiceGroup (defined in WSRF) which lists registered resources alongside dynamically collected information from those resources. This information can be examined using (for example) XPath queries to discover resources that match desired constraints.
Deprecated Features
- Support for Xindice and aggregated data persistence has been removed.
- The ability to specify default aggregations in service config file has been removed (see item in 'New Features').
- Non-ServiceGroupRegistration methods for managing aggregations have been removed; the equivalent functionality is available through the ServiceGroupRegistration port type which is available in both GT3.2 and GT 3.9.3.0 versions of the index service.
Tested Platforms for WS-MDS Index Service:
- Linux on i386
- Windows XP
Tested containers for WS-MDS Index Service:
- Java WS Core container
- Tomcat 5.0.28
Protocol changes since GT version 4.0.4
- Generally incompatible with the GT 3.2 index service as the service has been remodelled to use WSRF instead of OGSI.
API changes since GT version 4.0.4
- The Aggregator Framework API used internally has retained the general flavor of the GT 3.2 Aggregator Framework API, but is not directly compatible with it.
Schema changes since GT version 4.0.4
- Schemas used for configuration and publishing are conceptually similar but are not compatible, primarily due to WSRF remodelling.
The Index Service depends on the following GT components:
The Index Service depends on the following 3rd party software:
- None
The security considerations for the Aggregator Framework also apply to the Index Service:
By default, the aggregator sources do not use authentication credentials -- they retrieve information using anonymous SSL authentication or no authentication at all, and thus retrieve only publicly-available information. If a user or administrator changes that configuration so that a service's aggregator source uses credentials to acquire non-privileged data, then that user or administrator must configure the service's aggregator sink to limit access to authorized users.
There are essentially two interfaces to the Index Service -- one for getting information into the index, and one for retrieving information from the index.
Information is retrieved from the Index Service as service group entries using the standard WS MDS Core APIs for resource property queries or subscription/notification.
Because the Index is implemented as a WS MDS Aggregator Framework, the programmatic interface for getting information into the index is to create an aggregator source. The Aggregator Framework's architecture is described in the next section.
The WS MDS Aggregator Framework is the software framework on which WS MDS services are built. The Aggregator Framework collects data from an aggregator source and sends that data to an aggregator sink for processing. Aggregator sources distributed with the Globus Toolkit include modules that query resource properties, acquire data through subscription/notification, and execute programs to generate data. Another way of describing the Aggregator Framework is that it is designed to facilitate the collecting of information from or about WS-Resources via plugin aggregator sources and the feeding of that information to plugin aggregator sinks, which can then perform actions such as re-publishing, logging, or archiving the information.
Aggregators work on a type of service group called an
AggregatorServiceGroupRP. Resources may be
registered to an AggregatorServiceGroupRP using the service
group add operation, which will cause an entry to be
added to the service group. The entry will include configuration
parameters for the aggregator source; when the registration is
made, the appropriate aggregation source and sinks will be
informed; the aggregator source will begin collecting data and
inserting it into the corresponding service group entry, and the
aggregator sink will begin processing the information in the
service group entries.
The method of collection by source and processing by the sink is dependent on the particular instantiation of the aggregator framework.
The aggregator sinks distributed with the toolkit
(org.globus.mds.aggregator.impl.ServiceGroupEntryAggregatorSink
and org.globus.mds.trigger.impl.TriggerResource) are
described in the following table.
Table 1. Standard aggregator sinks
| Aggregator Sink | Service Implemented | Description |
|---|---|---|
ServiceGroupEntryAggregatorSink | Index Service | The servicegroup sink (used by the Index Service) publishes received data as content in the AggregatingServiceGroup entry used to manage the registration. This data can therefore be retrieved by querying the index for its 'entries' resource property. |
TriggerResource | Trigger Service | The Trigger Service provides an aggregator sink which receives data, applies tests to that data, and if the tests match, runs a specified executable. See the WS MDS Trigger Service documentation for more information. |
The aggregator sources supplied with the toolkit collect information using resource property queries (query sources), subscription/notification (subscription sources), and execution of external programs (execution sources).
The aggregator sources supplied with the Globus Toolkit are listed in the following table.
![[Note]](/docbook-images/note.gif) | Note |
|---|---|
All aggregator sources listed in this
table are in the |
Table 2. Standard aggregator sources
| Aggregator Source | Description |
|---|---|
| QueryAggregatorSource | The query source collects information from a registered resource by using WS-Resource Properties polling mechanisms:
Polls are made periodically, with both the period and target Resource Properties specified in the registration message. |
| SubscriptionAggregatorSource | The subscription source collects information from a registered resource using WS-Notification mechanisms. Data is delivered when property values change, rather than periodically. |
| ExecutionAggregatorSource | The execution source collects information about (not necessarily from) a registered resource by execution of a local executable, which is passed as input the identity of the registered resource. Details of the interface between the execution source and local executables are in Execution Aggregator Sources Reference. |
Index Service queries are performed using resource property requests; consult Java WS Core for details.
The contents of an index are maintained using the aggregator framework programming model, and can receive data from any aggregator source. Information about how to configure existing aggregator sources (such as the aggregator sources distributed with the Globus Toolkit, which include one that polls for resource property information, one that collects resource property information through subscription/notification, and one that collects information by executing an executable program) is found in the Aggregator Sources Reference; information about how to create new aggregator sources can be found in Aggregator Developer's Guide.
The index service inherits its WSDL interface from the Aggregator Framework module, included below:
The Aggregator Framework builds on the WS-ServiceGroup and WS-ResourceLifetime specifications. Those specifications should be consulted for details on the syntax of each operation.
Each Aggregator Framework is represented as a WS-ServiceGroup (specifically, an AggregatorServiceGroup).
Resources may be registered to an AggregatorServiceGroup using
the AggregatorServiceGroup Add operation. Each registration will be
represented as a ServiceGroupEntry resource. Resources may be
registered to an AggregatorServiceGroup using the service
group add operation, which will cause an entry to be
added to the service group.
The entry will include configuration parameters for the aggregator source; when the registration is made, the following will happen:
- The appropriate aggregation source and sinks will be informed,
- the aggregator source will begin collecting data and inserting it into the corresponding service group entry,
- and the aggregator sink will begin processing the information in the service group entries.
The method of collection by source and processing by the sink is dependent on the particular instantiation of the aggregator framework (see per-source documentation for source information and per-service documentation for sink information - for example the Index Service and the Trigger Service.)
add:This operation is used to register a specified resource with the Aggregator Framework. In addition to the requirements made by the WS-ServiceGroup specification, the Content element of each registration must be an AggregatorContent type, with the AggregatorConfig element containing configuration information specific to each source and sink (documented in the Aggregator System Administrator's Guide).
Entry:This resource property publishes details of each registered resource, including both an EPR to the resource, the Aggregator Framework configuration information, and data from the sink.RegistrationCount:This resource property publishes registration load information (the total number of registrations since service startup and decaying averages)
The Aggregator Framework throws standard WS-ServiceGroup, WS-ResourceLifetime, and WS-ResourceProperties faults and does not define any new faults of its own.
Other relevant source files are the:
- WSRF service group schema
- WSRF resource lifetime schema
- MDS Usefulrp schema.
The index service exposes information via service groups and is accessed using the same command-line tools used to query other WSRF services for information. These tools are part of Java WS Core.
- wsrf-query performs a query on a resource property document
- wsrf-get-property gets a single resource property
- wsrf-get-properties gets multiple resource properties.
A set of functionally equivalent tools exist written using WS C core. They tend to be faster alternatives to the above java programs.
- globus-wsrf-query(1) performs a query on a resource property document
- globus-wsrf-get-property(1) gets a single resource property
- globus-wsrf-get-properties(1) gets multiple resource properties.
The mds-servicegroup-add(1) command in the Aggregator Framework is used to configure the Index Service to query information from various sources.
The globus-index-add(1) command line tool is written using WS C and offers similar functionality to mds-servicegroup-add(1) with a few new options.
There is no GUI specifically for the Index Service. The release contains WebMDS" which can be used to display monitoring information collected in an Index Service in a normal web browser.
There is no domain-specific interface specific to the Index Service, however, the ExecutionAggregatorSource, which may be used by the Index Service, does have a domain-specific interface (specifically, the inputs provided to and outputs expected from the executable program).
The syntax of the execution source's domain-specific interface is described in Execution Aggregator Sources Reference.
Please see the Configuring section of the System Administrator's Guide.
Information is retrieved from the index using the standard Java WS Core API calls for getting resource property information:
GetResourcePropertyto request a single resource property by name,GetResourcePropertiesto request several resource properties by name,QueryResourcePropertyto perform an XPath query on a resource property document, and- the notification/subscription mechanism.
See the GT 4.1.1 Java WS Core : Developer's Guide for API details.
Information is added to an index by way of an aggregator source. The Globus Toolkit distribution includes several standard aggregator sources (see the Aggregator Sources Reference for more details). To create your own custom information source, see the Aggregator Developer's Guide.
Use of the index service is covered in the Build a Grid Service Tutorial (GlobusWORLD 2005).
See Debug section of the Java WS Core Developer's Guide for general information on logging, including which files to edit to set logging properties.
To turn on debug logging for the Index Service, add the line:
log4j.category.org.globus.mds.index=DEBUG
to the appropriate properties file. Since the Index Service is implemented using the Aggregator Framework, you may also want to turn on aggregator debugging by adding this line:
log4j.category.org.globus.mds.aggregator=DEBUG
General troubleshooting information can be found in the Section 8, “Troubleshooting”.
Specifications for resource properties, service groups, and subscription/notification are available at http://www.globus.org/wsrf/.