|
Home -> Toolkit -> Docs -> 4.2 -> 4.2.0 -> Execution -> Gridway -> Rn
Could this page be better? Send us your feedback. |
|---|
|
Home -> Toolkit -> Docs -> 4.2 -> 4.2.0 -> Execution -> Gridway -> Rn
Could this page be better? Send us your feedback. |
|---|
Table of Contents
The GridWay Metascheduler enables large-scale, reliable and efficient sharing of computing resources (clusters, computing farms, servers, supercomputers...), managed by different LRM (Local Resource Management) systems, such as PBS, SGE, LSF, Condor..., within a single organization (enterprise grid) or scattered across several administrative domains (partner or supply-chain grid).
GridWay implements several state-of-the-art Grid-aware scheduling policies, comprising job prioritization policies (fixed priority, urgency, share, deadline and waiting-time) and resource prioritization policies (fixed priority, usage, failure and rank).
These policies are combined with:
Adaptive Scheduling, to periodically adapt the schedule considering applications' demands and Grid resource characteristics.
Adaptive Execution to migrate running applications in terms of resource availability, capacity or cost, and new application requirements or preferences.
GridWay interfaces infrastructures with different middleware stacks. With GridWay, users can access heterogeneous resources in a transparent way. For example, it can access resources configured with both GRAM pre web services and GRAM web services. It also permits the use of different grids with different software stacks, for example, Teragrid with Globus and EGEE with gLite.
GridWay supports multiple-user operation mode, and does not require additional middleware installation (apart from standard Globus services). Globus installation is not required in each end-user system.
GridWay allows different Grid deployment strategies, like Enterprise Grids, Partner Grids or Utility Grids.
GridWay executes different Grid application profiles:
Array (Bulk) jobs, for parameter sweep applications
DAG Workflows
Single-site MPI applications
GridWay is able to detect several problems that can occur when executing a remote job. It also implements mechanisms that make the execution more reliable. It can detect a remote system crash, a job failure (via the job exit code) or even a network disconnection (using the polling mechanism) and migrate the problematic job to another resource.
GridWay also performs periodic saves of its state in order to recover from local failure.
GridWay provides detailed statistics of Grid usage. In this way, the Grid administrator can properly plan usage policies and forecast workload. In addition, these statistics can be used by the scheduler to predict (per user) Grid resource response time.
GridWay is an open-source project, flexible and completely based on standards to leverage its usability and interoperability. For example, users can describe their jobs using JSDL. Similarly, programmers can build grid enabled applications using the DRMAA standard.
GridWay provides end-users with a familiar environment similar to that found on classical LRM systems. So GridWay CLI eases the adoption of Grid technologies.
GridWay version shipped with GT4.2.0 is based on GridWay 5.4, a new stable version with increased functionality, including support to access the NorduGrid infrastructure, new bindings for DRMAA and support for DAGMAN workflows.
New to 4.2.0:
Following the trend from previous releases, GridWay is capable of integrate with another major infrastructure: NorduGrid. Drivers for Execution and Transfer are included in this release.
Previous GridWay releases come with support for C and Java bindings implementing the DRMAA standard. Scripting languages are interesting from the point of view of rapid application prototyping among others. From dsa-research.org we believe this to be an interesting aspect, and that is why GridWay includes bindings to access the DRMAA standard from scripting languages such as Perl, Ruby and Python.
There is an increasing interest from the scientific community to be able to run complex workflows on a Grid environment. An almost de-facto standard to represent DAG workflows is Condor DAGMAN. In this development release GridWay includes a new workflow launcher (gwdag) that handles this kind of workflows.
The following problems and limitations are known to exist for GridWay at the time of the 4.2.0 release:
GridWay uses different Globus services to perform the tasks of information gathering, job execution and data transfer.
GridWay depends on the following GT components for job execution:
GridWay depends on the following GT components for data staging:
GridWay depends on the following GT component for information gathering:
GridWay relies on the Globus basic security infrastructure for authentication and authorization. On top of that, GridWay can use other Globus services and components to complement this infrastructure:
Tested platforms for GridWay:
GridWay builds successfully for the following platforms:
In addition, GridWay has been tested with the major Grid infrastructures. Click the following links to find more information on how to use GridWay with EGEE,TeraGrid,OSG and NorduGrid.
GT 4.2.0 includes a new stable release, GridWay 5.4.0, that ships with functionality additions focused in extending usability for the end users. The following information regards compatibility with the previous stable version included in GT 4.0.7:
API changes since 4.0.7:
CLI changes since 4.0.7:
Configuration interface changes since 4.0.7:
GridWay is an open project, flexible and completely based on standards to leverage its usability and interoperability. GridWay supports several standards developed by the Open Grid Forum., namely:
The DRMAA working group has developed an API specification for the submission and control of jobs to one or more Distributed Resource Management (DRM) systems. The goal is to facilitate the direct interfacing of applications to today's DRM systems by application's builders, portal builders, and Independent Software Vendors (ISVs). The Distributed Resource Management Application API (DRMAA) provides a generalized API to distributed resource management systems (DRMSs) in order to facilitate integration of application programs.
The scope of DRMAA is limited to job submission, job monitoring and control, and retrieval of the finished job status. DRMAA provides application developers and distributed resource management builders with a programming model that enables the development of distributed applications tightly coupled to an underlying DRMS. DRMAA preserves flexibility and choice in system design.
GridWay provides support for the DRMAA API (JAVA and C bindings) to develop distributed applications. OGF document GFD.104: GridWay DRMAA 1.0 Implementation - Experience Report provides details of the implementation.
The goal of the JSDL working group is to produce a language that describes the requirements of jobs for submission to Grids. JSDL 1.0, published as OGF recommendation GFD-R-P.056 is an XML-based language that focuses mainly on computational jobs. The JSDL-WG is working on extending this language to address a wider class of jobs, including Web service invocations.
Last GridWay release provides support for JSDL, the POSIX Application profile and HTC Profile schemas can be used to describe jobs.
Click here for more information about this component.