Table of Contents
The Globus Toolkit development team is pleased to provide a new incremental release of the Globus Toolkit 4.0. Users who wish to receive the latest bug fixes are encouraged to install this release. Highlights of 4.0.4 include:
- Several bug fixes in all components, itemized below
- Mac OS/X intel support in C WS Core
- Finished integration between the SweGrid Accounting System (SGAS) tech preview and WS GRAM
This page describes only the changes since the 4.0.3 release. For details regarding previous 4.0 releases, see:
To download this release, go to the 4.0.4 Download page.
For information on installing this release, go to the 4.0 Installation Guide.
Use the existing documentation for 4.0, which starts at http://www.globus.org/toolkit/docs/4.0/. For more information about how our new documentation is structured, please see the new Documentation Overview.
An overview on GT support can be found here.
The Globus Toolkit is distributed and licensed for use under the terms of the Apache License, Version 2.0.
To view the licenses of the 3rd party software used by the developers of the Globus Toolkit, click here.
For full information about usage statistics collected by GT 4.0.4 and how to opt-out, see Usage Statistics Collection by the Globus Alliance.
- Refreshed version of CoG JGlobus library. Please see the CoG JGlobus Release Notes for more details.
Refreshed version of Apache Axis library with the following bug fixes:
- HTTP connections used for one-way requests were not always closed.
- TypeMapping was not thread-safe (bug 4858).
- Tested with Java SE 6 (RC).
- Upgraded SpiderMonkey version 1.60
- Ported to Mac OS X / Intel x86
- Improved bindings generation: better error messages and handling of more XML Schema constructs.
- Authorization portal framework for controlling resource and service instance level access.
- Expanded unit test coverage of SSL and GSI authentication/authorization mechanisms.
- "samples/globus" directory of sample client scripts for using various gt4 services.
- New WS Delegation client library (pyGridWare.common.delegationlib).
- New Reliable File Transfer client library (pyGridWare.common.rftlib).
- Script wsdl2web, all logic is now in a module (pyGridWare.utility.generate.wsdl2web).
- Updated GPT package from 3.6 to 3.7.
- Verify that credentials in the myproxy-server repository are still valid (i.e., not revoked) before performing delegation
- Added myproxy-admin-query --invalid option for listing, locking, or removing invalid credentials from repository
- Optionally check OCSP status of stored credentials before performing delegation via myproxy-server.config ocsp settings; requires GT 3.2 (OpenSSL 0.9.7) or later
- Updated etc.init.d.myproxy script to use pidfile to locate server to stop (rather than searching ps output), include a restart option, and exit with error if $GLOBUS_LOCATION isn't set
- If the myproxy-server hostname given by $MYPROXY_SERVER or the -s option resolves to multiple IP addresses, clients will connect to each address until a connection is established or all fail
- Added accepted_credentials_mapapp call-out version of accepted_credentials_mapfile in myproxy-server.config
- Now support unencrypted, although still signed, Pubcookie granting cookies as passwords
- Added myproxy-server.config syslog_ident option
- Improved MyProxy CA logging
- Added myproxy-server --listen to specify host/ip to bind to
- Updated GPT package from 3.7 to 3.9.
- Upgraded to OpenSSH 4.5p1.
- Upgraded to HPN12v14 patch.
Since 4.0.3, SGAS has transitioned from 2.0 beta to 2.0 "final" version. You can get the latest release at http://www.sgas.se/.
The most significant enhancement in 4.0.4 was the improvement in processing 1000+ job runs from a single client. This type and scale of processing is much more reliable in 4.0.4 than 4.0.3. Upgrading is recommended. More can be read from the details of these (ongoing) campaigns:
NOTE: Between GT releases 4.0.3 and 4.0.4, there was a GRAM release in VDT 1.6.x. The GRAM release in VDT 1.6.x included additional functionality that is NOT included in 4.0.4. This functionality will be added in the following point release - 4.0.5. Below is a list of the functionality in VDT 1.6 that is NOT in 4.0.4
- GRAM service auditing support
- For audit support, change use of client-generated job resource keys
- Default job description subtitution variables
- Cache user home directory to improve performance
- globus-job-*-ws tools
- job description extension support
- softenv extension support
- Pre-WS GRAM LRM job monitoring with SEG
A campaign tracking the work to add this functionality in 4.0.5 is here
- Bug #4772: globus-wsrf-cgen uses literal NULL type when certain schema errors occur
- Bug #4773: Bad error messages from globus-wsrf-cgen
- Bug #4807: Incorrect default SOAPAction
- Bug #4857 Bug 4857 - globus-wsrf-cgen treats all element declarations as global
- Bug #4921 Local non-namespaced attributes can conflict
- Bug 3371: cas-group-admin delete fails if grant all rights is given to the same group that was created.
- Bug 4731:close() not invoked on authorization chain.
- Fixed a bug in HostAuthorization class to deal with service certificates correctly.
- Bug 280: fix handling of usernames containing '/', '-', and '.' characters; note this required a change to the myproxy-server repository format, so credential data files written by a new myproxy-server won't be readable by an older myproxy-server
- Bug 4725: Version number typo in setup-simple-ca and a minor error in Makefile.in
You can find information about fixed bugs here.
- Bug 3141: globus-url-copy segfaults when attempting to glob with a server-side auth error
- Bug 4344: Globus-url-copy client does not reuse the connection when run in 3rd party mode.
- Bug 4571: 4.0.2 globus-url-copy core dumps with -cd -f option on x86_64
- Bug 4735: globus_l_ftp_control_read_cb() message pull up bug
- Bug 4786: Globus Gridftp server represent pipe as 'x' instead of 'p'?
- Bug 4825: gridftp-server process fails to exit and busy waits if you have a zero length CRL
- Bug 4945: GridFTP client accesses freed memory
- Bug 4975: globus-url-copy fails on some transfers to gass server
- Bug 4803: MDS stub classes are replicated in a number of jar files
- Bug 4803: MDS stub classes are replicated in a number of jar files
- Bug 5027: more efficient processing of internal job states
- Bug 4865: Events from SEG to JobStateMonitor are deleted too early i.e. jobs keep stuck
- Bug 4777: Excessive memory usage for SEG modules
- Bug 4983: Flag that triggers default SoftEnv keys usage in job submission
- Bug 4706: Fork jobs without arguments may receive uninitialized data as arguments
- Bug 4764: globusrun-ws output for bad -Ft is unhelpful.
- Bug 4732: globusrun-ws segmentation fault (FC5 x86_64)
- Bug 4884: GramJob post-submit subscriptions broken
- Bug 4860: LSF jobmanager doesn't preserve environment variables with spaces
Limitations
WS-Notification support:
- Only the Simple topic dialect is supported (others can be added)
- Only flat topic spaces are supported (architecture does allow for more advanced structures)
- Actions on the precondition, selector and policy fields in a subscription are not supported
- When a resource is removed its subscriptions are not removed automatically
- Only XPath resource property queries are supported (others can be added)
- A resource might not get destroyed at the exact time as indicated by the scheduled termination time. A sweeper thread that removes expired resources runs periodically (every 1 minute by default) so an expired resource might not get removed until the next time the sweeper thread runs.
- SOAP messages with attachments are not supported. In fact, the Axis version distributed with GT was compiled without attachment support.
- In certain cases, the "
dialect" attribute of TopicExpression or QueryExpression is not serialized properly as defined in the schema. An "org.globus.dialect.attr.qualified" Java system property was added to control how the serialization of thedialectattribute. Please see the Bug 3513 for details. - The saml protocol namespace, urn:oasis:names:tc:SAML:1.0:protocol is included in the default list of namespaces to exclude during stub generation. Since core does not use all elements of this namespace, stubs are not generated for all elements in the namespace. Since this namespace is in the default excludes list, stubs will not be generated if the target is used with defaul values for ns.excludes. This has been fixed since this release, but to override this behaviour, set the ns.excludes property appropriately for the generateStubs target. Setting that property to be an empty string will result in stubs being generated for all namespaces.
Known Bugs
- Bug 2471: Message security signature verification issues
- Bug 2445: Same input and output messages in WSDL confuse Axis
- Bug 2921: Support for TerminationTimeChangeRejectedFault
- Bug 2926: Local transport does not work without a current MessageContext
- Bug 3113: Processing by the WSDLPreprocessor produces output different depending on the JVM
- Bug 3482: wsa:From is not set correctly when service calls another service
- Bug 3483: xsd:anyType not serialized correctly
- Bug 4432: SimpleTopic.notify(SOAPElement element) drop child elements
- Bug 4566: globus-deploy-gar insists that ANT_HOME be set
- Bug 4831: GWSDL RP Inheritence Limited
- Bug 5071: Problems in building GT 4.0.4 with Java 1.6 and Ant 1.7
- Multiple schemas which use the same namespace prefixes can confuse the WSDL parser.
- Nillable elements are not serialized or deserialized correctly if the element does not contain the minOccurs="0" attribute
- The service engine and clients are not thread-safe
For information on existing bugs, click here.
- Bug 2219: xio_close in an xio callback hangs
- Bug 2530: END_OF_ENTITY
- Bug 3082: context leak
- Bug 3147: XIO doesn't provide for multiplexed drivers passing down driver_handle_cntl
- Bug 4186: Some GRAM security errors less informative than before
- Bug 4354: Encryption may cause a deadlock.
- Bug 4429: Memory leaks when using GSI driver from XIO library.
- Bug 4624: globus_xio_close() dumps a core
- Bug 4995: occasional hanging in threaded XIO
- Bug 5004: Failed assert in globus_l_xio_system_unregister_write on x86_64 platform
- Bug 484: Mem leak on deactivate_all
- Bug 686: globus_mutex_lock() behaves differently on Unix -vs- Win32
- Bug 930: inttypes.h included without testing if it's present
- Bug 1141: Misleading error messages using globus-makefile-header
- Bug 2791: Command globus-domainname needs g11n
- Bug 2793: Command globus-generalized-time needs g11n
- Bug 2794: Command globus-hostname needs g11n
- Bug 3081: Small leak in libltldl
- Bug 3107: Fatal error: tcp_init(): globus_io_tcp_create_listener() ...
- Bug 3436: globus_uuid MAC address retrieval doesn't work on solaris
- Bug 3509: GLOBUS_COMMON_MODULE->GLOBUS_CALLBACK_MODULE deactivation...
- Bug 3610: globus-makefile-header returns bad field
- Bug 4035: MacOS threading issues
- Bug 4097: Signal handling probrem on MacOS X
- Bug 4318: Pkgdata needs to make source dir explicit
- Bug 4348: globus-makefile-header generates invalid makefile for Index Service bindings
- Bug 2709: The MyProxy package isn't internationalized.
You can find information about known issues here.
- Bug 3121: Bug in enforcing maximum active transfers in a container
- Bug 3324: gridftpd croaking on startup - globus_l_io_tcp_contact_string failed
- Bug 3214: Backend of GT4 GridFTPd should ignore ~/.globus/certificates
- Bug 3387: client library doesn't respect GLOBUS_HOSTNAME
- Bug 3622: gridftp server improperly handles detach option in config file
- Bug 3761: globus_ftp_control library hangs at various places (depending on os)
- Bug 3807: "not a plain file" could be "file not found"
- Bug 3845: data channel authentication failure
- Bug 4119: Malformed UTF-8 character problem in globus-ftp-client tests
- Bug 4274: client library has no default striping layout.
- Bug 4861: Problem with HTTP chunked mode transfer in globus-url-copy
- Threading/Libc Problems: set LD_ASSUME_KERNEL=2.2.5 in your environment and see Platform Notes for more information.
- Bug 2601: Missing 'sigset' linking error
- Bug 2821: GT3.9.5: jni.h not found when compiling RLS on MacOS X
- Bug 3253: GT4 build fails on Mac OS X 10.4 Tiger
- Bug 3656: ACLs cannot be modified dynamically
- Bug 3828: RLSClient crashes when a null parameter is passed to a method
- Bug 4114: Java apis fail on 64 bit machine (patch available)
- Bug 4141: regexec call in auth.c's auth_getperms
- Bug 4142: globus-rls-admin -s always indicates RLI does not exist
- Bug 4512: RLS query returns incomplete result on 64bit system (patch available)
- Bug 4630: RLS java client cannot load libaries through JNI on Mac (see workaround)
The following problems are known to exist for OGSA-DAI at the time of the 4.0.4 release and will be addressed in the future. If you have any others that you feel should be added to this list then please let us know.
You may also want to consult the platform-specific FAQ and general FAQ. Additional information may be posted in the advisories page which report any problems since the release, or the OGSA-DAI bugzilla which may also have information on any problems with the current release.
Performance:
- Accessing the databaseSchema property for DB2/Cloudscape can sometimes retrieve unrelated meta-data and also cause OGSA-DAI to crash.
- Accessing the databaseSchema property for Oracle causes a server-side java.lang.OutOfMemoryError.
- Attempting to return a very large data set in a Response document can cause a server-side java.lang.OutOfMemoryError. This can be avoided by submitting an asynchronous request (one that uses an outputStream) and retrieving the results using a data service's data transport operations.
- Queries of tables for millions of small rows can eventually cause a server-side java.lang.OutOfMemoryError even if using stream activities and data transport operations.
- The fileWritingActivity attempts to read the whole file in a oner and so could cause a server-side java.lang.OutOfMemoryError for very large files.
Some databases, such as dBASE IV database on Microsoft Windows 2000, allow unusual characters, such as 0xC to be contained in certain field types. When field values containing this character, OGSA-DAI encodes them into invalid XML that subsequently causes Xerces-dependent components to raise an org.xml.sax.SAXParseException. Activities that have two outputs can sometimes cause java.lang.OutOfMemoryErrors to occur server-side if manipulating large amounts of data. This can arise when using the GZIP activities for example.
- Such activities are driven by one of the two connected activities and the other activity can accumulate data without processing it. So, for example for the GZIP activities OGSA-DAI focuses on the GZIP meta-data output steam and data only flows from this stream when a file has been completely unzipped. Therefore for large files the whole contents of the file flows into the activity and is buffered without being sent on.
- OGSA-DAI's GZIP activities will warn if memory usage is close to maximum and throw an OGSA-DAI exception server-side so that the web services container does not crash.
Error and exception handling:
- The failure of a single activity in a request can cause the status of all activities to be set to ERROR, even those that are not connected and succeeded.
- The data transport activities can gulp exceptions. For example if an sqlQueryStatement with a incorrectly formed SQL query statment is connected to an outputStream activity then attempting to pull data from the outputStream using the data transport operations results in no data - no indication of an error is given to the client.
- The uk.org.ogsadai.exception.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
Security:
- Encrypted role map files are currently unsupported.
- If a client does not provide a certificate then deliverFromGridFTP and deliverToGridFTP in a request will fail.
File activities:
- The fileAccessActivity encodes text line-by-line if Base 64 encoding is being used. This may give different results from encoding a whole file in a oner.
- In fileWritingActivity, EOF means "end of file" when perLine has value false but "end of line" when perLine has value true.
- In fileWritingActivity only strings from its input stream are handled. Other block types e.g. bytes are ignored.
General:
- Note that when deploying new data service resources dynamically via configurable data service this only works if the JARs required by the data service resource (e.g. database driver JARs) are already within the web service container's library directories. If this is not the case then the container has to be restarted.
- The data service terminate operation currently does nothing. It is intended as a placeholder for future development.
- Meta-data from a database, returned in the databaseSchema property, can be case-sensitive depending upon the database. For example MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
- The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang UnsupportedMethodException.
- Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.
- deliverFromGridFTP and deliverToGridFTP do not allow the setting of certain GridFTP parameters.
There is no support in the client toolkit for certain activities. These include:
- directoryAccessActivity
- fileAccessActivity
- fileManipulationActivity
- deliverFromFTP
- deliverToFTP
- gzipDecompression
- The deliverToStream activity only works if services are deployed in Tomcat.
- Bug 4761:Scheduler Tutorial is missing WS-GRAM setup package
- Bug 4778:WS-Fork job manager doesn't set environment up for mpi jobs
- Bug 4787:no lifetime management for WS Rendezvous
- Bug 4790:stdout RP gets null
- Bug 4817:Condor OS and ARCH do not have dynamic defaults
- Bug 4859:globusrun-ws staging error after gt4 deployed to tomcat
- Bug 4864:environment variables containing '=' get escaped
- Bug 4874:LRUCache problem during job submission from Condor-G to GRAM4
- Bug 4898:Streaming problem with globusrun-ws
- Bug 4908:Stage out failes in JobManager.pm
- Bug 4918:user account details are cached even for unknown users
- Bug 4944:Multijob resources never yield to memory pressure and can't be destroyed
- Bug 4968:WS-Notification error: java.lang.ClassCastException: org.globus.exec.generated.StateChangeNotificationMessageWrapperType cannot be cast to org.w3c.dom.Element
- Bug 4981:Hundreds of thousands of bogus entries in PBS server logs
- Bug 4984:Change in behaviour in usage of client-generated job resource keys
- Bug 4989:PBS environment var limit not tested
- Bug 5015:null pointer exception when submitting job to nightly build
- Bug 5020:implementation of myceil in pbs.pm
- Bug 4513:LD_LIBRARY_PATH should not be set if no library_path is specified
- Bug 4550:Multijob code not checking for existence of job credential
- Bug 4474:globus-gridmap-and-execute problem with additional PDPs
- Bug 3529:setup/postinstall fatal errors should be warnings
- Bug 3726:GlobusRun error message typo
- Bug 3910:Bad permissions on condor log file prevents job submissions
- Bug 4684:Loading persisted jobs with expired delegation resources causes stacktraces
- Bug 5009:globusrun-ws output for bad -Ft is unhelpful (part2)
- Bug 5017:gram[24] tests that need to be updated
- Bug 3948:Service must release all of its resources on deactivation
- Bug 3384:Inconsistent jobType/count parameter semantics
- Bug 3571:ant not found during install
- Bug 3575:SEG dependent on GLOBUS_LOCATION env var
- Bug 3672:Streaming with PBS fails
- Bug 3803:Default scratchDirectory doesn't exist
- Bug 4216:Empty submit_test/submitxxx.err causes FAILure in Local tests
- Bug 4452:job submission response is effected by java 1.5 thread processing
- Bug 4464:setting the mpirun/exec path used for mpi jobs
- Bug 4533:make-test-script.pl bug
- Bug 4597:Globus/Condor integration issues on HP-UX
- Bug 4719:globus runs /usr/bin/env without checking for \u
- Bug 4734:Missing wsa:Action for GRAM4 rendezvous register operations
- Bug 4749:client receives no state notifications if RSL gram:count is big
- Bug 1550: Fixes for race condition in job manager
- Bug 1934: Gatekeeper's syslog output cannot be controlled
- Bug 3373: globus removes the temporary job directory before pbs wri...
- Bug 3428: globus-personal-gatekeeper -list failing
- Bug 2739: Gatekeeper AuthZ/Gridmap Callout result logging
- Bug 2741: catching SIGSEGV if dynamic loading of authorization modu...
- Bug 4235: globus-job-manager doesn't exit if the job fails.
- Bug 4360: globus-job-get-output bug prevents output delivery, PBS jobmanager affected. See also globus-job-clean, globus-job-cancel
- Bug 4450: GT 4.0.1 globus-gatekeeper hang on UC/ANL TeraGrid
- Bug 4730: MPI Jobs using Globus LSF in HP XC Cluster....
- Bug 4747: Need evaluation of patch to JobManager.pm
- Bug 4771: date bug in job manager log file
- Bug 4905: setup-globus-gram-job-manager.pl fails silently when globus-version is missing
The
LeasedAccountsResourceimplementation will quarantine an account if the expiration script fails for any reason; but currently we provide no user friendly mechanism for manually un-quarantining the account when the situation is cleared by an administrator. We are planning to release an update shortly.
- Bug 1056: FTP client in extended block mode after receiving EODC does not wait for EOD on the same channel
- Bug 2277: GridFTPClient bug Parallel, Passive, EBlockImageDCReader does not receive EOF
- Bug 2345: Allow for a way to set cert and key location using environment variables
- Bug 2413: A rare decrypt error
- Bug 3299: Extended Key Usage certificate extension not supported
- Bug 4303: Class does not represent an RSA key
- Common Runtime Components
- Security
- Data Management
- Information Services
- Execution Management
- Cross-component Tools