Appendix B. Globus Toolkit 4.2.0 Errors

Table B.1. Java WS Core Errors

Error CodeDefinitionPossible Solutions
Failed to acquire notification consumer home instance from registryCaused by javax.naming.NameNotFoundException: Name services is not bound in this Context error. Please see Running client programs from any directory if a client fails with this error.
The WS-Addressing 'To' request header is missingThis warning is logged by the container if the request did not contain the necessary WS-Addressing headers. The client either did not attempt to send those headers at all or is somehow misconfigured.If you are using a Java client and launching it directly using the java executable, take a look at Appendix B, Running client programs from any directory.
java.io.IOException: Token length X > 33554432 If you see this error in the container log, it usually means you are trying to connect to HTTPS server using HTTP. For example, the service address specifies 8443 as a port number and http as the protocol name.In general, use 8443 port number with the https protocol, and 8080 port number with the http protocol.
java.lang.NoSuchFieldError: DOCUMENTThis error usually indicates a mismatch between the version of Apache Axis that the code was compiled with and the version of Axis that the code is currently running with. Make sure that the same version of Axis is used at compile time and at runtime.
org.globus.wsrf. InvalidResourceKeyException: Argument key is null / Resource key is missingThese errors usually indicate that a resource key was not passed with the request or that an invalid resource key was passed with the request (that is, the element QName of the resource key did not match what the service expected).Make sure that the EPR used to invoke the service that contains the appropriate resource key. If you are using some command-line tool make sure to specify the resource key using the -k option or pass a complete EPR from a file using the -e option.
Unable to connect to localhost:xxxCannot resolve localhost. The machine's /etc/hosts isn't set up correctly and/or you do not have DNS for these machines. There should always be an entry in /etc/hosts (or /etc/hostname/ on Debian) for localhost in the following format (IP address/fully qualified domain name/short name):
140.221.8.109   cognito.mcs.anl.gov cognito
org.globus.common.ChainedIOException: Failed to initialize security contextThis may indicate that the user's proxy is invalid.To correct the error, the user must properly initialize the user proxy. See grid-proxy-init for more information on proxy initialization.
Error: org.xml.sax.SAXException: Unregistered type: class xxxThis may indicate that an Axis generated XML type, defined by the WS RLS XSD, was not properly registered. While all the XML types should get registered upon deployment without intervention by the user, sometimes they do not.To remedy the situation add a typeMapping to the server-config.wsdd file under globus_wsrf_replicalocation_service. Use the format shown here.
No socket factory for 'https' protocol

When a client fails with the following exception:

 java.io.IOException: No socket factory for 'https' protocol at
        org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:179) at
        org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:397) at
        org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:135)

FIXME - it may have happened because...

Add the following to the client:

 import org.globus.axis.util.Util; ... static { Util.registerTransport(); }
...
No client transport named 'https' found

When a client fails with the following exception:

No client transport named 'https' found at
        org.apache.axis.client.AxisClient.invoke(AxisClient.java:170) at
        org.apache.axis.client.Call.invokeEngine(Call.java:2726)

The client is most likely loading an incorrect client-config.wsdd configuration file.

Ensure that the GT4 installation directory is listed as the first entry in the CLASSPATH of the client. For example:

CLASSPATH=/usr/local/globus-4.2.0:/foo/bar/others.jar:...

If you are seeing this problem in Tomcat, copy the client-config.wsdd from the GT4 installation directory to the Web application's WEB-INF/classes directory.

ConcurrentModificationException in Tomcat 5.0.x

If the following exception is visible in the Tomcat logs at startup, it might cause the HTTPSValve to fail:

java.util.ConcurrentModificationException at
        java.util.HashMap$HashIterator.nextEntry(HashMap.java:782) at
        java.util.HashMap$EntryIterator.next(HashMap.java:824) at
        java.util.HashMap.putAllForCreate(HashMap.java:424) at
        java.util.HashMap.clone(HashMap.java:656) at
        mx4j.server.DefaultMBeanRepository.clone(DefaultMBeanRepository.java:56)

The HTTPSValve might fail with the following exception:

java.lang.NullPointerException at
        org.apache.coyote.tomcat5.CoyoteRequest.setAttribute(CoyoteRequest.java:1472) at
        org.apache.coyote.tomcat5.CoyoteRequestFacade.setAttribute(CoyoteRequestFacade.java:351) at
        org.globus.tomcat.coyote.valves.HTTPSValve.expose(HTTPSValve.java:99)

These exceptions will prevent the transport security from working properly in Tomcat.

This is a Tomcat bug. Keep restarting Tomcat until it starts without the ConcurrentModificationException or switch to a different version of Tomcat.

java.net.SocketException: Invalid argument or cannot assign requested address

FIXME - what causes this?

If you see the java.net.SocketException: Invalid argument or cannot assign requested address error in the container log or on the client side, try setting the following property:

 $ export GLOBUS_OPTIONS="-Djava.net.preferIPv4Stack=true"
GAR deploy/undeploy fails with container is running error

A GAR file can only be deployed or undeployed locally while the container is off. However, GAR deployment/undeployment might still sometimes fail with this error even if the container is off. This usually happens if the container has crashed or was stopped improperly, preventing the container from cleaning up its state files.

To resolve this problem, delete any files under the $GLOBUS_LOCATION/var/state directory and try to redeploy/reundeploy the GAR file again.

Table B.2. C WS Core Errors

Error CodeDefinitionPossible Solutions
globus_soap_message_module: Failed sending request http://widgets.com/WidgetPortType/createWidgetRequest. globus_xio: Unable to connect to grid.example.org:8080 globus_xio: System error in connect: Connection refused globus_xio: A system call failed: Connection refused Unable to contact service container Check that the service endpoint refers to a running container.
globus_soap_message_module: Failed sending request http://widgets.com/WidgetPortType/createWidgetRequest. globus_xio_gsi: gss_init_sec_context failed. GSS Major Status: Unexpected Gatekeeper or Service Name globus_gsi_gssapi: Authorization denied: The name of the remote entity (/C=US/O=Globus Alliance/OU=Service/CN=host/grid.example.org), and the expected name for the remote entity (/C=US/O=Globus Alliance/OU=Service/CN=host/cloud.example.org) do not match Service is not running with the expected security credential. Verify that the service credential being presented by the service (first parenthesized name) is a reasonable certificate name for the service. If so, set the GLOBUS_SOAP_MESSAGE_PEER_IDENTITY_KEY attribute on the soap message handle to that identity. For most command-line wsrf tools, this can be done by passing it as an argument to the -z command-line parameter.
globus_soap_message_module: SOAP Fault Fault code: Client Fault string: globus_service_engine_module: Failed to find operation: {XXXX}YYYY for service: {ZZZZ}BBBB The service port type {ZZZZ}BBBB does not contain a {XXXX}YYYY operation. Verify that the client bindings are built from the same WSDL and XML Schema documents as the service.
globus_soap_message_module: Failed receiving response http://widgets.com/WidgetPortType/createWidgetResponse. ws_addressing: Addressing header is a draft version of WS Addressing: "http://schemas.xmlsoap.org/ws/2004/03/addressing". This could be a GT version mismatch, client is GT 4.2.x and response is from GT 4.0.x server The service is running on a container which is using a draft version of the WS-Addressing specification. This was used by GT 4.0.xUpdate the service to work with GT 4.2.x or compile your client with GT 4.0.x libraries.
globus_soap_message_module: Failed sending request http://widgets.com/WidgetPortType/createWidgetRequest. globus_xio: The GSI XIO driver failed to establish a secure connection. The failure occured during a handshake read. globus_xio: An end of file occurred The service container either did not support SSL authentication, or the service container did not trust the client certificateConsult the service administrator to verify that the service container supports SSL and that your certificate is issued by a certificate authority trusted by the service.

Table B.3. XIO Errors

Error CodeDefinitionPossible Solutions
Operation was canceledAn I/O operation has been canceled by a close or a cancel In most cases this will be intentionally performed by the application developer. In unexpected cases the applciation developer should verify that there is not a race condition relating to closing a handle.
Operation timed out Occurs when the application developer associates a timeout with a handle's I/O operations. If no I/O is performed before the timeout expires this error will be triggered. The remote side of connection might be hung and busy. The network could have higher latencies than expected. The filesystem might be over worked.
An end of file occurred This occurs when and EOF is detected on the file descriptor When doing file I/O this like means you read to the end of the file and thus you are finished and should now close it. On network connections however it means the socket was closed on the remote end. This can happen it the remote side suddenly dies (seg-fault is common here) or if the remote side chooses to close the connection.
Contact string invalidA poorly formed contact string was passed in to open Verify the format of the contact string with the documentation of the drivers in use.
Memory allocation failed on XXXXmalloc failed. The system is likely quite overloaded Free up memory in your application
System error in XXXXA low level system error occurred. The errno and errstring should indicate more information.
Invalid stack The requested stack does not meet XIO standards Most likely a transport driver is not on the bottom of the stack, or 2 transport drivers are in the stack.
Operation already registered With certain common drivers like TCP and FILE, only one specific operations can be registered at a time (1 read, 1 write). If another operation of the same type is posted to the handle before receiving the previous operations callback, this error can occur. Restructure the application code so that it waits for the callback before registering the next IO operation.
Unexpected stateThe internal logic of XIO came across a logical path that should not be possible. Often times this is due to application memory corruption or trying to perform an IO operation on a closed or otherwise invalid handle. Use valgrind or some sort of memory managment tool to verify there is no memory corruption. Try to recreate the problem in a small program. Submit the program and the memory trace at bugzilla.globus.org
Driver in handle has been unloadedA driver associated with the offending operation has already been unloaded by the application code. Verify that you are not unloading drivers until they are no longer in use.
Module not activatedglobus_module_activate(GLOBUS_XIO_MODULE); has not been called. Call this before making any other XIO API calls.

Table B.4. Credential Errors

Error CodeDefinitionPossible Solutions
Your proxy credential may have expiredYour proxy credential may have expired.Use grid-proxy-info to check whether the proxy credential has actually expired. If it has, generate a new proxy with grid-proxy-init.
The system clock on either the local or remote system is wrong.This may cause the server or client to conclude that a credential has expired.Check the system clocks on the local and remote system.
Your end-user certificate may have expiredYour end-user certificate may have expiredUse grid-cert-info to check your certificate's expiration date. If it has expired, follow your CA's procedures to get a new one.
The permissions may be wrong on your proxy fileIf the permissions on your proxy file are too lax (for example, if others can read your proxy file), Globus Toolkit clients will not use that file to authenticate.You can "fix" this problem by changing the permissions on the file or by destroying it (with grid-proxy-destroy) and creating a new one (with grid-proxy-init).

Important: However, it is still possible that someone else has made a copy of that file during the time that the permissions were wrong. In that case, they will be able to impersonate you until the proxy file expires or your permissions or end-user certificate are revoked, whichever happens first.

The permissions may be wrong on your private key fileIf the permissions on your end user certificate private key file are too lax (for example, if others can read the file), grid-proxy-init will refuse to create a proxy certificate.You can "fix" this by changing the permissions on the private key file.

Important: However, you will still have a much more serious problem: it is possible that someone has made a copy of your private key file. Although this file is encrypted, it is possible that someone will be able to decrypt the private key, at which point they will be able to impersonate you as long as your end user certificate is valid. You should contact your CA to have your end-user certificate revoked and get a new one.

The remote system may not trust your CAThe remote system may not trust your CAVerify that the remote system is configured to trust the CA that issued your end-entity certificate. See Installing GT 4.2.0 for details.
You may not trust the remote system's CAYou may not trust the remote system's CAVerify that your system is configured to trust the remote CA (or that your environment is set up to trust the remote CA). See Installing GT 4.2.0 for details.
There may be something wrong with the remote service's credentialsThere may be something wrong with the remote service's credentialsIt is sometimes difficult to distinguish between errors reported by the remote service regarding your credentials and errors reported by the client interface regarding the remote service's credentials. If you cannot find anything wrong with your credentials, check for the same conditions on the remote system (or ask a remote administrator to do so) .

Table B.5. Gridmap Errors

Error CodeDefinitionPossible Solutions
The content of the grid map file does not conform to the expected formatThe content of the grid map file does not conform to the expected format Run grid-mapfile-check-consistency to make sure that your gridmap file conforms to the expected format.
The grid map file does not contain a entry for your DNThe grid map file does not contain a entry for your DN Use grid-mapfile-add-entry to add the relevant entry.

Table B.6. Java WS A&A Errors

Error CodeDefinitionPossible Solutions
[JWSSEC-248] Secure container requires valid credentialsThis error occurs when globus-start-container is run without any valid credentials. Either a proxy certificate or service/host certificate needs to be configured for the container to start up.
  1. If you are not looking to start up a container that uses GSI Secure Transport, which is used by the container by default, use globus-start-container -nosec. You will be able to use insecure clients and services. However, this also implies that if you have not configured individual services with credentials, you will not be able to securely access the service.

  2. If you are running a personal container, generate a proxy certificate with grid-proxy-init. If the proxy certificate is not in the default location, configure the container security descriptor as described in Configuring Container Security Descriptor.

  3. If you want to use host certificates, configure the container security descriptor as described Configuring Credentials.

Failed to start container: Container failed to initialize [Caused by: [JWSSEC-250] Failed to load certificate/key file]This error occurs if the file path to the container certificate and key configured are invalid.
  1. The path to the container certificate and key are configured in $GLOBUS_LOCATION/etc/globus_wsrf_core/ global_security_descriptor.xml. This file is loaded as described [here - fixme link]. Ensure that the path is correct.

Failed to start container: Container failed to initialize [Caused by: [JWSSEC-249] Failed to load proxy file]This error occurs if container proxy file configured is invalid.
  1. The path to the container proxy certificates are configured in $GLOBUS_LOCATION/etc/globus_wsrf_core/ global_security_descriptor.xml. This file is loaded as described [here - fixme link]. Ensure that the path is correct.

Failed to start container: Container failed to initialize [Caused by: [JWSSEC-245] Error parsing file: "etc/globus_wsrf_core/ global_security_descriptor.xml" [Caused by: ...]This error occurs if the container security descriptor configured is invalid.
  1. The container security descriptor should conform to the Container Security Descriptor Schema.

  2. Refer to the "Caused by: " section for details on the specific element that is not correct.

[JGLOBUS-77] Unknown CAThis error occurs if the CA certificate for the credentials being used is not installed correctly.
  1. If this issue occurs on the server side, the container is not configured with CA certificates. The container looks for trusted certificates in the default location as described Java CoG Toolkit FAQ

  2. On the server side, the trusted certificates can be configured as described in Trusted Certificates

  3. On the client side, trusted certificates can be configured as described in Configuring Trusted Credentials

Table B.7. GridShib Errors

Error CodeDefinitionPossible Solutions
error1description1 solutions or links to solutions

Table B.8. MyProxy Errors

Error CodeDefinitionPossible Solutions
MyProxy server name does not match expected name

This error appears as a mutual authentication failure or a server authentication failure, and the error message should list two names: the expected name of the MyProxy server and the actual authenticated name.

By default, the MyProxy clients expect the MyProxy server to be running with a host certificate that matches the target hostname. This error can occur when running the MyProxy server under a non-host certificate or if the server is running on a machine with multiple hostnames.

The MyProxy clients authenticate the identity of the MyProxy server to avoid sending passphrases and credentials to rogue servers.

If the expected name contains an IP address, your system is unable to do a reverse lookup on that address to get the canonical hostname of the server, indicating either a problem with that machine's DNS record or a problem with the resolver on your system.

If the server name shown in the error message is acceptable, set the MYPROXY_SERVER_DN environment variable to that name to resolve the problem.
Error in bind(): Address already in useThis error indicates that the myproxy-server port (default: 7512) is in use by another process, probably another myproxy-server instance. You cannot run multiple instances of the myproxy-server on the same network port. If you want to run multiple instances of the myproxy-server on a machine, you can specify different ports with the -p option, and then give the same -p option to the MyProxy commands to tell them to use the myproxy-server on that port.
grid-proxy-init failedThis error indicates that the grid-proxy-init command failed when myproxy-init attempted to run it, which implies a problem with the underlying Globus installation. Run
grid-proxy-init -debug -verify
for more information.
User not authorizedAn error from the myproxy-server saying you are "not authorized" to complete an operation typically indicates that the myproxy-server.config file settings are restricting your access to the myproxy-server. It is possible that the myproxy-server is running with the default myproxy-server.config file, which does not authorize any operations. See Configuring for more information.

Table B.9. GSI-OpenSSH Errors

Error CodeDefinitionPossible Solutions
GSS-API error Failure acquiring GSSAPI credentials: GSS_S_CREDENTIALS_EXPIREDThis means that your proxy certificate has expired. Run grid-proxy-init to acquire a new proxy certificate, then run gsissh again.
...no proxy credentials...Failing to run grid-proxy-init to create a user proxy with which to connect will result in the client notifying you that no local credentials exist. Any attempt to authenticate using GSI will fail in this case. Verify that your GSI proxy has been properly initialized via grid-proxy-info. If you need to initialize the proxy, use the command grid-proxy-init.
...bad file system permissions on private key; key must only be readable by the user...The host key that the SSH server is using for GSI authentication must only be readable by the user which owns it. Any other permissions will cause this error. Make sure that the host key's UNIX permissions are mode 400 (that is, it should only have mode readable for the user that owns the file, and no other mode bits should be set).
...gssapi received empty username; failed to set username from gssapi context; Failed external-keyx for <user> from <host> <port>...If the server was passed an "implicit username" (i.e. requested to map the incoming connection to a username based on some contextual clues such as the certificate's subject), and no entry exists in the grid-mapfile for the incoming connection's certificate subject, the server should output a clue that states it is unable to set the username against which to authenticate. Add an entry for the user to the [grid-mapfile fixme link].
...INTERNAL ERROR: authenticated invalid user xxx...If the subject name given in the system's grid-mapfile points to a non-existent user, the server will give an internal error which is best caught when it is running in debugging mode. Add a new account to the system matching the username pointed at by the user's subject in the grid-mapfile.
...gssapi received empty username; no suitable client data; failed to set username from gssapi context; Failed external-keyx for <user> from <host> <port>... Should the user attempt to connect without first creating a proxy certificate, or if the user is connecting via a SSH client that does not support GSI authentication, the server will note that no GSSAPI data was sent to it. Verify that the client is able to connect through another GSI service (such as the gatekeeper) to make sure that the user's proxy has been created correctly. Verify that you are using a GSI-enabled SSH client and that your GSI proxy has been properly initialized via grid-proxy-info. If you need to initialize this proxy, use the command grid-proxy-init.

Table B.10. GridFTP Errors

Error CodeDefinitionPossible Solutions
globus_ftp_client: the server responded with an error 530 530-globus_xio: Authentication Error 530-OpenSSL Error: s3_srvr.c:2525: in library: SSL routines, function SSL3_GET_CLIENT_CERTIFICATE: no certificate returned 530-globus_gsi_callback_module: Could not verify credential 530-globus_gsi_callback_module: Can't get the local trusted CA certificate: Untrusted self-signed certificate in chain with hash d1b603c3 530 End. This error message indicates that the GridFTP server doesn't trust the certificate authority (CA) that issued your certificate. You need to ask the GridFTP server administrator to install your CA certificate chain in the GridFTP server's trusted certificates directory.
globus_ftp_control: gss_init_sec_context failed OpenSSL Error: s3_clnt.c:951: in library: SSL routines, function SSL3_GET_SERVER_CERTIFICATE: certificate verify failed globus_gsi_callback_module: Could not verify credential globus_gsi_callback_module: Can't get the local trusted CA certificate: Untrusted self-signed certificate in chain with hash d1b603c3 This error message indicates that your local system doesn't trust the certificate authority (CA) that issued the certificate on the resource you are connecting to. You need to ask the resource administrator which CA issued their certificate and install the CA certificate in the local trusted certificates directory.

Table B.11. Reliable File Transfer (RFT) Errors

Error CodeDefinitionPossible Solutions
Error creating RFT Home: Failed to connect to database ... Until this is corrected all RFT request will fail and all GRAM jobs that require staging will failThis occurs when you start the container if RFT is not configured properly to talk to a PostgreSQL database. The usual cause is that Postmaster is not accepting TCP connections, which means that you must restart Postmaster with the -i option (see Configuring RFT).
ERROR service.RFTResourceManager [Thread-13,transferCompleted:517] Unable to update on finished org.globus.transfer.reliable.service.database.RftDBException: RFT database update error [Caused by: Syntax error: Encountered ")" at line 1, column 47.] This error occurs as a result of a dynamically built SQL update string. The update occurs when a transfer completes. It is used to notify transfer requests using the same hosts that resources on that host have been freed. The error message occurs when no rows in the database match that host. Users of RFT may safely ignore this error. The message is harmless to the functionality of RFT and will not affect the results of a transfer in any way. The exception is safely caught. Future versions of RFT will have optimizations to avoid this step.

Table B.12. Replica Locator Service (RLS) Errors

Error CodeDefinitionPossible Solutions
Error with credential: The proxy credential: <credential> with subject: <subject> expired <minutes> minutes ago Expired proxy credential Create a new proxy with grid-proxy-init.
Unable to connect to localhost:xxxx Unable to connect to the local host. This can be due to a variety of reasons, including a wrong address or port number in the RLS connection URL or an issue with a firewall configuration.
  • Double-check the address and port number in the RLS connection URL. parameters are correct.

  • If a firewall configuration is preventing connections to the target host for a particular port, you may need to consult the system administrator.

"connection timeout"At times, a client may experience a connection timeout when interacting with the RLS server due to a variety of reasons:
  • One reason could simply be due to wide-area network latency or congestion.

  • Another situation that users eventually encounter is due to scaling of the system. As the RLS server's database of replica location mappings grows in size, some query operations, such as bulk queries involving large quantities of mappings or wildcard queries that result in a large subset of mappings, will begin to take more time both to process the query and to return the large results set to the client over the network.

If timeouts are experienced with increasing frequency, increase the RLS server's timeout configuration parameter found in the $GLOBUS_LOCATION/var/globus-rls-server.conf file. You may also use the -t timeout option of the globus-rls-cli tool.

Table B.13. WS Replica Location Service (WS RLS) Errors

Error CodeDefinitionPossible Solutions
Error: java.lang.NullPointerExceptionWhen invoking the WS RLS command-line clients, a system-level exception like the one above may be encountered. The admin should check the container logs for the exact error.
Error: A server error occured while processing the requestWhen invoking the WS RLS command-line clients, a server error like the one above may be encountered. The admin should check the container logs for the exact error.
java.lang.UnsatisfiedLinkErrorThis exception when using the WS RLS may indicate that the native RLS libraries that WS RLS depends on could not be located. To correct this problem, ensure that the $GLOBUS_LOCATION/lib directory is in the library search path (on some systems this is the LD_LIBRARY_PATH variable).
Unable to connect to localhost:39281The WS RLS is an interface layer that depends on the RLS for the replica location functionality. You must install and run RLS and configure WS RLS to use the RLS via its JNDI configuration. Check that RLS is installed, running, and check that the WS RLS JNDI configuration uses the correct hostname and port to connect to the RLS.
org.globus.common.ChainedIOException: Failed to initialize security contextIf this exception occurs while using WS RLS, it may indicate that the user's proxy is invalid. To correct the error, the user must properly initialize the user proxy. See grid-proxy-init for more information on proxy initialization.
Error: org.xml.sax.SAXException: Unregistered type: class xxxIf this exception occurs when using the WS RLS, it may indicate that an Axis generated XML type, defined by the WS RLS XSD, was not properly registered. While all the XML types should get registered upon deployment without intervention by the user, sometimes they do not. To remedy the situation add a typeMapping to the server-config.wsdd file under globus_wsrf_replicalocation_service. Use the format shown here.

Table B.14. Data Replication Service (DRS) Errors

Error CodeDefinitionPossible Solutions
Authorization failed. Expected <hostname1> target but received <hostname2>Did not receive expected hostname When authorization is enabled on the container, you may need to use the proper hostname when referencing the DRS service rather than using localhost.
org.globus.wsrf.ResourceException: Failed to create Replication: /scratch/testrun (No such file or directory)Cannot find the request file Ensure that the request file's filename is correct, that it is reachable by the DRS service, and that it has the appropriate permissions for the DRS service to access it.
org.globus.wsrf.ResourceException: Failed to create Replication: String index out of range: -1The request file is malformed (for example by using spaces instead of a delimiting tab character) which is resulting in a runtime exception. Make sure your request file is in the correct form as described here.

Table B.15. WS MDS Index Service Error Messages

Error CodeDefinitionPossible Solutions
error what causes this possible solutions
WS MDS is built on Java WS Core, please see Java WS Core Error Codes for more error code documentation.

Table B.16. WS MDS Trigger Service Error Messages

Error CodeDefinitionPossible Solutions
Error ; nested exception is: org.apache.commons.httpclient. NoHttpResponseException: The server xxx.x.x.x failed to respondHappens when trying to create a trigger for the Trigger Service. The above error is accompanied by the following error in container: [JWSCORE-192] Error processing request java.io.IOException: Token length 1347375956 > 33554432. FIXME - what causes this? Be sure that you have properly edited the client-config-settings file under globus_wsrf_mds_trigger. The DefaultServiceAddress parameter should properly reflect the service prefix from your container, e.g.: https://127.0.0.1:8444/wsrf/services/. The services you wish to monitor should also be consistent.
WS MDS is built on Java WS Core, please see Java WS Core Error Codes for more error code documentation.

Table B.17. WS MDS Aggregator Error Messages

Error CodeDefinitionPossible Solutions
error what causes this possible solutions
WS MDS is built on Java WS Core, please see Java WS Core Error Codes for more error code documentation.

Table B.18. WS MDS Trigger Service Error Messages

Error CodeDefinitionPossible Solutions
java.net.ConnectException: Connection refused If you attempt to use WebMDS to collect information from a service that is not running, you will see a stack trace that begins with:
org.globus.mds.webmds.xmlSources.resourceProperties.ResourcePropertySourceException: ; nested exception is: 
	java.net.ConnectException: Connection refused
Make sure the service you are trying to collect information from is running.
faultString: org.globus.common.ChainedIOException: Authentication failed [Caused by: Failure unspecified at GSS-API level [Caused by: Unknown CA]] When WebMDS sends resource property queries to a secure WSRF service instance (such as an WS MDS Index Server), the WebMDS server must trust the certificate authority that issued the certificate used by the WSRF service instance. If the WebMDS server does not trust the CA used by the remote service, then WebMDS queries will produce a stack trace that includes this message. This can be solved by configuring the Tomcat server that hosts WebMDS to trust the appropriate CA, by either:
  • placing the CA certificate in /etc/grid-security/certificates, or

  • placing the CA certificate somewhere else, and setting the Tomcat process's X509_CERT_DIR system parameter to the directory in which the CA certificate was installed. One way to do this is to set the CATALINA_OPTS environment variable and then restart Tomcat:

    export CATALINA_OPTS=-DX509_CERT_DIR=/path/to/cert/dir
    $CATALINA_HOME/bin/shutdown.sh
    $CATALINA_HOME/bin/startup.sh

WebMDS connections to secure Index Servers (or other secure WSRF servers) just hang If the JVM used by Tomcat is configured to use a blocking random-number source, WebMDS connections to secure Index Servers (or other secure WSRF servers) can hang. This is the default configuration for many installations. One solution is to set the CATALINA_OPTS environment variable to ensure that Tomcat's JVM will use a non-blocking random-number source:
export CATALINA_OPTS=-Djava.security.egd=/dev/urandom
$CATALINA_HOME/bin/shutdown.sh
$CATALINA_HOME/bin/startup.sh
[Note]Note

f you encounter this problem with WebMDS, you may also encounter a similar problem with the Globus container on the same system.

Table B.19. GRAM4 Errors

Error CodeDefinitionPossible Solutions
globusrun-ws - error querying job state

During job submission, an error like this occurs:

globusrun-ws failed: Delegating user credentials...Done. Submitting job...Done. Job ID: xxxx Termination time: xxxx Current job state: Unsubmitted globusrun-ws: Error querying job state globus_soap_message_module: Failed sending request ManagedJobPortType_GetMultipleResourceProperties. globus_xio: An end of file occurred
Periodically, globusrun-ws will query the GRAM service to check on the job state. The "End of file" indicates that the GRAM server dropped a connection when globusrun-ws tried to read a response. This could be caused by temporary network issues between the client and service, or possibly caused by an overloaded service host.
globusrun-ws - error querying job state

During job submission, an error like this occurs:

globusrun-ws failed: Delegating user credentials...Done. Submitting job...Done. Job ID: xxxx Termination time: xxxx Current job state: Unsubmitted globusrun-ws: Error querying job state globus_soap_message_module: Failed sending request ManagedJobPortType_GetMultipleResourceProperties. globus_xio: System error in read: Connection reset by peer globus_xio: A system call failed: Connection reset by peer
Periodically, globusrun-ws will query the GRAM service to check on the job state. The
System error in read: Connection reset by peer
indicates that the GRAM server dropped the connection while trying to write the response. This could be caused by temporary network issues between the client and service, or possibly caused by an overloaded service host.
globusrun-ws - error submitting job

During job submission, an error like this occurs:

globusrun-ws -Ft PBS -F https://host.teragrid.org:8444 -submit -b -f /tmp/wsgram.rsl -o /tmp/wsgram.epr failed: Submitting job...Failed. globusrun-ws: Error submitting job globus_soap_message_module: Failed sending request ManagedJobFactoryPortType_createManagedJob. globus_xio: Operation was canceled globus_xio: Operation timed out
The
Operation timed out
indicates that the GRAM service was not able to accept the job request and respond in time. This could be caused by temporary network issues between the client and service, or possibly caused by an overloaded service host.

Table B.20. GRAM2 Errors

Error CodeDefinitionPossible Solutions
error1description1 solutions or links to solutions

Table B.21. Gridway Errors

Error CodeDefinitionPossible Solutions
Lock file existsAnother GWD may be running. Be sure that no other GWD is running, then remove the lock file and try again.
Error in MAD initializationThere may be problems with the proxy certificate, bin directory, or the executable name of a MAD may not be in the correct location. Check that you have generated a valid proxy (for example with the grid-proxy-info command). Also, check that the directory $GW_LOCATION/bin is in your path, and the executable name of all the MADs is defined in gwd.conf.
Could not connect to gwdGridWay may not be running or there may be something wrong with the connection. Be sure that GWD is running; for example:
pgrep -l gwd
If it is running, check that you can connect to GWD; for example:
telnet `cat $GW_LOCATION/var/gwd.port`