There is simply too much information on DBMS installation for us to provide useful tips. In general, a binary package with a native installer (e.g., RPM, PKG, etc.) is the best option if you have root privileges. A binary package without a native installer is an excellent option if you do not have root privileges. And if all else fails, you can always try source installers. We have found most source installers of the DBMS to be reliable.

We have used RLS with MySQL versions 4.0.x, 4.1.x, and 5.0.x.

There are probably numerous things you could do, but one simple check is to see if you can log on to the database. Of course, this requires that you have a user account. Assuming you installed the DBMS yourself, you should be able to log on as root:

% /path/to/mysql/bin/mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 64 to server version: 4.0.18-standard-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show databases;
+-------------------+
| Database          |
+-------------------+
| mysql             |
| test              |
+-------------------+
2 rows in set (0.03 sec)

mysql> quit
Bye
        

If you did not install the DBMS yourself but were given an account on a remote server, you should be able to log on using your user account:

% /path/to/mysql/bin/mysql -h hostname -P 3306 -u dbuser -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 66 to server version: 4.0.18-standard-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>
        

There is simply too much information on DBMS installation for us to provide useful tips. In general, a binary package with a native installer (e.g., RPM, PKG, etc.) is the best option if you have root privileges. A binary package without a native installer is an excellent option if you do not have root privileges. And if all else fails, you can always try source installers. We have found most source installers of the DBMS to be reliable.

We have used RLS with PostgreSQL versions 7.4.x and it seems to work well with 8.x also.

There are probably numerous things you could do, but one simple check is to see if you can log on to the DBMS. Of course, this requires that you have a user account. Assuming you installed the DBMS yourself, you should be able to log on as root:

% /path/to/pgsql/bin/psql
Welcome to psql 7.4.6, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help on internal slash commands
       \g or terminate with semicolon to execute query
       \q to quit

user=#
        

If you did not install the DBMS yourself but were given an account on a remote server, you should be able to log on using your user account:

% /path/to/pgsql/bin/psql -h hostname -p 5432 -U dbuser -W dbname
Password:
Welcome to psql 7.4.6, the PostgreSQL interactive terminal.
 
Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help on internal slash commands
       \g or terminate with semicolon to execute query
       \q to quit
 
dbname=>
        

If you cannot connect to PostgreSQL but you know that the process is alive, you may have forgotten to use the -i when starting the DBMS. This option instructs postmaster to listen on TCP/IP socket. Without it, postmaster will not accept socket connections from clients.

Be sure to do periodic vacuum and analyze commands on all your PostgreSQL databases. The PostgreSQL documentation recommends doing this daily from cron. Failure to do this can seriously degrade performance, to the point where routine RLS operations (such as LRC to RLI soft state updates) timeout and fail. Please see the PostgreSQL documentation for further details.

Please be aware that we have relatively limited exposure to using RLS with SQLite than other database management systems. As of the GT 4.2 development branch, we have added SQLite setup scripts to create the RLS schema in a sqlite embedded database. Our testing has been limited to various Linux flavors on 32-bit platforms.

You are encouraged to review the SQLite site for authoritative details on installation of sqlite3. The following commands have been used by us to install sqlite-3.3.6:

% tar zxvf sqlite-3.3.6.tar.gz
% mkdir bld
% cd bld
% ../sqlite-3.3.6/configure --prefix=/path/to/install --enable-threadsafe
% make
% make install
        

There are probably numerous things you could do, but one simple check is to see if you can open a test database. You should be able to open a test database and perform various SQL operations:

% /path/to/sqlite/bin/sqlite3 test.db
SQLite version 3.3.6
Enter ".help" for instructions
sqlite>
        

The iodbc.org website contains information on installation of the iODBC (or libiodbc) library. We have used the following commands to install iODBC version 3.52.4 in our development and test sites. From the libiodbc build directory:

% ./configure --prefix=/path/to/libiodbc-3.52.4 \
--with-iodbc-inidir=/path/to/etc --disable-gui --disable-gtktest
% make
% make install
        

The configure parameters are optional. The defaults enable the features required by RLS, namely pthreads. We prefer to install to a specified directory due to testing against multiple ODBC managers, however, this would not apply to most end users. We also prefer not to install the GUI features.

Keep in mind, if you install the ODBC Manager library to a non-standard location, you will need to update your library path (e.g., LD_LIBRARY_PATH) to reflect your non-standard installation.

The unixodbc.org website contains information on installation of the unixODBC library. We have used the following commands to install unixODBC version 2.2.11 in our development and test sites. From the unixODBC build directory:

% ./configure --prefix=/path/to/unixODBC --disable-gui \
  --disable-drivers
% make
% make install
        

The configure parameters are optional. The defaults enable the features required by RLS, namely threads. We prefer to install to a specified directory due to testing against multiple ODBC managers, however, this would not apply to most end users. We also prefer not to install the GUI features or the supplied drivers.

Keep in mind, if you install the ODBC Manager library to a non-standard location, you will need to update your library path (e.g., LD_LIBRARY_PATH) to reflect your non-standard installation.

Building unixODBC-2.2.11 on OS X failed for us too. Our unixODBC tarball came from the unixodbc.org project site. It may be that other sources of this tarball may have been ported properly. What we found by doing a brief search was that making a small change to one of the files corrected the problem. In the file Drivers/txt/SQLTables.c, we made the following change from:

#ifdef OSXHEADER
#include <i386/types.h>
#endif
        

to:

/*
#ifdef OSXHEADER
#include <i386/types.h>
#endif
*/
        

The MySQL project/company provide several binary packages for several major platforms using a variety of native installers. We recommend using this approach if possible, though it may require root privileges and help from your system administator. There are also binary packages available that do not use native installers, which can be used without root privileges and provide a better option than source installation.

From what we can tell, the 3.51.12 version of MySQL Connector/ODBC from the binary bundles provided by MySQL will work with unixODBC-2.2.11 but not with libiodbc-3.52.4 or earlier. The last known combinations of drivers from MySQL that worked with iODBC that we know of are:

Unfortunately, MySQL does not provide links to packages of MyODBC-3.51.06 as far as we can tell. You may find mirrors of the MySQL site that still have this driver available by searching the web for "myodbc 3.51.06". We found this one.

% tar -zxvf /path/tomysql-connector-odbc-3.51.12-linux-i686.tar.gz
            

% ./bin/iodbctest "DSN=lrc1000_mysql_3_51_12;UID=dbpassword;PWD:dbpassword"
iODBC Demonstration program
This program shows an interactive SQL processor
Driver Manager: 03.52.0406.0126
1: SQLDriverConnect = [MySQL][ODBC 3.51 Driver]Invalid window handle for
connection completion argument. (0) SQLSTATE=IM008
1: ODBC_Connect = [MySQL][ODBC 3.51 Driver]Invalid window handle for connection
completion argument. (0) SQLSTATE=IM008

Have a nice day.
            

% tar -zxvf /path/to/MyODBC-3.51.06-pc-linux-gnu-i686
            

% ./configure --prefix=/path/to/odbc/Drivers/MyODBC-3.51.06-unixODBC \
 --with-unixODBC --with-unixODBC-includes=/path/to/odbc/unixODBC-2.2.11/include \
 --with-unixODBC-libs=/path/to/odbc/unixODBC-2.2.11/lib \
 --with-odbc-ini=/path/to/odbc/etc/odbc.ini \
 --with-mysql-includes=/path/to/mysql/include --with-mysql-libs=/path/to/mysql/lib \
 --enable-thread-safe --without-samples
% gmake
% gmake install
            

% ./configure --prefix=/path/to/odbc/Drivers/MyODBC-3.51.06-iODBC \
 --with-iodbc=/path/to/libiodbc-3.51.2 --with-odbc-ini=/path/to/odbc/etc/odbc.ini \
 --with-mysql-includes=/path/to/mysql/include --with-mysql-libs=/path/to/mysql/lib \
 --enable-thread-safe --without-samples
% gmake
% gmake install
            

The README file or docs directory included with the psqlODBC package contains information on installation of the psqlodbc library. We have noticed some differences between building and installing psqlodbc-7.2.5, psqlodbc-08.00.0102, and psqlodbc-08.01.0200 againt iODBC and unixODBC.

Note: In earlier versions of this guide, we warned that the --with-iodbc configure option resulted in an unstable psqlodbc library that caused memory corruption and failure (e.g., core dump) when an application (RLS Server) attempted to open the database connection. We have not experienced this problem with the more recent versions of psqlodbc that we have tested and which are documented below.

% ./configure --prefix=/path/to/odbc/Drivers/psqlodbc-7.2.5-unixODBC \
  --enable-pthreads --with-unixodbc
% gmake
% gmake install
            

% ./configure --prefix=/path/to/odbc/Drivers/psqlodbc-7.2.5-iODBC \
  --enable-pthreads --with-iodbc
% gmake
% gmake install
            

% ./configure --prefix=/path/to/odbc/Drivers/psqlodbc-08.00.0102-unixODBC \
  --enable-pthreads --with-unixodbc LDFLAGS="-L/path/to/odbc/unixODBC-2.2.11/lib" \
  CPPFLAGS="-I/path/to/odbc/unixODBC-2.2.11/include"
% gmake
% gmake install
            

% ./configure --prefix=/path/to/odbc/Drivers/psqlodbc-08.00.0102-iODBC \
  --enable-pthreads --with-iodbc LDFLAGS=-L/path/to/odbc/libiodbc-3.52.4/lib \
  CPPFLAGS=-I/path/to/odbc/libiodbc-3.52.4/include
% gmake
...ends in errors...
            

% ./configure --prefix=/path/to/odbc/Drivers/psqlodbc-08.00.0102-iODBC \
  --enable-pthreads --with-iodbc LDFLAGS=-L/path/to/odbc/libiodbc-3.51.2/lib \
  CPPFLAGS=-I/path/to/odbc/libiodbc-3.51.2/include
% gmake
% gmake install
            

The README file included with the SQLite ODBC Driver package contains information on installation of the sqliteodbc library. Please be aware that the author of the SQLite ODBC Driver states that the package is of experimental quality.

% ./configure  --prefix=/path/to/install --disable-winterface --with-sqlite3=/path/to/sqlite-3.3.6 --with-odbc=/path/to/libiodbc-3.51.2
% make
% make install
            

The short answer, consider the following options:

The long answer follows. First of all, if you are a new user and not confident installing system level software, you will save yourself a lot of time and trouble by consulting with your system administrator. If you are an administrator and you still find this troubling, then you are in good company.

As of the writing of this guide, we have experienced a higher success rate using unixODBC (current stable release at this time is version 2.2.11) than we have with iODBC (current stable release is version 3.52.4).

As of the writing of this guide, we have found PostgreSQL and psqlODBC installation from source to be reliable. We would recommend the use of psqlODBC version 08.00.0102 with unixODBC. If you are an iODBC user, you will need to use psqlODBC 7.2.5.

As of the writing of this guide, we have found MySQL and MySQL Connector/ODBC installation from binary to be reliable. We would recommend the use of MySQL Connector/ODBC version 3.51.12 with unixODBC. If you are a iODBC user, you will need to find a binary of MyODBC 3.51.06.

There are a variety of ways that you can specify the Data Source Names (DSNs) in the odbc.ini file. You may use a GUI or command-line tool provided with your ODBC Manager (unixODBC or iODBC). We tend to edit the odbc.ini file directly when working in our development and test sites, however, you should avoid this practice unless you are experienced with using ODBC tools (a small typo such as an unwanted space character can prevent your configuration from working -- with little in the way of useful error messages).

The following is an example of a typical setup when using PostgreSQL:

[lrc1000]
Driver      = /path/to/lib/psqlodbc.so
Database    = lrc1000
Servername  = hostname
Port        = 5432

[rli1000]
Driver      = /path/to/lib/psqlodbc.so
Database    = rli1000
Servername  = hostname
Port        = 5432
        

The following is an example of a typical setup when using MySQL:

[lrc1000]
Driver      = /path/to/lib/libmyodbc3.so
Database    = lrc1000
SERVER      = hostname
PORT        = 3306
SOCKET      = /path/to/mysql/mysql.sock

[rli1000]
Driver      = /path/to/lib/libmyodbc3.so
Database    = rli1000
SERVER      = hostname
PORT        = 3306
SOCKET      = /path/to/mysql/mysql.sock
        

The following is an example of a typical setup when using SQLite:

[lrc1000]
Driver      = /path/to/lib/libsqlite3odbc.so
Database    = /path/to/globus-rls-lrc1000.db

[rli1000]
Driver      = /path/to/lib/libsqlite3odbc.so
Database    = /path/to/globus-rls-rli1000.db
        

The ODBC libraries look for the odbc.ini file along a search path. We are not experts in this matter and it seems that it varies from vendor to vendor over time. But to the best of our knowledge the general search path is in the following order:

When we are troubleshooting a system, we tend to use the strace utility to see exactly which files it is reading.

The commands below demonstrate the show tables command with a MySQL database. Use the equivalent of this command if you are using a different DBMS. Alternately, you may simply perform a few SELECT queries against the listed tables.

Testing with unixODBC, run:

% /path/to/unixODBC-2.2.11/bin/isql lrc1000 dbuser dbpassword
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> show tables;
+------------------+
| Tables_in_lrc1000|
+------------------+
| t_attribute      |
| t_date_attr      |
| t_flt_attr       |
| t_int_attr       |
| t_lfn            |
| t_map            |
| t_pfn            |
| t_rli            |
| t_rlipartition   |
| t_str_attr       |
+------------------+
SQLRowCount returns 10
10 rows fetched
SQL> quit
        

Testing with iODBC, run:

% /path/to/bin/iodbctest "DSN=lrc1000;UID=dbuser;PWD=dbpassword"
iODBC Demonstration program
This program shows an interactive SQL processor
Driver Manager: 03.51.0002.0224
Driver: 03.51.06

SQL>show tables;
 
Tables_in_lrc1000
-----------------
t_attribute
t_date_attr
t_flt_attr
t_int_attr
t_lfn
t_map
t_pfn
t_rli
t_rlipartition
t_str_attr
 
 result set 1 returned 10 rows.

SQL>quit
 
Have a nice day.
        

Please be advised (and advise other users responsible for bringing up the RLS) that the startup initialization may take a few minutes before the RLS may be accessible. The initialization involves two key operations that may consume significant resources causing the server to appear temporarily unresponsive. Users of RLS may mistakenly assume that RLS failed to startup and may kill the server and start over. Some users may fall into this in a repeated cycle, believing that the RLS is unable to startup properly.

If the RLS is configured to send compressed updates (Bloom filters) to other RLIs, the RLS startup will involve initialization of the Bloom filter representing the current contents of the local replica catalog (LRC). This step is a prerequisite before any additional operations may be allowed, therefore no client connections are permitted until the initialization is complete. In our test environment, we have seen over 30 seconds delay due to creation of the Bloom filter corresponding to 1 million LFN names on a system with Dual 1 GHz CPU and 1.5 GB RAM. You may experience greater delays at larger scales and/or when running RLS with more limited system resources.

If the RLS is configured to send uncompressed updates (LFN lists) to other RLIs, the RLS startup will not involve any additional initialization delay. However, the RLS will spawn an initial full catalog update to all RLIs it updates. Though these updates will take place on separate threads of execution after the initialization of the system, they will consume a great amount of processor activity. Depending on the volume of the local replica catalog (LRC), this processor activity may initially interfere with a client operation. In our test environment, we have seen our initial "globus-rls-admin ping..." operation may suffer a delay and timeout in 30 seconds, the second "ping" may delay for a few seconds but will successfully return, and the third and every subsequent "ping" operation will successfully return immediately throughout the duration of the update. The system exhibits the same behavior for any other client operation, such as a globus-rls-cli query... operation.

This note originated based on observations of RLS Server running on RedHat 9 but could also apply to other Linux distributions. There have been occurrences of RLS servers hanging on RedHat 9 systems. The external symptoms are:

#!/bin/sh

GLOBUS_LOCATION=/opt/gt3.2
MYSQL=/opt/mysql
IODBC=/opt/iodbc

export GLOBUS_LOCATION

#RedHat 9 workaround
LD_ASSUME_KERNEL=2.4.1
export LD_ASSUME_KERNEL
...
            

...
LD_ASSUME_KERNEL=2.2.5
...