Clustering at UCT

For more recent documentation using mod_proxy_ajp with Apache 2.2 for load balancing, see http://confluence.sakaiproject.org/confluence/display/DOC/Sakai+2.4+Admin+Guide+-+Advanced+Tomcat++%28and+Apache%29+Configuration

Stephen Marquard, University of Cape Town

Overview

Our test Sakai cluster uses:

Two test servers (espresso and latte), P4 2.6GHz, 1G RAM each
Shared filesystem between espresso and latte using (smb or nfs - to be done)
mysql database on espresso
LDAP for authentication (to external LDAP server)
apache2 httpd as frontend handling SSL connections and load balancing with mod_jk
Sakai (tomcat) instances on espresso and latte

tomcat_jk.conf (included in httpd.conf):

tomcat_jk.conf

# Set up apache2 to talk to Sakai tomcat instances using AJP connector

LoadModule jk_module /usr/lib/apache2/mod_jk.so

# Location of the worker file
JkWorkersFile "/etc/apache2/workers.properties"

# Location of the log file
JkLogFile "/var/log/apache2/mod_jk.log"

# Log level : debug, info, error or emerg
JkLogLevel info

workers.properties:

workers.properties

# List the workers name
worker.list=loadbalancer

# ----------------
# First worker
# ----------------
worker.espresso.port=8009
worker.espresso.host=espresso.cet.uct.ac.za
worker.espresso.type=ajp13

# Load balance factor
worker.espresso.lbfactor=1

# ----------------
# Second worker
# ----------------
worker.latte.port=8009
worker.latte.host=latte.cet.uct.ac.za
worker.latte.type=ajp13
worker.latte.lbfactor=1

# ----------------------
# Load Balancer worker
# ----------------------
worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=espresso,latte

Virtual host definition for sakai:

Virtual Host Definition

<VirtualHost _default_:443>
        # Usual site entries, enable SSL, logfiles, etc.

        # Redirect URLs to loadbalancer for tomcat app instances
        JkMount /* loadbalancer
</VirtualHost>

Tomcat configuration

In SAKAI_HOME/conf/server.xml, enable the AJP13 connector

server.xml

<!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009"
     enableLookups="false" redirectPort="8443" protocol="AJP/1.3" URIEncoding="UTF-8" />

Not clear whether redirectPort is necessary above.
It's useful to have tomcat still support SSL connections on another port (e.g. 8443) so as to be able to test an individual app server.

Sakai configuration

SAKAI_HOME/sakai/sakai.properties files are identical between app servers, and excludes the serverId setting.
SAKAI_HOME/sakai/local.properties has the serverId, and is different between servers, e.g.:

sakai and local properties

# Local server name
serverId=espresso

In subsequent Sakai releases, serverId will default to the machine's hostname, so it will not be necessary to specify this.
The worker names in mod_jk's workers.properties must match the serverId's in each app server's local.properties
Database auth info (mysql username/password) could be moved to security.properties.

Email configuration

Exim is running on port 25 on both servers. Incoming mail to the sakai domain is forwarded to the Sakai JAMES server running on port 8225.
Exim configuration:

Exim configuration

# Router (in routers section)
sakai_local:
  driver = manualroute
  domains = sakai.cet.uct.ac.za
  transport = sakai_smtp
  route_list = "* localhost"
  self = send

# ...

# Transport (in transport section)
sakai_smtp:
  driver = smtp
  allow_localhost = true
  port = 8025

Sakai is configured to send outgoing mail to localhost (exim)