Configuring the Cluster/Load Balancer with GlassFish V2

Contents (Updated on Apr 25th 2007, send comments)

• Installing and Configuring GlassFish
• Installing and Configuring GlassFish Loadbalancer Plugin
  
• Setting up a One-machine Cluster using GlassFish V2
• Configuring the Load Balancer Plugin
• Deploying and Running the clusterjsp Sample Application
• High Availability/Failover using In-memory Replication feature
  
• Stop and Cleanup Cluster Settings
• Create Cluster using Glassfish Admin CLI commands
  
• Trouble Shooting

Installing and Configuring GlassFish

Install the GlassFish server by downloading it from GlassFish Download Page at: http://glassfish.java.net/public/downloadsindex.html

GlasssFish installation requires JDK 5 (1.5.0_10) and ant 1.6.5 (included).

1. Download one of the bundles to disk
2. Run:

   java -Xmx256m -jar filename.jar
   

3. cd glassfish
4. ant -f setup-cluster.xml

Installing and Configuring GlassFish Loadbalancer Plugin

To install and configure Load Balancer plugin to work with GlassFish, user can use GlassFish Loadbalancer Configurator. It has easy to use graphical interface which accepts user inputs, and then installs and configures loadbalancer plugin based on that. As of now beta version is available for the user. GlassFish Loadbalancer Configurator can install and configure load-balancer plugin onSun Java System Web Server, Apache and IIS. However GlassFish V2 is officially tested and supported only with Sun Java System Web Server and not supported for other webserver for the load balancer plugin.

Before you install the load balancer plugin, you must install Sun Java System Web Server 7.x (SJSWS) first. You can download the Web Server from: http://www.sun.com/software/products/web_srvr/get_it.jsp

User can also install and configure it manually.  Steps to install and configure GlassFish Loadbalancer Plugin on Sun Java System Web Server 7.x plugin are provided below :

Legends:
websrvr_install_dir - Sun Java System Web Server install Directory
websrvr_admin_user - Sun Java System Web Server admin user
websrvr_instance_config - Directory corresponding to Web Server instance config
websrvr_instance_dir - Directory corresponding to Web Server instance
as_install_dir - GlassFish installation Directory
OS - Operating System. It refers to 'linux' for the linux platform, 'solaris' for the solaris and 'windows' for windows platform.


1. Download aslb (appserver load balancer plugin) from the link: SunOS_X86 , SunOS , Linux , Windows
2. Unjar to install and configure in GlassFish installation (this is ${glassfish.home}same as ${S1AS_HOME) directory

• Unjar aslb-9.1.1-b13.jar file in "${glassfish.home}/lib/lbplugin"
• There are 2 zip files bundled inside the aslb jar: SUNWaslb.zip, SUNWaspx.zip, unzip these 2 zip files in the same directory and delete the zip files.
  
• Change permissions on all shared libraries of lbplugin as below
• chmod -R 755 ${glassfish.home}/lib/lbplugin/lib

• This is just an example for notational purpose, the user should manually edit (if choosen to do this way) this file prior to using this
  as loadbalancer.xml, to reflect the correct cluster configuration. Refer to “Configuring the Load Balancer Plugin” for this.
  Note manual editing is not the endorsed way to configure the load balancer. GlassFish Admin CLI or GUI are the two supported approaches for generating this file, while providing for error free load balancer configuration generation.

      ##BEGIN EE LB Plugin Parameters
    Init fn="load-modules" shlib="<websrvr_install_dir>/plugins/lbplugin/bin/libpassthrough.so" funcs="init-passthrough,service-passthrough,name-trans-passthrough" Thread="no"
    Init fn="init-passthrough"
      ##END EE LB Plugin Parameters

Init fn="load-modules" shlib=".../libj2eeplugin.so" shlib_flags="(global|now)" 

NameTrans fn="name-trans-passthrough" name="lbplugin" config-file="loadbalancer.xml"

     <Object name="lbplugin">
       ObjectType fn="force-type" type="magnus-internal/lbplugin"
        PathCheck fn="deny-existence" path="*/WEB-INF/*"
        Service type="magnus-internal/lbplugin" fn="service-passthrough"
        Error reason="Bad Gateway" fn="send-error" uri="$docroot/badgateway.html"
    </Object>

Setting Up a One Machine Cluster

If you haven't already done so, download and install GlassFish latest promoted build.

Now to create the cluster, we need to first configure and start GlassFish V2 domain (which is in cluster mode), run following commands:

% ant -f setup-cluster.xml
% cd ${glassfish.home}/bin
% asadmin start-domain --user ${adminuser} --passwordfile ${Admin-PasswordFile} domain-name

To create a cluster on the machine running the above Domain Administartion Server (DAS), follow these steps:

1. In a shell window, change your working directory to ${glassfish.home}/samples/quickstart/clusterjsp
2. Run the command: ${glassfish.home}/bin/asant setup-one-machine-cluster which automates following steps.
   
   • The values of cluster.name and nodeagent.name are specified here, change it appropriately.
     ${glassfish.home}/samples/quickstart/clusterjsp/cluster.properties.
   • Creates a cluster with name cluster.name (cluster1)
   • Creates a node-agent with name nodeagent.name (cluster1-nodeagent)

   • Starts the node-agent
   • Creates two instances under the cluster that will use the node agent just created
   • The default value for master password is "changeit".  This is different from the admin password.
     Most of the default values are used from ${glassfish.home}/samples/common.properties or cluster.properties above.
3. Starts the cluster ${glassfish.home}/bin/asant start_cluster

4. Troble-shooting:
   

• if you see the errors while starting the cluster, you can ignore some harmless errors. For more details, refer to troubleshooting section.
• On some Windows XP PCs, if this command hangs at "start-node-agent", do CNTR-C and retry again. Second time this goes through, starts node-agent, cluster successfully.
• If asant setup-one-machine-cluster is not starting the cluster, in that case asant start_cluster must be explicitly invoked.

Configuring the Load Balancer Plugin

<!DOCTYPE loadbalancer PUBLIC "-//Sun Microsystems Inc.//DTD Sun One Application Server 8.1//EN"
"sun-loadbalancer_1_2.dtd">
<loadbalancer>
 <cluster name="cluster1" policy="round-robin">
 <instance name="instance1" enabled="true" disable-timeout-in-minutes="60" 
	listeners="http://instance-one-host:instance-one-port" weight="100"/>
 <instance name="instance2" enabled="true" disable-timeout-in-minutes="60" 
	listeners="http://instance-two-host:instance-two-port weight="100"/>
 <web-module context-root="clusterjsp" enabled="true" disable-timeout-in-minutes="60" 
	error-url="sun-http-lberror.html" />
 <health-checker url="/" interval-in-seconds="10" timeout-in-seconds="30" />
 </cluster>

 <property name="reload-poll-interval-in-seconds" value="60"/>
 <property name="response-timeout-in-seconds" value="30"/>
 <property name="https-routing" value="true"/>
 <property name="require-monitor-data" value="false"/>
 <property name="active-healthcheck-enabled" value="false"/>
 <property name="number-healthcheck-retries" value="3"/>
 <property name="rewrite-location" value="true"/>
</loadbalancer>

To turn on load balancer plug-in monitor messages, set the require-monitor-data property value to true. The load balancer plug-in logs record the following information:

• Request start/stop information for every request.
• Failed-over request information when a request fails over from an unhealthy instance to a healthy instance.
• List of unhealthy instances at the end of every health-check cycle.

Deploying and Running the clusterjsp Sample Application

To test cluster1 of two GlassFish V2 Server instances, use the "clusterjsp" application located in  ${glassfish.home}/samples/quickstart/clusterjsp directory as clusterjsp.ear

• To deploy the clusterjsp application on "cluster1" of two instances, run following command

• cd ${JAVAEE_HOME}/samples/quickstart/clusterjsp

• % ${JAVAEE_HOME}/bin/asant deploy

• Now all the clustering, load balancing configurations and application deployment is done.
  

• You can access the clusterjsp application using http://hostname:web-server-port/clusterjsp (webserver port) and add sessions to the application.

• Clusterjsp http response will show which GlassFish Server instance (instance-ONE or instance-TWO) serving the request of the client.  To simulate application server clustering along with load-balancing, follow steps below

1. You will need to use two Firefox browsers or at least two seperate sessions of the browsers, and clear cache/cookies from the browser before every new request (this is needed to simulate unique clients) to access a deployed clusterjsp sample application.
   
2. You access the clusterjsp application through webserver http://web-server-host:web-server-port/clusterjsp
   and see the request is served by first instance that is running (instance-ONE in our case) in the http response you get in browser.  Healthy and the Unhealthy instance is been detected by webserver load-balancer plugin.
   
3. Open another Firefox browser, clear cache/cookies from the browser (Do it couple of times to really clear cache/cookies) and access clusterjsp application again through webserver http://web-server-host:web-server-port/clusterjsp This time, you can see the request is served by second instance that is running (instance-TWO in this case) in the http response you get in browser. This ensures that requests are served by instances in the order set in the loadbalancer.xml file in Round-Robin fashion.  Screenshots below.

High Availability/Failover using In-memory Replication feature

To verify GlassFish server High-Availability or failover while accessing a deployed clusterjsp sample application, follow the steps below:

1. You can make a particular instance serve a request by restarting the web server that has the load balancer plugin installed before deploying an application. This ensures that requests are served by instances in the order set in the loadbalancer.xml file. From the above loadbalancer.xml instance-ONE will serve the first request.
2. We have already deployed a sample application that stores session data, e.g. clusterjsp web application as above. With this we should be able to see that successive requests from same client are served by the same instance that the served first request and the session data is maintained across the requests.
   
3. Send few requests and note down the instance that served those requests and shutdown that particular instance. Use this command to stop the instance:

• asadmin stop-instance --user adminuser --password adminpassword instance-name

Stop and Cleanup Cluster Settings

1. To stop the cluster1 Cluster of two instances and cluster1-nodeagent nodeagent, run following command

• cd ${glassfish.home}/samples/quickstart/clusterjsp

• % ${glassfish.home}/bin/asant stop_nodeagent

• cd ${glassfish.home}/samples/quickstart/clusterjsp

• % ${glassfish.home}/bin/asant clean-one-machine-cluster

Create Cluster using GlassFish Admin CLI commands

• The values of cluster.name and nodeagent.name are specified here, change it appropriately. ${glassfish.home}/samples/quickstart/clusterjsp/cluster.properties.
• Creates a cluster with name cluster.name (cluster1)

• asadmin create-cluster --user admin --passwordfile adminpassword.txt --host localhost -port 4848 cluster1

• asadmin create-node-agent --user admin --passwordfile adminpassword.txt --host localhost -port 4848 cluster1-nodeagent

• asadmin start-node-agent --user admin --passwordfile adminpassword.txt --host localhost -port 4848 cluster1-nodeagent

asadmin create-instance --user admin --passwordfile
adminpassword.txt --host localhost -port 4848 --cluster cluster1
--nodeagent cluster1-nodeagent --systemproperties
"JMX_SYSTEM_CONNECTOR_PORT=8687:IIOP_LISTENER_PORT=3330:IIOP_SSL_LISTENER_PORT=4440:IIOP_SSL_

MUTUALAUTH_PORT=5550:HTTP_LISTENER_PORT=1110:HTTP_SSL_LISTENER_PORT=2220"
instance-ONE
 
asadmin create-instance --user admin --passwordfile
adminpassword.txt
--host localhost -port 4848 --cluster cluster1 --nodeagent
cluster1-nodeagent --systemproperties
"JMX_SYSTEM_CONNECTOR_PORT=8688:IIOP_LISTENER_PORT=3331:IIOP_SSL_LISTENER_PORT=4441:IIOP_SSL_

MUTUALAUTH_PORT=5551:HTTP_LISTENER_PORT=1111:HTTP_SSL_LISTENER_PORT=2221"
instance-TWO

• asadmin start-cluster --user admin --passwordfile adminpassword.txt
  --host localhost -port 4848 cluster1

Trouble Shooting

1. If you see the following error message during start-cluster, you can safely ignore it, we are working on resolving this.
start_cluster: