Home > Documentation > Configuring and administering > Configuring and administering the Apache Geronimo Server > Clustering and farming > Tomcat Native Clustering |
The Tomcat Web container provides a native clustering solution that can be configured through Geronimo by using gbean definitions within config.xml
or by using server.xml
starting from v2.2 just like you did for a standalone Tomcat server. This document will go through the available GBeans and how to configure them for clustering in a Geronimo server with a tomcat web container.
Following are parameters or attributes that you will need to configure for a Tomcat native clustering.
Although sufficient for many applications, Tomcat clusters have some limitations:
Consider a cluster configuration to improve the scalability and availability of your Web application. The following sections provide detailed instructions about how to set up your cluster nodes.
To set up a small cluster, you need at least 2 nodes and 1 HTTP server. The HTTP server is used to serve requests from clients and ensure well-balanced traffic load among different nodes. Similarly, each node is configured to use the same logical Tomcat engine and enable session affinity.
The Tomcat cluster replicates HTTP session data by memory-to-memory multicast communication.
Every node transmits its session data to every other node in the cluster. This algorithm is efficient only when the clusters are small. If the clusters grow too large, the overhead in storage utilization and network traffic becomes excessive. To avoid excessive overhead, consider dividing your nodes into several smaller clusters.
HTTP session data is replicated among the nodes in the cluster by using a multicast broadcast. All nodes in the cluster must be on the same physical subnet and multicast broadcast must be supported by that subnet.
To participate in a cluster configuration, your Web application must be implemented correctly.
web.xml
file in the Web archive, must indicate that your Web application is distributable. To do this, insert the distributable element in the deployment descriptor.
<?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN" "http://java.sun.com/dtd/web-app_2_3.dtd"> <web-app> <display-name> ... </display-name> <description> ... </description> <distributable> <distributable/> ... </web-app>
Support for session affinity, also known as sticky session support, allows a load balancing service to route an HTTP request back to the same node that created the HTTP session associated with that request until that node fails. You must use session affinity if you configure an asynchronous type of session replication. With asynchronous replication, the reply is returned before the HTTP session is replicated, and the next request using that session might arrive before the replication is complete. In this case, route the request to the node that sent the reply to the last request and originated the replication to ensure that the request is processed using the correct session data..
In Geronimo servers, users can have the same experiences as in Tomcat Web container to configure session affinity. To enable session affinity, you must modify servel.xml
under the <GERONIMO_HOME>/var/catalina
directory and configure the <Engine> with a jvmRoute attribute value that is unique for each node in the cluster. The load balancer will return this value in the session cookie or the encoded URL returned to the browser. When a related request arrives, it can use the value to route the request to the correct node. See the following example:
server.xml
as follows after the server is stopped:
... <Engine name="Catalina" defaultHost="${ServerHostname}" jvmRoute="nodeId"> ...
workers.properties
.Initially, the server configuration includes an AJP connector suitable for exchanging messages with a load balancing service. See Configuring a remote Apache HTTP server for more information about the HTTP server configuration.
To configure application level clustering, you have to input all related GBean settings in your deployment plan to make sure HTTP sessions are replicated successfully. Moreover, you must use the gbean definitions within deployment plans to configure the cluster configuration elements of Tomcat Web containers. If you want to deploy your Web application to a cluster, install your WAR files to the appropriate cluster member, assuring that you use the correct deployment plan for each member.
The template for your Web application deployment plan is shown as follows. See Creating deployment plans for Web applications for more information about the parameters.
<web-app xmlns="http://geronimo.apache.org/xml/ns/j2ee/web/tomcat-1.2" xmlns:dep="http://geronimo.apache.org/xml/ns/deployment-1.2"> <dep:environment> <dep:moduleId> <dep:groupId>org.mygroup</dep:groupId> <dep:artifactId>web-cluster-server1</dep:artifactId> <dep:version>2.2</dep:version> <dep:type>war</dep:type> </dep:moduleId> <dep:dependencies> <dep:dependency> <dep:groupId>console.realm</dep:groupId> <dep:artifactId>geronimo-properties-realm</dep:artifactId> <dep:version>1.0</dep:version> <dep:type>car</dep:type> </dep:dependency> </dep:dependencies> <dep:hidden-classes/> <dep:non-overridable-classes/> </dep:environment> <context-root>/servlet-examples-cluster</context-root> <security-realm-name>geronimo-properties-realm</security-realm-name> <security> <default-principal> <principal name="anonymous" class="org.apache.geronimo.security.realm.providers.GeronimoUserPrincipal"/> </default-principal> <role-mappings> <role role-name="tomcat"> <principal name="admin" class="org.apache.geronimo.security.realm.providers.GeronimoGroupPrincipal"/> </role> </role-mappings> </security> <cluster>TomcatCluster</cluster> <gbean class="org.apache.geronimo.tomcat.cluster.CatalinaClusterGBean" name="TomcatCluster"> <attribute name="className">org.apache.catalina.ha.tcp.SimpleTcpCluster</attribute> <attribute name="initParams"/> <reference name="ClusterManager"> <name>TomcatClusterManager</name> </reference> <reference name="TomcatValveChain"> <name>ReplicationValve</name> </reference> <reference name="ClusterListenerChain"> <name>ClusterSessionListener</name> </reference> <reference name="Channel"> <name>TomcatGroupChannel</name> </reference> </gbean> <!-- Cluster Manager --> <gbean name="TomcatClusterManager" class="org.apache.geronimo.tomcat.cluster.ClusterManagerGBean"> <attribute name="className">org.apache.catalina.ha.session.DeltaManager</attribute> <attribute name="initParams">name=${clusterName} channelSendOptions=6 expireSessionsOnShutdown=false notifyListenersOnReplication=true mapSendOptions=6</attribute> </gbean> <!-- Cluster Channel --> <gbean class="org.apache.geronimo.tomcat.cluster.ChannelGBean" name="TomcatGroupChannel"> <attribute name="className">org.apache.catalina.tribes.group.GroupChannel</attribute> <attribute name="initParams"/> <reference name="Membership"> <name>TomcatMembership</name> </reference> <reference name="Receiver"> <name>TomcatReceiver</name> </reference> <reference name="Sender"> <name>TomcatSender</name> </reference> <reference name="ChannelInterceptor"> <name>TomcatChannelInterceptor</name> </reference> </gbean> <!-- Cluster Membership --> <gbean class="org.apache.geronimo.tomcat.cluster.MembershipServiceGBean" name="TomcatMembership"> <attribute name="className">org.apache.catalina.tribes.membership.McastService</attribute> <attribute name="initParams"> mcastAddr=228.0.0.4 mcastPort=45564 mcastFrequency=500 mcastDropTime=3000 </attribute> </gbean> <!-- Cluster Receiver --> <gbean class="org.apache.geronimo.tomcat.cluster.ReceiverGBean" name="TomcatReceiver"> <attribute name="className">org.apache.catalina.tribes.transport.nio.NioReceiver</attribute> <attribute name="initParams"> tcpListenAddress=auto tcpListenPort=4001 tcpSelectorTimeout=100 tcpThreadCount=6 </attribute> </gbean> <!-- Cluster Sender --> <gbean class="org.apache.geronimo.tomcat.cluster.SenderGBean" name="TomcatSender"> <attribute name="className">org.apache.catalina.tribes.transport.ReplicationTransmitter</attribute> <attribute name="initParams"> replicationMode=pooled waitForAck=true </attribute> </gbean> <!-- Cluster Valves--> <gbean class="org.apache.geronimo.tomcat.ValveGBean" name="ReplicationValve"> <attribute name="className">org.apache.catalina.ha.tcp.ReplicationValve</attribute> <attribute name="initParams">filter=.*\.gif;.*\.js;.*\.css;.*\.png;.*\.jpeg;.*\.jpg;.*\.htm;.*\.html;.*\.txt;</attribute> <reference name="NextValve"> <name>JvmRouteBinderValve</name> </reference> </gbean> <!-- Cluster Route Binder --> <gbean class="org.apache.geronimo.tomcat.ValveGBean" name="JvmRouteBinderValve"> <attribute name="className">org.apache.catalina.ha.session.JvmRouteBinderValve</attribute> <attribute name="initParams">enabled=true</attribute> </gbean> <!-- Cluster Listener --> <gbean class="org.apache.geronimo.tomcat.cluster.ClusterListenerGBean" name="ClusterSessionListener"> <attribute name="className">org.apache.catalina.ha.session.ClusterSessionListener</attribute> <reference name="NextListener"> <name>JvmRouteSessionIDBinderListener</name> </reference> </gbean> <!-- Cluster Binder Listener--> <gbean class="org.apache.geronimo.tomcat.cluster.ClusterListenerGBean" name="JvmRouteSessionIDBinderListener"> <attribute name="className">org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener</attribute> </gbean> <!-- Cluster Channel Interceptor --> <gbean class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean" name="TomcatChannelInterceptor"> <attribute name="className">org.apache.catalina.tribes.group.interceptors.TcpFailureDetector</attribute> </gbean> </web-app>
where
On each node, deploy your Web application, by using either the administration console or the deploy command, following this syntax:
deploy --user name --password word deploy archive plan
where
Note: After the server installation, the default user name is system, and the default password is manager.
The following code snippet is part of a clustering example that uses unicast configuration. Static members are defined using org.apache.geronimo.tomcat.cluster.StaticMemberGBean
. You have to define each static member within your deployment plan to make sure that your application is clustered successfully, and make sure that TCP ports for session replication are defined consistently on each node.
... <cluster>TomcatCluster</cluster> <gbean name="TomcatCluster" class="org.apache.geronimo.tomcat.cluster.CatalinaClusterGBean"> <attribute name="className">org.apache.catalina.ha.tcp.SimpleTcpCluster</attribute> <attribute name="initParams"/> <reference name="ClusterListenerChain"> <name>TomcatClusterListenerChain</name> </reference> <reference name="TomcatValveChain"> <name>ReplicationValve</name> </reference> <reference name="Channel"> <name>TomcatChannel</name> </reference> <reference name="ClusterManager"> <name>TomcatClusterManager</name> </reference> </gbean> <gbean name="TomcatClusterListenerChain" class="org.apache.geronimo.tomcat.cluster.ClusterListenerGBean"> <attribute name="className">org.apache.catalina.ha.session.ClusterSessionListener</attribute> <attribute name="initParams"/> </gbean> <gbean name="ReplicationValve" class="org.apache.geronimo.tomcat.ValveGBean"> <attribute name="className">org.apache.catalina.ha.tcp.ReplicationValve</attribute> <attribute name="initParams">.*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;</attribute> <reference name="NextValve"> <type>TomcatValve</type> <name>JvmRouteBinderValve</name> </reference> </gbean> <gbean name="JvmRouteBinderValve" class="org.apache.geronimo.tomcat.ValveGBean"> <attribute name="className">org.apache.catalina.ha.session.JvmRouteBinderValve</attribute> <attribute name="initParams">enabled=true</attribute> </gbean> <gbean name="TomcatClusterManager" class="org.apache.geronimo.tomcat.cluster.ClusterManagerGBean"> <attribute name="className">org.apache.catalina.ha.session.DeltaManager</attribute> <attribute name="initParams">name=${clusterName} channelSendOptions=6 expireSessionsOnShutdown=false notifyListenersOnReplication=true mapSendOptions=6 </attribute> </gbean> <gbean name="TomcatChannel" class="org.apache.geronimo.tomcat.cluster.ChannelGBean"> <attribute name="className">org.apache.catalina.tribes.group.GroupChannel</attribute> <attribute name="initParams"/> <reference name="Membership"> <name>ClusterMembership</name> </reference> <reference name="Receiver"> <name>ClusterReceiver</name> </reference> <reference name="Sender"> <name>ClusterSender</name> </reference> <reference name="ChannelInterceptor"> <name>DisableMcastInterceptor</name> </reference> </gbean> <gbean name="ClusterMembership" class="org.apache.geronimo.tomcat.cluster.MembershipServiceGBean"> <attribute name="className">org.apache.catalina.tribes.membership.McastService</attribute> <attribute name="initParams">address=228.0.0.4 port=45564 frequency=500 dropTime=3000 </attribute> </gbean> <gbean name="ClusterReceiver" class="org.apache.geronimo.tomcat.cluster.ReceiverGBean"> <attribute name="className">org.apache.catalina.tribes.transport.nio.NioReceiver</attribute> <attribute name="initParams">address=IPAddress1 port=TCP_port1 selectorTimeout=100 maxThreads=6 </attribute> </gbean> <gbean name="ClusterSender" class="org.apache.geronimo.tomcat.cluster.SenderGBean"> <attribute name="className">org.apache.catalina.tribes.transport.ReplicationTransmitter</attribute> </gbean> <gbean name="DisableMcastInterceptor" class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean"> <attribute name="className">org.apache.geronimo.tomcat.interceptor.DisableMcastInterceptor</attribute> <attribute name="initParams"/> <reference name="NextInterceptor"> <name>TcpPingInterceptor</name> </reference> </gbean> <gbean name="TcpPingInterceptor" class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean"> <attribute name="className">org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor</attribute> <attribute name="initParams">Interval=60</attribute> <reference name="NextInterceptor"> <name>TcpFailureDetector</name> </reference> </gbean> <gbean name="TcpFailureDetector" class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean"> <attribute name="className">org.apache.catalina.tribes.group.interceptors.TcpFailureDetector</attribute> <attribute name="initParams"/> <reference name="NextInterceptor"> <name>StaticMember1Interceptor</name> </reference> </gbean> <gbean name="StaticMember1Interceptor" class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean"> <attribute name="className">org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor</attribute> <attribute name="initParams"/> <reference name="StaticMember"> <name>StaticMember2</name> </reference> <reference name="NextInterceptor"> <name>MessageDispatch15Interceptor</name> </reference> </gbean> <gbean name="MessageDispatch15Interceptor" class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean"> <attribute name="className">org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor</attribute> <attribute name="initParams"/> <reference name="NextInterceptor"> <name>ThroughputInterceptor</name> </reference> </gbean> <gbean name="ThroughputInterceptor" class="org.apache.geronimo.tomcat.cluster.ChannelInterceptorGBean"> <attribute name="className">org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor</attribute> <attribute name="initParams"/> </gbean> <gbean name="StaticMember2" class="org.apache.geronimo.tomcat.cluster.StaticMemberGBean"> <attribute name="className">org.apache.catalina.tribes.membership.StaticMember</attribute> <attribute name="initParams">port=TCP_port2 securePort=-1 host=IPAddress2 domain=test-domain UniqueId={2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0} </attribute> </gbean> ...
Where
To convert this example to a multicast configuration, removed the DisableMCastInterceptor, StaticMemberInterceptor, and StaticMember definitions. Also, change the value for the "address" attribute for the TomcatReceiver definition to "auto".
You can configure the cluster just like you always did for a standalone Tomcat Web container. Input all the clustering configuration into the var/catalina/server.xml
file according to your requirements in Geronimo. Both multicast and unicat are supported. See the following sample codes when you want to configure an Engine or Host level cluster. Make sure that these segments are enclosed within the <Engine> or <Host> element in server.xml
. ${clusterName} is from var/config/config-substitutions.properties
, which should be identical within all nodes in a cluster.
Details about each component of the cluster and configuration options can be found in the tomcat documentation at Tomcat6 clustering.
... <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true" name="${clusterName}" channelSendOptions="6" mapSendOptions="6"/> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="228.0.0.4" port="45564" frequency="500" dropTime="3000" /> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6" /> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" /> </Sender> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" interval="60"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor" /> <Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor" /> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;" /> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve" /> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" /> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" /> </Cluster> ...
... <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true" name="${clusterName}" channelSendOptions="6" mapSendOptions="6"/> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="228.0.0.4" port="45564" frequency="500" dropTime="3000" /> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6" /> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" /> </Sender> <Interceptor className="org.apache.geronimo.tomcat.interceptor.DisableMcastInterceptor" /> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" interval="60"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor"> <Member port="TCP_port" securePort="-1" host="IPAddress" domain="test-domain" UniqueId="{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}" /> </Interceptor> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor" /> <Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor" /> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;" /> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve" /> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" /> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" /> </Cluster> ...
Where
Bookmark this on Delicious Digg this | Privacy Policy - Copyright © 2003-2013, The Apache Software Foundation, Licensed under ASL 2.0. |