Like what you see? Have a play with our trial version.

Error rendering macro 'rw-search'

null

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

 

Overview

Yellowfin can be clustered on multiple servers to allow for high-availability and load-balancing. The function of load-balancing (multiplexing requests from external requests) can be achieved with a hardware load-balancer, or load-balancing software. This guide will outline the modifications required to Yellowfin's configuration to enable clustering, but not the external environment that directs incoming requests to particular nodes.

 It is required that the load-balancing infrastructure delivers network traffic to the Yellowfin application server transparently - as if the packets were connecting directly to the application server directly. If Session Replication is not enabled, the load-balancing infrastructure will need to provide “sticky-session” functionality. This is where traffic for a user’s session is sent to the same node for the duration of their session.

There are two components of a clustered environment. Application Messaging, which is required for the cluster to operate correctly, and Session Replication, which is optional.

 

Application Level Messaging

A Yellowfin node will communicate with other clustered nodes when the application needs to make changes that affect the entire cluster. This is usually to maintain the integrity of caches on remote nodes, but also facilitates functions such as user synchronisation and licence management. Application Level Messaging is configured by the ClusterManagement servlet in the web.xml file.

 

Container Level Session Replication

The Yellowfin application has also been written to allow user session information to be replicated onto multiple cluster nodes concurrently. User session information is the memory footprint that is stored on the server for each user session. Enabling Session Replication allows for the user’s session to continue, even if the cluster node they were connected to experiences a failure.

Without Session Replication, failure of a cluster node will destroy a user’s session, and they will need to log in again to another node.

Session Replication is a function of the Java Application Server, or other external infrastructure. An example of how this can be achieved with Tomcat will be discussed in this document.

 

 

Yellowfin Database Clustering

A Yellowfin Cluster must share a single Yellowfin Database instance. The database can be a single database instance, that is shared across all Yellowfin nodes, or a database that is clustered itself.  It is important that the Yellowfin database is scaled to handle database requests from all the nodes in the Cluster.

Database Clustering / Replication should be transparent to Yellowfin, as Yellowfin will have a single database URL for the entire cluster. Each node should be connecting to the same logical database, irrespective of how it is configured or clustered.

Licensing

The Yellowfin Licence file is stored in the Yellowfin database, and because of this, the Licence must contain a reference to all hostnames in the cluster. A clustered licence can be requested from Yellowfin.

 

 

Installing a Yellowfin Cluster

As a Yellowfin Cluster requires only one database instance, it means that the Yellowfin Installation process will differ slightly when installing a cluster.

The first Yellowfin Cluster node can be installed the same as a standalone instance of Yellowfin. The command-line or GUI installer can be used.

Additional Yellowfin Cluster nodes do not require the Yellowfin database.  There are several ways for installing additional nodes:

No Database Access Installation

After the initial node is installed, and the Yellowfin database is created, the Yellowfin installer can be used for the installation of subsequent nodes without affecting the initial database.

Use the Yellowfin Installer using the command:

java -jar yellowfin-20170928-full.jar action.nodbaccess=true

 

The action.nodbaccess=true option will run the installer as usual, and prompt for database credentials, but not create or alter the database during installation. This will only generate the filesystem component of Yellowfin. If Legacy Clustering is used, individual changes may need to be made to the web.xml file on each of the nodes.

File System Replication

As the filesystem structure on each node is the same, a new node can be created by duplicating the Yellowfin installation directory onto another computer. If Legacy Clustering is used, changes will need to be made to the web.xml file on each node.

Virtual Machine Replication

A Virtual Machine (or AWS AMI or Azure Compute image or Docker image) could be pre-configured with the Application Server component of Yellowfin. A copy of the Virtual Machine could be started for each application node in a cluster.

 

 

Yellowfin for Application Messaging

Each Yellowfin node must have the ClusterManagement servlet enabled for the node to become “Cluster Aware”. The ClusterManagement servlet is enabled by adding an additional configuration block to the web.xml file on each node.

Application Messaging is performed differently depending on the implementation mode. There are currently three modes available, REPOSITORY, DYNAMIC and LEGACY.

 

Multicast Cluster Messaging (DYNAMIC mode)

Yellowfin application messaging is handled by a multicast messaging library called JGroups. Using this method will automatically find other nodes in the cluster sharing the same Yellowfin database.

 The default configuration of JGroups uses UDP multicast messages to determine group membership and find new nodes. There may be environments where these types of messages cannot be sent. For example, Amazon does not allow multicast packets on its internal network between nodes. The Multicast Cluster Messaging adapter allows you to pass an XML configuration file to configure JGroups to use other methods for node discovery. This file can be referenced by passing the path to the BroadcastConfiguration servlet parameter within the ClusterManagement servlet.

 

The following servlet definition needs to be added to the web.xml on each node:

 

 
<!-- Cluster Management -->
<servlet>
       <servlet-name>ClusterManagement</servlet-name> 
       <servlet-class>com.hof.mi.servlet.ClusterManagement</servlet-class> 
       <init-param> 
             <param-name>ClusterType</param-name> 
             <param-value>DYNAMIC</param-value> 
       </init-param> 
       <init-param> 
             <param-name>SerialiseWebserviceSessions</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>CheckSumRows</param-name>
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>EncryptSessionId</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>EncryptSessionData</param-name> 
             <param-value>true</param-value> 
       </init-param>
<init-param> 
             <param-name>AutoTaskDelegation</param-name> 
             <param-value>true</param-value> 
       </init-param>
       <load-on-startup>11</load-on-startup> 
 </servlet>



 

 

 

Multicast with Repository Discovery (REPOSITORY mode)

Repository Discovery is an implementation of DYNAMIC mode, but with a custom plugin for discovering nodes via the shared Yellowfin Repository. This can be useful for enabling clustering on environments where Multicast packets do not work.

This functionality can also be enabled with DYNAMIC mode with the RepositoryDiscovery servlet parameter set to true.

 

 

 The following servlet definition needs to be added to the web.xml on each node:

 

 
<!-- Cluster Management -->
<servlet>
       <servlet-name>ClusterManagement</servlet-name> 
       <servlet-class>com.hof.mi.servlet.ClusterManagement</servlet-class> 
       <init-param> 
             <param-name>ClusterType</param-name> 
             <param-value>REPOSITORY</param-value> 
       </init-param> 
       <init-param> 
             <param-name>SerialiseWebserviceSessions</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>CheckSumRows</param-name>
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>EncryptSessionId</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>EncryptSessionData</param-name> 
             <param-value>true</param-value> 
       </init-param>
<init-param> 
             <param-name>AutoTaskDelegation</param-name> 
             <param-value>true</param-value> 
       </init-param>
       <load-on-startup>11</load-on-startup> 
 </servlet>



 

 

 

Web Service Cluster Messaging (LEGACY mode)

Yellowfin’s legacy cluster messaging is handled by AXIS web services. This requires that all nodes be defined at start-up, and that the service end-point, port, user and password be defined in each node’s web.xml file. Legacy mode does not allow cluster instances to reside on the same host.

 

 

The following servlet definition needs to be added to the web.xml on each node:

 

 
<!-- Cluster Management -->
<servlet>
       <servlet-name>ClusterManagement</servlet-name> 
       <servlet-class>com.hof.mi.servlet.ClusterManagement</servlet-class> 
       <init-param> 
             <param-name>ServiceUser</param-name> 
             <param-value>admin@yellowfin.com.au</param-value> 
       </init-param> 
       <init-param> 
             <param-name>ServicePassword</param-name> 
             <param-value>test</param-value> 
       </init-param> 
       <init-param> 
             <param-name>ServiceAddress</param-name> 
             <param-value>/services/AdministrationService</param-value> 
       </init-param> 
       <init-param> 
             <param-name>ServicePort</param-name> 
             <param-value>80</param-value> 
       </init-param> 
       <init-param> 
             <param-name>ClusterHosts</param-name> 
             <param-value> 
                   192.168.4.184 
             </param-value> 
       </init-param> 
       <init-param> 
             <param-name>SerialiseWebserviceSessions</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>CheckSumRows</param-name>
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>EncryptSessionId</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <init-param> 
             <param-name>EncryptSessionData</param-name> 
             <param-value>true</param-value> 
       </init-param> 
       <load-on-startup>11</load-on-startup> 
 </servlet>



 

 

 

Web.xml File Parameters

The following properties outline the options in the ClusterManagement servlet definition.

 

Property

Value

ClusterType

Either DYNAMIC or LEGACY. DYNAMIC will use multicast messaging and automatically find other nodes in the cluster. LEGACY is the default, and will use web services to communicate with a defined list of cluster nodes.

BroadcastConfiguration

A JGroups configuration file. This allows for a custom Jgroups configuration to be used in environments where multicast networking is not available. This is for DYNAMIC mode only and is optional. By default, JGroups will use the configuration defined in udp.xml.

ServiceUser

User that will validate the web service connection to other nodes. For LEGACY mode only.

ServicePassword

Password for the ServiceUser. For LEGACY mode only.

ServicePasswordEncrypted

True/False.
Is the contents of the ServicePassword encrypted.

ServiceAddress

Address of the Yellowfin web service. For LEGACY mode only.

ServicePort

Port on which Yellowfin is running. For LEGACY mode only.

ClusterHosts

This is a comma-separated list of all nodes in the Cluster. These can include IP addresses or hostnames. For LEGACY mode only.

SerialiseWebserviceSessions

True/False.

This is required if using Single Sign-On on the Cluster. It can serialise tokens to the database so that the token can be accessed from any node.

CheckSumRows

True/False.

Security option to check sum the serialised web service session records in the database. This helps prevent modification to the table which could lead to the creation of unauthorised sessions in Yellowfin.

EncryptSessionId

True/False.

Security option to encrypt the serialised web service session id in the database. This helps prevent modification to the table which could lead to the creation of unauthorised sessions in Yellowfin.

EncryptSessionData

True/False.

Security option to encrypt the serialised web service session records in the database. This helps prevent modification to the table which could lead to the creation of unauthorised sessions in Yellowfin.

AutoTaskDelegation

True/False.
This is for DYNAMIC mode only and is optional.

When enabled, the Cluster will automatically assign a node to perform background tasks. When this is turned on, you do not need to manually configure a node to run background tasks. Yellowfin will turn off System Tasks on a node at start-up if another node is already delegated to perform this task.

 

SessionReplication

True/False.

Set this to true if container level Session Replication is enabled. This will modify the logic used for destroying sessions in a cluster.

RepositoryDiscovery

True/False

This option is for DYNAMIC mode only. This enables RepositoryDiscovery on builds prior to the REPOSITORY mode being supported. REPOSITORY mode is an alias for DYNAMIC mode with RepositoryDiscovery enabled.

 

Additional Parameters

Under some circumstances DYNAMIC mode may not work in some environments. This will usually be due to the networking configuration on the server. By default, JGroups will use IPv6 if it is available. IPv4 can be forced, which may allow for it to work correctly. Add the following to catalina.sh/catalina.bat file:

 

Catalina.sh

JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"

  

Catalina.bat

set JAVA_OPTS=%JAVA_OPTS% -Djava.net.preferIPv4Stack=true

  

When using the default Jgroups configuration (udp.xml), the multicast address and port can also be configured.  This may help in environments where cluster nodes are not discovering each other.

 

Catalina.sh

JAVA_OPTS="$JAVA_OPTS -Djgroups.udp.mcast_addr=228.0.0.5 -Djgroups.udp.mcast_port=47885"

  

Catalina.bat

set JAVA_OPTS=%JAVA_OPTS% -Djgroups.udp.mcast_addr=228.0.0.5 -Djgroups.udp.mcast_port=47885

 

 

Background Tasks

Each Yellowfin node will be configured by default to run background tasks, which includes the broadcasting of reports. This potentially could send reports multiple times to end users. Because of this it is recommended that background tasks be enabled on one node only.

 

System Tasks

Disable the background system tasks by removing (or commenting-out) the following XML block from the web.xml. This will disable system tasks, like Group Flattening, LDAP synchronization, Event Archiving, Document Cleanup and Average Run time calculations.

 

 
<servlet> 
      <servlet-name>SystemTaskManager</servlet-name> 
      <servlet-class>com.hof.servlet.SystemTaskManager</servlet-class> 
      <load-on-startup>8</load-on-startup> 
</servlet>
 

 

Task Scheduler

Disable the task scheduler by adding the following to the web.xml file, inside the MIStartup Servlet block.

This will disable all tasks visible in the Task Schedule menu within the Yellowfin interface. This includes broadcasting, populating cached filters, scheduled reports and populating cached views.

 

 
<init-param> 
        <param-name>DisableTaskScheduler</param-name> 
        <param-value>TRUE</param-value> 
</init-param>
 

 

 

Dynamic Delegation

In DYNAMIC and REPOSITORY modes, delegation of background tasks can be done dynamically. If a node running background tasks fails, tasks will be delegated to another node.

 

 
<init-param> 
        <param-name>AutoTaskDelegation</param-name> 
        <param-value>TRUE</param-value> 
</init-param>
 

 

 

Container Level Session Replication

Container Level Session Replication allows Yellowfin to be deployed in a fail-safe environment where in the event of a node in the cluster failing, a currently connected Yellowfin user can continue their session on another node. This is achieved with Session Replication at the Application Server level, and is not a function of Yellowfin itself, and hence is not a supported component of Yellowfin. This configuration also requires a load-balancer to evenly distribute incoming requests to different cluster nodes, which will not be addressed in this guide.

 

Different Application Servers may have different ways of implementing this functionality. This example is shown using Tomcat, which is shipped with Yellowfin.

Setting up session replication consists of the following steps:

  1. Modifications to web.xml
  2. Modifications to server.xml
  3. Modifications to ROOT.xml

 

More information regarding Tomcat Session Replication can be found at http://tomcat.apache.org

 

Caveats

There are several caveats to using Session Replication with Yellowfin (or any Java web application). The two main downsides of Session Replication are processing overhead, and memory usage.

There is additional processing overhead in replicating each session to all nodes in the cluster. This happens after each HTTP request, and by default, only the changed session elements are copied to other nodes. This means all nodes will be doing additional work each time a user does something on any connected node.

The function of Session Replication is to replicate a user's memory footprint on the server, to all nodes in the cluster. This means the sessions for all users in the cluster will be stored on each node, taking up additional memory. Expanding the cluster with additional nodes is then restricted by the amount of memory that each node has.

It should be considered if this overhead is worth the added fail-safe functionality that it brings. If Session Replication is not used, and a node fails, a user’s session will be destroyed, but they'll be able to log in to another node to continue their work. 

 

Modifications to web.xml

The text “<distributable/>” needs to be added to the web.xml file on each node underneath the “<webapp>” line.

 

 
<web-app>
      <distributable/>
      <!-- System Event and Debug classes -->
      <listener> 
            <listener-class>com.hof.servlet.SysSessionListener</listener-class> 
      </listener> 
 

 

 

Modifications to server.xml

Changes need to be made to the server.xml file in the Yellowfin/appserver/conf directory.

Add the additional XML entry within the “<host>” XML block.

 

Add:

 <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>

 

The IP address “228.0.0.4” is a multicast address that session replication messages will be received on. This does not need to be modified, but a linux/unix server may need to register a valid multicast route to receive broadcasts on this address. This can be done by running the following command on the console:

sudo route add -net 224.0.0.0 netmask 224.0.0.0 dev eth0

 

This syntax may be different for different flavours of linux / unix. This example uses “eth0” as the network device that will be used for this route.

 

Modifications to ROOT.xml

Remove the lines from the ROOT.xml in the Yellowfin/appserver/conf/Catalina/localhost directory:

 

 
<Manager className="org.apache.catalina.session.PersistentManager" debug="0" distributable="false" saveOnRestart="false"> 
      <Store className="org.apache.catalina.session.FileStore" /> 
</Manager>
 



These lines were previously added to suppress session serialisation to disk in Yellowfin.

 

Additional Information

The Session Replication functionality in Tomcat may change in different versions or subversions. If the process outlined here is not working, check the Tomcat website for updated documentation for the version of Tomcat that Yellowfin is running on.

 

 

 

  • No labels