<document>
<header>
<title>Resin Clustering</title>
<description>
<p>Resin's HTTP Web Server includes load balancing for scalability
and reliability.
</p>

</description>
</header>

<body>

<localtoc/>

<s1 name="resin" title="Using Resin as the Load Balancer">

<p>Resin Professional includes a
<a href="javadoc|com.caucho.servlets.LoadBalanceServlet">LoadBalanceServlet</a>
that can balance requests to backend servers.
Because it is implemented as a servlet,
this configuration is the most flexible. A site might use 192.168.0.1 as 
the frontend load balancer, and send all requests for /foo to the backend host
192.168.0.10 and all requests to /bar to the backend host 192.168.0.11. Since
Resin has an integrated HTTP proxy cache, the web-tier machine can cache
results for the backend servers.</p>

<p>Load balancing divides the Resin servers into two clusters:
the web-tier and the app-tier. In Resin 3.1, all the cluster 
and load balance configuration is in a single resin.conf.
The actual deployed server is selected with the <var>-server web-a</var>
<a href="config-cmdline.xtp">command-line argument</a>.</p>

<p>Using Resin as the load balancing web server requires a minimum
of two configuration files: one for the load balancing server, and one
for the backend servers.  The front configuration will dispatch
to the backend servers, while the backend will actually serve the
requests.</p>

<s2 title="The web-tier server does the load balancing">

<p>In the following example, there are three servers and two conf files.  The
first server (192.168.0.1), which uses web-tier.conf, is the load balancer.  It
has an &lt;http&gt; listener, it receives requests from browsers, and
dispatches them to the backend servers (192.168.0.10 and 192.168.0.11).</p>

<example title="resin.conf">
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;

&lt;cluster id="web-tier"&gt;
  &lt;server-default>
    &lt;http port="80"/&gt;
  &lt;/serve-default&gt;

  &lt;server id="web-a" address="192.168.0.1"/&gt;
  &lt;server id="web-b" address="192.168.0.1"/&gt;

  &lt;cache disk-size="1024M" memory-size="256M"/&gt;

  &lt;host id=""&gt;
    &lt;web-app id="/"&gt;
      &lt;!-- balance all requests to cluster app-tier --&gt;
      &lt;rewrite-dispatch>
        &lt;load-balance regexp="" cluster="app-tier"/>
      &lt;/rewrite-dispatch>
    &lt;/web-app&gt;
  &lt;/host&gt;
&lt;/cluster&gt;

&lt;cluster id="app-tier"&gt;
  &lt;server id="app-a" address="192.168.0.10" port="6800"/&gt;
  &lt;server id="app-b" address="192.168.0.11" port="6800"/&gt;

  &lt;persistent-store type="cluster"&gt;
    &lt;init path="cluster"/&gt;
  &lt;/persistent-store&gt;

  &lt;web-app-default&gt;
    &lt;session-config&gt;
      &lt;use-persistent-store/&gt;
    &lt;/session-config&gt;
  &lt;/web-app-default&gt;

  &lt;host id="www.foo.com"&gt;
    ...
  &lt;/host&gt;
&lt;/cluster&gt;

&lt;/resin&gt;
</example>

<p>The
LoadBalanceServlet selects a backend server using a round-robin policy.
Although the round-robin policy is simple, in practice it is as effective as
complicated balancing policies.  In addition, because it's simple, round-robin
is more robust and faster than adaptive policies.  </p>
</s2>

<s2 title="The backend server respond to the requests">

<p>A seperate conf file is used to configure all of the backend servers.
In this case, there are two backend servers, both configured in the conf file
<code>app-tier.conf</code>.</p>

<p>Sites using sessions will configure <a href="config-sessions.xtp">distributed
sessions</a> to make sure the users see the same session values.</p>

</s2>


<s2 title="Starting the servers">

<example title="Starting each server">
192.168.0.1&gt; java lib/resin.jar -server web-a start
192.168.0.2&gt; java lib/resin.jar -server web-b start

192.168.0.10&gt; java lib/resin.jar -server app-a start
192.168.0.11&gt; java lib/resin.jar -server app-b start
</example>

</s2>

</s1> <!-- resin -->

<s1 title="Dispatching">

<p>In most cases, the web-tier will dispatch
everything to the app-tier servers.  Because of Resin's
<a href="proxy-cache.xtp">proxy cache</a>, the web-tier servers
will serve static pages as fast as if they were local pages.</p>

<p>In some cases, though, it may be important to send different
requests to different backend clusters.  The
&lt;<a href="rewrite-tags.xtp#load-balance">load-balance</a>&gt; tag can
choose clusters based on URL patterns.</p>

<p>The following &lt;<a href="rewrite-tags.xtp">rewrite-dispatch</a>&gt;
keeps all *.png, *.gif, and *.jpg files on the web-tier, sends
everything in /foo/* to the foo-tier cluster, everything in /bar/* to
the bar-tier cluster, and keeps anything else on the web-tier.</p>

<example title="split dispatching">
&lt;resin xmlns="http://caucho.com/ns/resin">
  &lt;cluster id="web-tier">
    &lt;server id="web-a">
      &lt;http port="80"/>
    &lt;/server>

    &lt;cache memory-size="64m"/>

    &lt;host id="">
      &lt;web-app id="/">

        &lt;rewrite-dispatch>
          &lt;dispatch regexp="(\.png|\.gif|\.jpg)"/>

          &lt;load-balance regexp="^/foo" cluster="foo-tier"/>

          &lt;load-balance regexp="^/bar" cluster="bar-tier"/>
        &lt;/rewrite-dispatch>

      &lt;/web-app>
    &lt;/host>
  &lt;/cluster>

  &lt;cluster id="foo-tier">
    ...
  &lt;/cluster>

  &lt;cluster id="bar-tier">
    ...
  &lt;/cluster>
&lt;/resin>
</example>

</s1>

<s1 title="Distributed Sessions">

<p>A session needs to stay on the same JVM that started it.
Otherwise, each JVM would only see every second or third request and
get confused.</p>

<p>To make sure that sessions stay on the same JVM, Resin encodes the
cookie with the host number.  In the previous example, the hosts would
generate cookies like:</p>

<deftable>
<tr>
  <th>index</th>
  <th>cookie prefix</th>
</tr>
<tr>
  <td>1</td>
  <td><var>a</var>xxx</td>
</tr>
<tr>
  <td>2</td>
  <td><var>b</var>xxx</td>
</tr>
<tr>
  <td>3</td>
  <td><var>c</var>xxx</td>
</tr>
</deftable>

<p>On the web-tier, Resin will decode the cookie and send it
to the appropriate host.  So <var>bacX8ZwooOz</var> would go to app-b.</p>

<p>In the infrequent case that app-b fails, Resin will send the
request to app-a.  The user might lose the session but that's a minor
problem compared to showing a connection failure error.  To save sessions,
you'll need to use <a href="config-sessions.xtp">distributed sessions</a>.
Also take a look at <a href="tcp-sessions.xtp">tcp sessions</a>.</p>

<p>The following example is a typical configuration for a distributed
server using an external hardware load-balancer, i.e. where each Resin is
acting as the HTTP server.  Each server will be started
as <var>-server a</var> or <var>-server b</var> to grab its specific configuration.</p>

<p>In this example, sessions will only be stored when the server shuts down,
either for maintenance or with a new version of the server.  This is the most
lightweight configuration, and doesn't affect performance significantly.
If the hardware or the JVM crashes, however, the sessions will be lost.
(If you want to save sessions for hardware or JVM crashes,
remove the &lt;save-only-on-shutdown/&gt; flag.)</p>

<example title="resin.conf">
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
&lt;cluster id="app-tier"&gt;
  &lt;server-default>
    &lt;http port='80'/&gt;
  &lt;/server-default>

  &lt;server id='app-a' address='192.168.0.1'/&gt;
  &lt;server id='app-b' address='192.168.0.2'/&gt;
  &lt;server id='app-c' address='192.168.0.3'/&gt;

  &lt;persistent-store type="cluster"&gt;
    &lt;init path="cluster"/&gt;
  &lt;/persistent-store&gt;

  &lt;web-app-default&gt;
    &lt;!-- enable tcp-store for all hosts/web-apps --&gt;
    &lt;session-config&gt;
      &lt;use-persistent-store/&gt;
      &lt;save-only-on-shutdown/&gt;
    &lt;/session-config&gt;
  &lt;/web-app-default&gt;

  ...
&lt;/cluster&gt;
&lt;/resin&gt;
</example>

</s1>

<s1 title="Choosing a backend server">
<p>
Requests can be made to specific servers in the app-tier.  The web-tier uses
the value of the jsessionid to maintain sticky sessions.  You can include an
explicit jsessionid to force the web-tier to use a particular server in the app-tier.
</p>

<p>
Resin uses the first character of the jsessionid to identify the backend server
to use, starting with `a' as the first backend server.  If wwww.example.com
resolves to your web-tier, then you can use:
</p>

<ol>
<li>http://www.example.com/proxooladmin;jsessionid=abc</li>
<li>http://www.example.com/proxooladmin;jsessionid=bcd</li>
<li>http://www.example.com/proxooladmin;jsessionid=cde</li>
<li>http://www.example.com/proxooladmin;jsessionid=def</li>
<li>http://www.example.com/proxooladmin;jsessionid=efg</li>
<li>etc.</li>
</ol>
</s1>

<s1 title="&lt;persistent-store&gt;">
<p>Configuration for persistent store uses
the <a href="resin-tags.xtp#persistent-store">persistent-store</a> tag.</p>
</s1>

<s1 title="File Based">

<ul>
<li>For single-server configurations
</li><li>Useful in development when classes change often
</li></ul>

<p>Persistent sessions are configured in the <var>web-app</var>.
File-based sessions use <var>file-store</var>.</p>

<example>
&lt;web-app xmlns="http://caucho.com/ns/resin"&gt;
  &lt;session-config&gt;
    &lt;file-store&gt;WEB-INF/sessions&lt;/file-store&gt;
  &lt;/session-config&gt;
&lt;/web-app&gt;
</example>

<p>Sessions are stored as files in the <var>file-store</var>
directory.  When the session changes, the updates will be written to
the file.  After Resin loads an Application, it will load the stored
sessions.</p>

<p>File-based persistence is not useful in multi-server
environments.  Although a network filesystem such as NFS will allow all
the servers to access the same filesystem, it's not designed for the
fine-grained access.  For example, NFS will cache pages.  So if one
server modifies the page, e.g. a session value, the other servers may not see
the change for several seconds.</p>

</s1>

<s1 title="Distributed Sessions">

<p>Distributed sessions are intrinsically more complicated than single-server
sessions.  Single-server session can be implemented as a simple memory-based
Hashtable.  Distributed sessions must communicate between machines to ensure
the session state remains consistent.</p>

<p>Load balancing with multiple machines either uses <var>sticky sessions</var> or
<var>symmetrical sessions</var>.  Sticky sessions put more intelligence on the
load balancer, and symmetrical sessions puts more intelligence on the JVMs.
The choice of which to use depends on what kind of hardware you have,
how many machines you're using and how you use sessions.</p>

<p>Distributed sessions can use a database as a backing store, or they can
distribute the backup among all the servers using TCP.</p>

<s2 title="Symmetrical Sessions">

<p>Symmetrical sessions happen with dumb load balancers like DNS
round-robin.  A single session may bounce from machine A
to machine B and back to machine B.  For JDBC sessions, the symmetrical
session case needs the <var>always-load-session</var> attribute described below.
Each request must load the most up-to-date version of the session.</p>

<p>Distributed sessions in a symmetrical environment are required to make
sessions work at all.  Otherwise the state will end up spread across the JVMs.
However, because each request must update its session information, it is
less efficient than sticky sessions.</p>

</s2>
<s2 title="Sticky Sessions">

<p>Sticky sessions require more intelligence on the load-balancer, but
are easier for the JVM.  Once a session starts, the load-balancer will
always send it to the same JVM.  Resin's load balancing, for example, encodes
the session id as 'aaaXXX' and 'baaXXX'.  The 'aaa' session will always go
to JVM-a and 'baa' will always go to JVM-b.</p>

<p>Distributed sessions with a sticky session environment add reliability.
If JVM-a goes down, JVM-b can pick up the session without the user
noticing any change.  In addition, distributed sticky sessions are more
efficient.  The distributor only needs to update sessions when they change.
So if you update the session once when the user logs in, the distributed
sessions can be very efficient.</p>

</s2>

<s2 title="always-load-session">

<p>Symmetrical sessions must use the 'always-load-session' flag to
update each session data on each request.  always-load-session is only
needed for jdbc-store sessions.  tcp-store sessions use a more-sophisticated
protocol that eliminates the need for always-load-session, so tcp-store
ignores the always-load-session flag.</p>

<p>The <var>always-load-session</var> attribute forces sessions to check the store for
each request.  By default, sessions are only loaded from persistent
store when they are created.  In a configuration with multiple symmetric
web servers, sessions can be loaded on each request to ensure consistency.</p>

</s2>

<s2 title="always-save-session">

<p>By default, Resin only saves session data when you add new values
to the session object, i.e. if the request calls <var>setAttribute</var>.
This may be insufficient when storing large objects.  For example, if you
change an internal field of a large object, Resin will not automatically
detect that change and will not save the session object.</p>

<p>With <var>always-save-session</var> Resin will always write the session
to the store at the end of each request.  Although this is less efficient,
it guarantees that updates will get stored in the backup after each
request.</p>

</s2>

</s1>

<s1 title="Database Based">

<p>Database backed sessions are the easiest to understand.  Session data
gets serialized and stored in a database.  The data is loaded on the
next request.</p>

<p>For efficiency, the owning JVM keeps a cache of the session value, so
it only needs to query the database when the session changes.  If another JVM
stores a new session value, it will notify the owner of the change so
the owner can update its cache.  Because of this notification, the database
store is cluster-aware.</p>

<p>In some cases, the database can become a bottleneck.
By adding load to an already-loaded
system, you may harm performance.  One way around that bottleneck is to use
a small, quick database like MySQL for your session store and save the "Big
Iron" database like Oracle for your core database needs.</p>

<p>The database must be specified using a <var>&lt;database&gt;</var>.
The database store will automatically create a <var>session</var> table.</p>

<p>The JDBC store needs to know about the other servers in the cluster
in order to efficiently update them when changes occur to the server.</p>

<example title="JDBC store">
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
&lt;cluster id="app-tier"&gt;
  &lt;server-default>
    &lt;http port="80"/>
  &lt;/server-default>

  &lt;server id="app-a" address="192.168.2.10" port="6800"/>
  &lt;server id="app-b" address="192.168.2.11" port="6800"/>

  &lt;database jndi-name="jdbc/session"&gt;
    ...
  &lt;/database&gt;

  &lt;persistent-store type="jdbc"&gt;
    &lt;init&gt;
      &lt;data-source&gt;jdbc/session&lt;data-source&gt;
    &lt;/init&gt;
  &lt;/persistent-store&gt;
  ...

  &lt;web-app-default&gt;
    &lt;session-config&gt;
      &lt;use-persistent-store/&gt;
    &lt;/session-config&gt;
  &lt;/web-app-default&gt;
  ...
&lt;/cluster>
&lt;/resin>
</example>

<p>The persistent store is configured in the &lt;server&gt; with
<a href="resin-tags.xtp#persistent-store">persistent-store</a>.
Each web-app which needs distributed sessions must enable
the persistent store with a
<a href="webapp-tags.xtp#session-config">use-persistent-store</a>
tag in the session-config.</p>

<deftable>
<tr>
  <td>data-source</td>
  <td>data source name for the table</td>
</tr>
<tr>
  <td>table-name</td>
  <td>database table for the session data</td>
</tr>
<tr>
  <td>blob-type</td>
  <td>database type for a blob</td>
</tr>
<tr>
  <td>max-idle-time</td>
  <td>cleanup time</td>
</tr>
</deftable>

<example>
CREATE TABLE persistent_session (
  id VARCHAR(64) NOT NULL,
  data BLOB,
  access_time int(11),
  expire_interval int(11),
  PRIMARY KEY(id)
)
</example>

<p>The store is enabled with &lt;use-persistent-store&gt; in the session config.
</p>

<example>
&lt;web-app xmlns="http://caucho.com/ns/resin"&gt;
  &lt;session-config&gt;
    &lt;use-persistent-store/&gt;
    &lt;always-save-session/&gt;
  &lt;/session-config&gt;
&lt;/web-app&gt;
</example>

</s1>

<s1 title="Cluster Sessions">

<p>The distributed cluster stores the sessions across the
cluster servers.  In some configurations, the cluster store
may be more efficient than the database store, in others the database
store will be more efficient.</p>

<p>With cluster sessions, each session has an owning JVM and a backup JVM.
The session is always stored in both the owning JVM and the backup JVM.</p>

<p>The cluster store is configured in the in the &lt;cluster&gt;.
It uses the &lt;server&gt; hosts in the &lt;cluster&gt; to distribute
the sessions.  The session store is enabled in the &lt;session-config&gt;
with the &lt;use-persistent-store&gt;.</p>

<example>
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
  ...

  &lt;cluster id="app-tier"&gt;
    &lt;server id="app-a" host="192.168.0.1" port="6802"/>
    &lt;server id="app-b" host="192.168.0.2" port="6802"/>

    &lt;persistent-store type="cluster"&gt;
      &lt;init path="cluster"/&gt;
    &lt;/persistent-store&gt;

    ...
  &lt;/cluster>
&lt;/resin>
</example>

<p>The configuration is enabled in the <var>web-app</var>.</p>

<example>
&lt;web-app xmlns="http://caucho.com/ns/resin"&gt;
  &lt;session-config&gt;
    &lt;use-persistent-store="true"/&gt;
  &lt;/session-config&gt;
&lt;/web-app&gt;
</example>

<p>The &lt;srun&gt; and &lt;srun-backup&gt; hosts are treated as a cluster
of hosts.  Each host uses the other hosts as a backup.  When the session
changes, the updates will be sent to the backup host.  When the host starts, it
looks up old sessions in the other hosts to update its own version of the
persistent store.
</p>

<example title="Symmetric load-balanced servers">
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
&lt;cluster id="app-tier"&gt;

  &lt;server-default&gt;
    &lt;http port='80'/&gt;
  &lt;/server-default&gt;

  &lt;server id="app-a" address="192.168.2.10" port="6802"/>
  &lt;server id="app-b" address="192.168.2.11" port="6803"/>

  &lt;persistent-store type="cluster"&gt;
    &lt;init path="cluster"/&gt;
  &lt;/persistent-store&gt;

  &lt;host id=''&gt;
  &lt;web-app id=''&gt;

  &lt;session-config&gt;
    &lt;use-persistent-store="true"/&gt;
  &lt;/session-config&gt;

  &lt;/web-app&gt;
  &lt;/host&gt;
&lt;/cluster&gt;
&lt;/resin&gt;
</example>
</s1>

<s1 title="Clustered Distributed Sessions">
<p>Resin's cluster protocol for distributed sessions can
is an alternative to JDBC-based distributed sessions.  In some
configurations, the cluster-stored sessions will be more efficient
than JDBC-based sessions.
Because sessions are always duplicated on separate servers, cluster
sessions do not have a single point of failure.
As the number of
servers increases, JDBC-based sessions can start overloading the
backing database.  With clustered sessions, each additional server
shares the backup load, so the main scalability issue reduces to network
bandwidth.  Like the JDBC-based sessions, the cluster store sessions
uses sticky-session caching to avoid unnecessary network traffic.</p>
</s1>

<s1 title="Configuration">

<p>The cluster configuration must tell each host the servers in the
cluster
and it must enable the persistent in the session configuration
with <a href="session-tags.xtp#session-config">use-persistent-store</a>.
Because session configuration is specific to a virtual host and a
web-application, each web-app needs <var>use-persistent-store</var> enabled
individually.  The <a href="webapp-tags.xtp#web-app-default">web-app-default</a>
tag can be used to enable distributed sessions across an entire site.
</p>

<p>Most sites using Resin's load balancing will already have the cluster
<var>&lt;srun&gt;</var> configured.  Each <var>&lt;srun&gt;</var> block corresponds to a
host, including the current host.  Since cluster sessions uses
Resin's srun protocol, each host must listen for srun requests.</p>

<example title="resin.conf fragment">
&lt;resin xmlns="http://caucho.com/ns/resin">
  &lt;cluster id="app-tier"&gt;

    &lt;server id="app-a" host="192.168.0.1"/>
    &lt;server id="app-b" host="192.168.0.2"/>
    &lt;server id="app-c" host="192.168.0.3"/>
    &lt;server id="app-d" host="192.168.0.4"/>

    &lt;persistent-store type="cluster"&gt;
      &lt;init path="cluster"/&gt;
    &lt;/persistent-store&gt;

    ...
    &lt;host id=""&gt;
    &lt;web-app id='myapp'&gt;
      ...
      &lt;session-config&gt;
        &lt;use-persistent-store/&gt;
      &lt;/session-config&gt;
    &lt;/web-app&gt;
    &lt;/host&gt;
  &lt;/cluster&gt;
&lt;/resin&gt;
</example>

<p>Usually, hosts will share the same resin.conf.  Each host will be
started with a different <var>-server xx</var> to select the correct
block.  On Unix, startup will look like:</p>

<example title="Starting Host&#160;C on Unix">
resin-3.0.x&gt; bin/httpd.sh -conf conf/resin.conf -server c start
</example>

<p>On Windows, Resin will generally be configured as a service:</p>

<example title="Starting Host&#160;C on Windows">
resin-3.0.x&gt; bin/httpd -conf conf/resin.conf -server c -install-as ResinC
</example>

<s2 title="always-save-session">

<p>Resin's distributed sessions needs to know when a session has
changed in order to save the new session value.  Although Resin can
detect when an application calls <var>HttpSession.setAttribute</var>, it
can't tell if an internal session value has changed.  The following
Counter class shows the issue:</p>

<example title="Counter.java">
package test;

public class Counter implements java.io.Serializable {
  private int _count;

  public int nextCount() { return _count++; }
}
</example>

<p>Assuming a copy of the Counter is saved as a session attribute,
Resin doesn't know if the application has called <var>nextCount</var>.  If it
can't detect a change, Resin will not backup the new session, unless
<var>always-save-session</var> is set.  When <var>always-save-session</var> is
true, Resin will back up the session on every request.</p>

<example>
...
&lt;web-app id="/foo"&gt;
...
&lt;session-config&gt;
  &lt;use-persistent-store/&gt;
  &lt;always-save-session/&gt;
&lt;/session-config&gt;
...
&lt;/web-app&gt;
</example>

<p>Like the JDBC-based sessions, Resin will ignore the
<var>always-load-session</var> flag for cluster sessions.  Because the
cluster protocol notifies servers of changes, <var>always-load-session</var> is
not needed.</p>

</s2>

<s2 title="Serialization">

<p>Resin's distributed sessions relies on Java serialization to save and
restore sessions.  Application object must <var>implement
java.io.Serializable</var> for distributed sessions to work.</p>

</s2>

</s1>

<s1 title="Protocol Examples">

<s2 title="Session Request">

<p>To see how cluster sessions work, consider a case where
the load balancer sends the request to a random host.  Host&#160;C owns the
session but the load balancer gives the request to Host&#160;A.  In the
following figure, the request modifies the session so it must be saved
as well as loaded.</p>

<figure src="srunc.gif"/>

<p>The session id encodes the owning host.  The example session
id, <var>ca8MbyA</var>, decodes to an srun-index of 3, mapping
to Host&#160;C.  Resin determines the backup host from the cookie
as well.
Host&#160;A must know the owning host
for every cookie so it can communicate with the owning srun.
The example configuration defines all the sruns Host&#160;A needs to
know about.  If Host&#160;C is unavailable, Host&#160;A can use its
configuration knowledge to use Host&#160;D as a backup
for <var>ca8MbyA</var> instead..</p>

<p>When the request first accesses the session, Host&#160;A asks
Host&#160;C for the serialized session data (<var>2:load</var>).
Since Host&#160;A doesn't cache the session data, it must
ask Host&#160;C for an update on each request.  For requests that
only read the session, this TCP load is the only extra overhead,
i.e. they can skip <var>3-5</var>.  The <var>always-save-session</var>
flag, in contrast, will always force a write.</p>

<p>At the end of the request, Host&#160;A writes any session
updates to Host&#160;C (<var>3:store</var>). If always-save-session
is false and the session doesn't change, this step can be skipped.
Host&#160;A sends
the new serialized session contents to Host&#160;C.  Host&#160;C saves
the session on its local disk (<var>4:save</var>) and saves a backup
to Host&#160;D (<var>5:backup</var>).</p>

</s2>

<s2 title="Sticky Session Request">

<p>Smart load balancers that implement sticky sessions can improve
cluster performance.  In the previous request, Resin's cluster
sessions maintain consistency for dumb load balancers or twisted
clients like the AOL browsers.  The cost is the additional network
traffic for <var>2:load</var> and <var>3:store</var>.  Smart load-balancers
can avoid the network traffic of <var>2</var> and <var>3</var>.</p>

<figure src="same_srun.gif"/>

<p>Host&#160;C decodes the session id, <var>caaMbyA</var>.  Since it owns
the session, Host&#160;C gives the session to the servlet with no work
and no network traffic.  For a read-only request, there's zero
overhead for cluster sessions.  So even a semi-intelligent load
balancer will gain a performance advantage.  Normal browsers will have
zero overhead, and bogus AOL browsers will have the non-sticky
session overhead.</p>

<p>A session write saves the new serialized session to disk
(<var>2:save</var>) and to Host&#160;D (<var>3:backup</var>).
<var>always-save-session</var> will determine if Resin can take advantage
of read-only sessions or must save the session on each request.</p>

</s2>

<s2 title="Disk copy">
<p>Resin stores a disk copy of the session information, in the location
specified by the <var>path</var>.  The disk copy serves two purposes.  The first is
that it allows Resin to keep session information for a large number of
sessions. An efficient memory cache keeps the most active sessions in memory
and the disk holds all of the sessions without requiring large amounts of
memory.  The second purpose of the disk copy is that the sessions are recovered
from disk when the server is restarted.</p>
</s2>

<s2 title="Failover">

<p>Since the session always has a current copy on two servers, the load
balancer can direct requests to the next server in the ring.  The
backup server is always ready to take control.  The failover will
succeed even for dumb load balancers, as in the non-sticky-session
case, because the srun hosts will use the backup as the new owning
server.</p>

<p>In the example, either Host&#160;C or Host&#160;D can stop and
the sessions will use the backup.  Of course, the failover will work
for scheduled downtime as well as server crashes.  A site could
upgrade one server at a time with no observable downtime.</p>

</s2>

<s2 title="Recovery">

<p>When Host&#160;C restarts, possibly with an upgraded version of Resin,
it needs to use the most up-to-date version of the session; its
file-saved session will probably be obsolete.  When a "new" session
arrives, Host&#160;C loads the saved session from both the file and
from Host&#160;D.  It will use the newest session as the current
value.  Once it's loaded the "new" session, it will remain consistent
as if the server had never stopped.</p>

</s2>

<s2 title="No Distributed Locking">

<p>Resin's cluster sessions does not lock sessions.  For browser-based
sessions, only one request will execute at a time.  Since browser
sessions have no concurrently, there's no need for distributed
locking.  However, it's a good idea to be aware of the lack of
distributed locking.</p>

</s2>

</s1>

<s1 title="&lt;persistent-store&gt;">
<p>Configuration for persistent store uses
the <a href="resin-tags.xtp#persistent-store">persistent-store</a> tag.</p>
</s1>

<s1 title="File Based">

<ul>
<li>For single-server configurations
</li><li>Useful in development when classes change often
</li></ul>

<p>Persistent sessions are configured in the <var>web-app</var>.
File-based sessions use <var>file-store</var>.</p>

<example>
&lt;web-app xmlns="http://caucho.com/ns/resin"&gt;
  &lt;session-config&gt;
    &lt;file-store&gt;WEB-INF/sessions&lt;/file-store&gt;
  &lt;/session-config&gt;
&lt;/web-app&gt;
</example>

<p>Sessions are stored as files in the <var>file-store</var>
directory.  When the session changes, the updates will be written to
the file.  After Resin loads an Application, it will load the stored
sessions.</p>

<p>File-based persistence is not useful in multi-server
environments.  Although a network filesystem such as NFS will allow all
the servers to access the same filesystem, it's not designed for the
fine-grained access.  For example, NFS will cache pages.  So if one
server modifies the page, e.g. a session value, the other servers may not see
the change for several seconds.</p>

</s1>

<s1 title="Distributed Sessions">

<p>Distributed sessions are intrinsically more complicated than single-server
sessions.  Single-server session can be implemented as a simple memory-based
Hashtable.  Distributed sessions must communicate between machines to ensure
the session state remains consistent.</p>

<p>Load balancing with multiple machines either uses <var>sticky sessions</var> or
<var>symmetrical sessions</var>.  Sticky sessions put more intelligence on the
load balancer, and symmetrical sessions puts more intelligence on the JVMs.
The choice of which to use depends on what kind of hardware you have,
how many machines you're using and how you use sessions.</p>

<p>Distributed sessions can use a database as a backing store, or they can
distribute the backup among all the servers using TCP.</p>

<s2 title="Symmetrical Sessions">

<p>Symmetrical sessions happen with dumb load balancers like DNS
round-robin.  A single session may bounce from machine A
to machine B and back to machine B.  For JDBC sessions, the symmetrical
session case needs the <var>always-load-session</var> attribute described below.
Each request must load the most up-to-date version of the session.</p>

<p>Distributed sessions in a symmetrical environment are required to make
sessions work at all.  Otherwise the state will end up spread across the JVMs.
However, because each request must update its session information, it is
less efficient than sticky sessions.</p>

</s2>
<s2 title="Sticky Sessions">

<p>Sticky sessions require more intelligence on the load-balancer, but
are easier for the JVM.  Once a session starts, the load-balancer will
always send it to the same JVM.  Resin's load balancing, for example, encodes
the session id as 'aaaXXX' and 'baaXXX'.  The 'aaa' session will always go
to JVM-a and 'baa' will always go to JVM-b.</p>

<p>Distributed sessions with a sticky session environment add reliability.
If JVM-a goes down, JVM-b can pick up the session without the user
noticing any change.  In addition, distributed sticky sessions are more
efficient.  The distributor only needs to update sessions when they change.
So if you update the session once when the user logs in, the distributed
sessions can be very efficient.</p>

</s2>

<s2 title="always-load-session">

<p>Symmetrical sessions must use the 'always-load-session' flag to
update each session data on each request.  always-load-session is only
needed for jdbc-store sessions.  tcp-store sessions use a more-sophisticated
protocol that eliminates the need for always-load-session, so tcp-store
ignores the always-load-session flag.</p>

<p>The <var>always-load-session</var> attribute forces sessions to check the store for
each request.  By default, sessions are only loaded from persistent
store when they are created.  In a configuration with multiple symmetric
web servers, sessions can be loaded on each request to ensure consistency.</p>

</s2>

<s2 title="always-save-session">

<p>By default, Resin only saves session data when you add new values
to the session object, i.e. if the request calls <var>setAttribute</var>.
This may be insufficient when storing large objects.  For example, if you
change an internal field of a large object, Resin will not automatically
detect that change and will not save the session object.</p>

<p>With <var>always-save-session</var> Resin will always write the session
to the store at the end of each request.  Although this is less efficient,
it guarantees that updates will get stored in the backup after each
request.</p>

</s2>

</s1>

<s1 title="Database Based">

<p>Database backed sessions are the easiest to understand.  Session data
gets serialized and stored in a database.  The data is loaded on the
next request.</p>

<p>For efficiency, the owning JVM keeps a cache of the session value, so
it only needs to query the database when the session changes.  If another JVM
stores a new session value, it will notify the owner of the change so
the owner can update its cache.  Because of this notification, the database
store is cluster-aware.</p>

<p>In some cases, the database can become a bottleneck.
By adding load to an already-loaded
system, you may harm performance.  One way around that bottleneck is to use
a small, quick database like MySQL for your session store and save the "Big
Iron" database like Oracle for your core database needs.</p>

<p>The database must be specified using a <var>&lt;database&gt;</var>.
The database store will automatically create a <var>session</var> table.</p>

<p>The JDBC store needs to know about the other servers in the cluster
in order to efficiently update them when changes occur to the server.</p>

<example title="JDBC store">
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
&lt;cluster id="app-tier"&gt;
  &lt;server-default>
    &lt;http port="80"/>
  &lt;/server-default>

  &lt;server id="app-a" address="192.168.2.10" port="6800"/>
  &lt;server id="app-b" address="192.168.2.11" port="6800"/>

  &lt;database jndi-name="jdbc/session"&gt;
    ...
  &lt;/database&gt;

  &lt;persistent-store type="jdbc"&gt;
    &lt;init&gt;
      &lt;data-source&gt;jdbc/session&lt;data-source&gt;
    &lt;/init&gt;
  &lt;/persistent-store&gt;
  ...

  &lt;web-app-default&gt;
    &lt;session-config&gt;
      &lt;use-persistent-store/&gt;
    &lt;/session-config&gt;
  &lt;/web-app-default&gt;
  ...
&lt;/cluster>
&lt;/resin>
</example>

<p>The persistent store is configured in the &lt;server&gt; with
<a href="resin-tags.xtp#persistent-store">persistent-store</a>.
Each web-app which needs distributed sessions must enable
the persistent store with a
<a href="webapp-tags.xtp#session-config">use-persistent-store</a>
tag in the session-config.</p>

<deftable>
<tr>
  <td>data-source</td>
  <td>data source name for the table</td>
</tr>
<tr>
  <td>table-name</td>
  <td>database table for the session data</td>
</tr>
<tr>
  <td>blob-type</td>
  <td>database type for a blob</td>
</tr>
<tr>
  <td>max-idle-time</td>
  <td>cleanup time</td>
</tr>
</deftable>

<example>
CREATE TABLE persistent_session (
  id VARCHAR(64) NOT NULL,
  data BLOB,
  access_time int(11),
  expire_interval int(11),
  PRIMARY KEY(id)
)
</example>

<p>The store is enabled with &lt;use-persistent-store&gt; in the session config.
</p>

<example>
&lt;web-app xmlns="http://caucho.com/ns/resin"&gt;
  &lt;session-config&gt;
    &lt;use-persistent-store/&gt;
    &lt;always-save-session/&gt;
  &lt;/session-config&gt;
&lt;/web-app&gt;
</example>

</s1>

<s1 title="Cluster Sessions">

<p>The distributed cluster stores the sessions across the
cluster servers.  In some configurations, the cluster store
may be more efficient than the database store, in others the database
store will be more efficient.</p>

<p>With cluster sessions, each session has an owning JVM and a backup JVM.
The session is always stored in both the owning JVM and the backup JVM.</p>

<p>The cluster store is configured in the in the &lt;cluster&gt;.
It uses the &lt;server&gt; hosts in the &lt;cluster&gt; to distribute
the sessions.  The session store is enabled in the &lt;session-config&gt;
with the &lt;use-persistent-store&gt;.</p>

<example>
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
  ...

  &lt;cluster id="app-tier"&gt;
    &lt;server id="app-a" host="192.168.0.1" port="6802"/>
    &lt;server id="app-b" host="192.168.0.2" port="6802"/>

    &lt;persistent-store type="cluster"&gt;
      &lt;init path="cluster"/&gt;
    &lt;/persistent-store&gt;

    ...
  &lt;/cluster>
&lt;/resin>
</example>

<p>The configuration is enabled in the <var>web-app</var>.</p>

<example>
&lt;web-app xmlns="http://caucho.com/ns/resin"&gt;
  &lt;session-config&gt;
    &lt;use-persistent-store="true"/&gt;
  &lt;/session-config&gt;
&lt;/web-app&gt;
</example>

<p>The &lt;srun&gt; and &lt;srun-backup&gt; hosts are treated as a cluster
of hosts.  Each host uses the other hosts as a backup.  When the session
changes, the updates will be sent to the backup host.  When the host starts, it
looks up old sessions in the other hosts to update its own version of the
persistent store.
</p>

<example title="Symmetric load-balanced servers">
&lt;resin xmlns="http://caucho.com/ns/resin"&gt;
&lt;cluster id="app-tier"&gt;

  &lt;server-default&gt;
    &lt;http port='80'/&gt;
  &lt;/server-default&gt;

  &lt;server id="app-a" address="192.168.2.10" port="6802"/>
  &lt;server id="app-b" address="192.168.2.11" port="6803"/>

  &lt;persistent-store type="cluster"&gt;
    &lt;init path="cluster"/&gt;
  &lt;/persistent-store&gt;

  &lt;host id=''&gt;
  &lt;web-app id=''&gt;

  &lt;session-config&gt;
    &lt;use-persistent-store="true"/&gt;
  &lt;/session-config&gt;

  &lt;/web-app&gt;
  &lt;/host&gt;
&lt;/cluster&gt;
&lt;/resin&gt;
</example>
</s1>

<s1 title="See Also">

<ul>
<li><a href="config-sessions.xtp">Distributed Sessions</a>
</li><li><a href="tcp-sessions.xtp">Distributed Sessions with Cluster Store</a>
</li><li><a href="cluster-tags.xtp#persistent-store">&lt;persistent-store&gt;</a>
</li></ul>

</s1>

  </body>
</document>
