SDA - Server Directed Affinity

Intended Audience

This document is intended for Dispatcher customers who wish to write their own custom agents for Server Directed Affinity.

Background

For several releases, the Dispatcher product has supported the sticky port feature, also known as "affinity". This function was implemented by Dispatcher internally, and Dispatcher provided the affinity feature without external interaction. The original affinity feature is described herein as "classical" affinity. The new SDA feature provides an API which allows an external agent to influence the Dispatcher affinity behavior.

Purpose of Affinity

The load balancing function in Dispatcher is designed to distribute client requests among its multiple servers. If one client's transactions happen to use multiple TCP connections, it is very likely that those connections will be distributed to and handled by various servers. For most applications, such as serving web pages, this is a very desirable behavior. However, some protocols or applications would work better or more efficiently if Dispatcher were to send all of the connections from a single client to the same server for a period of time. For example, a "shopping-cart" server application which does not have the ability to share state information between all of the server machines would require that the client come back to the same server for all of the TCP connections during a purchase. This mapping of client IP addresses to selected server is the purpose for the classical Dispatcher affinity feature.

How classical Dispatcher affinity works

Configuration

The customer enables the affinity feature when she (or he) configures a cluster's port to be sticky. This is done by setting "port stickytime" to some number of seconds. The feature is disabled by setting stickytime to zero.

Behavior when disabled

With the feature disabled, whenever a new TCP connection is received from a client, Dispatcher picks the right server at that moment in time and forwards the packets to it. If a subsequent connection comes in from the same client, Dispatcher treats it as an unrelated new connection, and again picks the right server at that moment in time.

Behavior when enabled

With the feature enabled, when the first request from a new client comes in to a sticky port, Dispatcher selects a server and forwards packets as normal. In addition, because the port is sticky, Dispatcher also makes a record of the connection in one of its affinity tables. When a subsequent request comes in from the same client to the same port, rather than selecting a new server, Dispatcher will consult the affinity table, find the prior entry, and send the request to the same server as before. This satisfies the customer request that we send packets to the same server.

Over time, the client will finish all of his (or her) transactions, and the affinity record will go away. Hence the meaning of the sticky "time". Each affinity record lives for the "stickytime" in seconds. When subsequent connections are received within the stickytime, the affinity record is still valid and the request will go to the same server. If a subsequent connection is not received during the sticky time, the record is purged; a connection which is received after that time will have a new server selected for it (and a new affinity record created!).

Customer request for Server Directed Affinity (SDA)

Customers have requested the ability to influence the Dispatcher's server selection mechanism. They believe that their servers (and the databases behind them) have the a priori knowledge to best direct client requests to particular server machines. Rather than having a client be "stuck" to the same server as selected by Dispatcher's load balancing selection, customers have requested that the client to be "stuck" to the server of their choosing. The SDA feature provides this API.

Customers can now write their own software to implement an SDA agent, which communicates with a listener in Dispatcher. It can then manipulate the Dispatcher affinity tables to:

Records inserted in an affinity table by an SDA agent remain in the table indefinitely. They do not timeout. They are removed only when the SDA agent removes them or if an Dispatcher advisor detects that the server is unavailable.

Important: Since Dispatcher removes records from the affinity table when it detects a server is dead, the SDA agent is responsible for re-inserting the desired records when the server comes online again.

Note: Because the SDA code uses the IP address to indentify the server, SDA cannot be used in conjunction with the server partitioning feature of Network Dispatcher. In the ND configuration, all servers must have a unique IP address.


The SDA Feature

Dispatcher's SDA components

Dispatcher implements a new socket listener to accept and handle requests from an SDA agent. When an SDA agent opens a connection with Dispatcher, the listener will accept it and leave the connection open. Multiple requests and responses can flow over this persistent connection. The socket will close when the SDA agent closes it or if Dispatcher detects an unrecoverable error.

Inside Dispatcher, the listener takes each request from the SDA agent, communicates with the appropriate affinity table in the Dispatcher executor kernel, and prepares a response for the SDA agent.

Configuration

The Dispatcher SDA listener is automatically started when the customer starts ndserver. There is no special command to enable or disable the listener. While ndserver is running, the SDA listener is always enabled and listening.

To Enable SDA

Each cluster/port must be individually configured to allow SDA access. To enable SDA access, the customer must define "port stickytime" to be exactly one second. Example:
  ndcontrol port set MyCluster:MyPort stickytime 1

To Disable SDA

To disable SDA access, define stickytime to a value other than one.

Summary

Sticky time Resultant behavior
0 No affinity (normal load balancing)
1 Server Directed Affinity
Other Classical Affinity.

Port number

The port number on which Dispatcher listens is specified in the "ndserver" script file, under environment variable ND_AFFINITY_PORT. The default is 10005.

Logs

Information from the SDA listener is stored in the Dispatcher "server.log" file.

The SDA API

Scope

An SDA agent can issue requests for any configured cluster/port combination over one SDA connection with Dispatcher. In each request message, the SDA agent specifies the desired cluster address and port number, along with a command and associated data. The response pertains to that cluster/port pair. A subsequent request for the same or different cluster/port combinations will be handled by Dispatcher appropriately.

SDA Messages

Bits and Bytes

Message Flows

Message Definitions

Sample Code