Use the information that is provided to help you solve problems that can occur in Content Based Routing.
Symptom | Possible Cause |
---|---|
Dispatcher not running correctly | Conflicting port numbers |
Connections from client machines not being served or connections timing out |
|
Dispatcher, Microsoft IIS, and SSL are not working or will not continue | Unable to send encrypted data across protocols |
The dscontrol or lbadmin command fails with Server not responding' or unable to access RMI server' message |
|
Advisors not working correctly | Advisors are not running |
Cannot Find the File..." error message, when running Netscape as default browser to view online help (Windows platform) | Incorrect setting for HTML file association |
Graphical user interface does not start correctly | Insufficient paging space |
Graphical user interface does not display correctly. | Resolution is incorrect. |
Help panels sometimes disappear behind other windows | Java limitation |
GUI hangs (or unexpected behavior) when trying to load a large configuration file. | Java does not have access to enough memory to handle such a large change to the GUI |
Korean Load Balancer interface displays overlapping or undesirable fonts on AIX® and Linux systems | Default fonts must be changed |
Unexpected GUI behavior when using Windows platform paired with Matrox AGP video card | Problem occurs when using Matrox AGP video cards while running the Load Balancer GUI |
Slow response time when running commands on the Dispatcher machine | Slow response time can be due to machine overloading from a high volume of client traffic |
SSL or HTTPS advisor not registering server loads | Problem occurs because the SSL server application not configured with the cluster IP address |
On Windows platform, corrupted Latin-1 national characters appear in command prompt | Change font properties of command prompt window |
On Windows platform, advisors and reach targets mark all servers down | Task offloading is not disabled or may need to enable ICMP. |
On Windows platform, advisors not working in a high availability setup after a network outage | When the system detects a network outage, it clears its Address Resolution Protocol (ARP) cache |
On Linux systems, "IP address add" command and multiple cluster loopback aliases are incompatible | When aliasing more than one address on the loopback device, should use ifconfig command, not ip address add |
On Solaris systems, Load Balancer processes end when you exit the terminal session window from which they started | Use the nohup command to prevent the processes that you started from receiving a hangup signal when you exit the terminal session. |
Slow down occurs when loading Load Balancer configurations | The delay might be due to Domain Name System (DNS) calls that are made to resolve and verify the server address. |
On Windows systems, the following error message appears: There is an IP address conflict with another system on the network | If high availability is configured, cluster addresses may be configured on both machines for a brief period which causes this error message to appear. |
On Windows systems, "Server not responding" error occurs when issuing a dscontrol or lbadmin command | When more than one IP address exists on a Windows system and the host file does not specify the address to associate with the hostname. |
Dispatcher MAC forwarding configuration limitations with zSeries® and S/390® platforms | On Linux, there are limitations when using zSeries or S/390 servers that have Open System Adapter (OSA) cards. Possible workarounds are provided. |
On Linux systems, iptables can interfere with the routing of packets | Linux iptables can interfere with load balancing of traffic and must be disabled on the Load Balancer machine. |
On Solaris systems, when you try to configure an IPv6 server on the Dispatcher machine, the message "unable to add server" appears | This can be caused by the way the Solaris operating system handles the ping request for an IPv6 address. |
Upgrading the Java fileset provided with the Load Balancer installations | If a problem is found with the Java file set, you should report the problem to IBM® Service so that you can receive an upgrade for the Java file set that was provided with the Load Balancer installation. |
Client requests fail when forwarding to HP-UX back-end servers | After setting up Load Balancer for IPv6 on the HP-UX operating system, client requests to the cluster address fail. This error is a result of the interaction between the neighbor discovery function for the operating system and the Load Balancer. |
Load Balancer for IPv4 and IPv6 conflicts with IP security (IPsec) | If you are using the Load Balancer for IPv4 and IPv6 with IP security (IPsec) enabled, output packets might be incorrect and dispatcher configuration information might display incorrectly in the command line interface and administrative console for WebSphere® Application Server. Load Balancer reports that it is forwarding connections, but clients do not receive responses. |
The serverUp script might run when you issue commands for Load Balancer that affect the status of servers | You might experience problems if you run a command that affects the status of a server, such as the dscontrol manager unquiesce and dscontrol manager quiesce commands, after a manager cycle has already retrieved the weights of the servers. If you run these commands, it might overwrite the values that are saved during the manager cycle and cause the serverUp script to run unexpectedly. |
This problem can occur when another application is using one of the ports used by the Load Balancer. For more information, read the Configuring the Load Balancer machine topic.
Use netstat
-ni to check.
Use netstat
-nr to check.
When using Dispatcher, Microsoft IIS, and SSL, if they do not work together, there may be a problem with enabling SSL security. For more information about generating a key pair, acquiring a certificate, installing a certificate with a key pair, and configuring a directory to require SSL, see the Microsoft Information and Peer Web Services documentation.
EXCLUDE-MODULE java
EXCLUDE-MODULE javaw
This can cause problems when one of the administration consoles runs on the same machine as a firewall or through a firewall. For example, when Load Balancer runs on the same machine as a firewall, and you issue dscontrol commands, you might see errors such as Error: Server not responding.
To avoid this problem, edit the dsserver script file to set the port used by RMI for the firewall (or other application). Change the line: LB_RMISERVERPORT=10199 to LB_RMISERVERPORT=yourPort. Where yourPort is a different port.
When complete, restart dsserver and open traffic for ports 10099, 10004, 10199, and 10100, or for the chosen port for the host address from which the administration console will be run.
For example: java -Djava.rmi.server.hostname="10.1.1.1"
LB_ADV_NO_PING="true"
java -DLB_ADV_NO_PING="true"
For Windows platforms, when using Netscape as your default browser, the following error message may result: Cannot find the file '<filename>.html' (or one of its components). Make sure the path and filename are correct and that all required libraries are available."
The graphical user interface (GUI), which is lbadmin, requires a sufficient amount of paging space to function correctly. If insufficient paging space is available, the GUI might not start up completely. If this occurs, check your paging space and increase it if necessary.
If you experience a problem with the appearance of the Load Balancer GUI, check the setting for the operating system's desktop resolution. The GUI is best viewed at a resolution of 1024x768 pixels.
On Windows platform, when you first open help windows, they sometimes disappear into the background behind existing windows. If this occurs, click on the window to bring it forward again.
When using lbadmin to load a large configuration file (roughly 200 or more add commands), the GUI may hang or display unexpected behavior, such as responding to screen changes at an extremely slow rate of speed.
This occurs because Java does not have access to enough memory to handle such a large configuration.
There is an option on the runtime environment that can be specified to increase the memory allocation pool available to Java.
The option is -Xmxn where n is the maximum size, in bytes, for the memory allocation pool. n must be a multiple of 1024 and must be greater than 2MB. The value n may be followed by k or K to indicate kilobytes, or m or M to indicate megabytes. For example, -Xmx128M and -Xmx81920k are both valid. The default value is 64M. Solaris 8 has a maximum value of 4000M.
java -Xmx256m -cp $LB_CLASSPATH $LB_INSTALL_PATH $LB_CLIENT_KEYS
com.ibm.internet.nd.framework.FWK_Main 1>/dev/null 2>&1 &
java -Xmx256m -cp $LB_CLASSPATH $LB_INSTALL_PATH $LB_CLIENT_KEYS
com.ibm.internet.nd.framework.FWK_Main 1>/dev/null 2>&1 &
javaw -Xmx256m -cp $LB_CLASSPATH $LB_INSTALL_PATH $LB_CLIENT_KEYS
com.ibm.internet.nd.framework.FWK_Main 1>/dev/null 2>&1 &
java -Xmx256m -cp $LB_CLASSPATH $LB_INSTALL_PATH $LB_CLIENT_KEYS
com.ibm.internet.nd.framework.FWK_Main 1>/dev/null 2>&1 &
START javaw -Xmx256m -cp %LB_CLASSPATH% %LB_INSTALL_PATH%
%LB_CLIENT_KEYS% com.ibm.internet.nd.framework.FWK_Main
There is no recommended value for n , but it should be greater than the default option. A good place to start would be with twice the default value.
-Monotype-TimesNewRomanWT-medium-r-normal
--*-%d-75-75-*-*-ksc5601.1987-0
-Monotype-SansMonoWT-medium-r-normal
--*-%d-75-75-*-*-ksc5601.1987-0
-monotype-
timesnewromanwt-medium-r-normal--*-%d-75-75-p-*-microsoft-symbol
-monotype-sansmonowt-medium-r-normal--*-%d-75-75-p-*-microsoft-symbol
On Windows platform when using a Matrox AGP card, unexpected behavior can occur in the Load Balancer GUI. When clicking the mouse, a block of space slightly larger than the mouse pointer can become corrupted causing possible highlighting reversal or images to shift out of place on the screen. Older Matrox cards have not shown this behavior. There is no known fix when using Matrox AGP cards.
If you are running the Dispatcher component for load balancing, it is possible to overload the computer with client traffic. The Load Balancer kernel module has the highest priority, and if it is constantly handling client packets, the rest of the system may become unresponsive. Running commands in user space may take a very long time to complete, or may never complete.
If this happens, you should begin to restructure your setup to avoid overloading the Load Balancer machine with traffic. Alternatives include spreading the load across several Load Balancer machines, or replacing the machine with a stronger and faster computer.
When trying to decide if the slow response time on the machine is due to high client traffic, consider whether this occurs during client peak traffic times. Misconfigured systems that cause routing loops can also cause the same symptoms. But before changing the Load Balancer setup, determine whether the symptoms may be due to high client load.
Load Balancer will send packets to the servers using the cluster address that is aliased on the loopback. Some server applications (such as SSL) require that configuration information, such as certificates, are based on the IP address. The IP address must be the cluster address which is configured on the loopback in order to match the contents of the incoming packets. If the IP address of the cluster is not used when configuring the server application, then the client request will not get properly forwarded to the server.
By default, when the Windows operating system detects a network outage, it clears its address resolution protocol (ARP) cache, including all static entries. After the network is available, the ARP cache is repopulated by ARP requests sent on the network.
With a high availability configuration, both servers take over primary operations when a loss of network connectivity affects one or both. When the ARP request is sent to repopulate the ARP cache, both servers respond, which causes the ARP cache to mark the entry as not valid. Therefore, the advisors are not able to create a socket to the backup servers.
Preventing the Windows operating system from clearing the ARP cache when there is a loss of connectivity solves this problem. Microsoft has published an article that explains how to accomplish this task. This article is on the Microsoft Web site, located in the Microsoft Knowledge Base, article number 239924: http://support.microsoft.com/default.aspx?scid=kb;en-us;239924.
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters
Certain considerations must be taken when using Linux kernel 2.4.x servers. If the server has a cluster address configured on the loopback device using the ip address add command, only one cluster address can be aliased.
ifconfig lo:num clusterAddress netmask 255.255.255.255 up
Additionally, there are incompatibilities between the ifconfig method of configuring interfaces and the ip method of configuring interfaces. Best practice suggests that a site choose one method and use that method exclusively.
On Solaris systems, after starting Load Balancer scripts (such as dsserver or lbadmin) from a terminal window, if you exit from that window, the Load Balancer process also exits.
To resolve this problem, start the Load Balancer scripts with the nohup command. For example: nohup dsserver. This command prevents the processes started from the terminal session from receiving a hangup signal from the terminal when it exits, allowing the processes to continue even after the terminal session has ended. Use the nohup command in front of any Load Balancer scripts that you want to continue to process beyond the end of a terminal session.
Loading a Load Balancer configuration might take a long time due to Domain Name System (DNS) calls that are made to resolve and verify the server address.
If the DNS of the Load Balancer machine is configured incorrectly, or if DNS in general takes a long time, this will cause a slow down in loading the configuration due to the Java processes that are sending DNS requests on the network.
A workaround for this is to add your server addresses and hostnames to your local /etc/hosts file.
If high availability is configured, the cluster addresses may be configured on both machines for a brief period and cause the following error message to occur: There is an IP address conflict with another system on the network. In this case, you can safely ignore the message. It is possible for a cluster address to be briefly configured on both high availability machines at the same time, especially during startup of either machine, or when a takeover has been initiated.
To resolve this problem, update the c:\Windows\system32\drivers\etc\hosts file with your machine host name and the IP address that you want to associate with the host name.
dscontrol host@@<ip_address or host_name> <command>
There is a limitation for zSeries and S/390 servers that share the OSA card, because this adapter operates differently than most network cards. The OSA card has its own virtual link layer implementation, which has nothing to do with ethernet, that is presented to the Linux and z/OS® hosts behind it. Effectively, each OSA card looks just like ethernet-to-ethernet hosts (and not to the OSA hosts), and hosts that use it will respond to it as if it is ethernet.
The OSA card also performs some functions that relate to the IP layer directly. Responding to ARP (address resolution protocol) requests is one example of a function that it performs. Another is that shared OSA routes IP packets based on destination IP address, instead of on ethernet address as a layer 2 switch. Effectively, the OSA card is a bridged network segment unto itself.
Load Balancer that runs on an S/390 Linux or zSeries Linux host can forward to hosts on the same OSA or to hosts on the ethernet. All the hosts on the same shared OSA are effectively on the same segment.
Load Balancer can forward out of a shared OSA because of the nature of the OSA bridge. The bridge knows the OSA port that owns the cluster IP. The bridge knows the MAC address of hosts directly connected to the ethernet segment. Therefore, Load Balancer can MAC-forward across one OSA bridge.
However, Load Balancer cannot forward into a shared OSA. This includes the Load Balancer on an S/390 Linux when the back-end server is on a different OSA card than the Load Balancer. The OSA for the back-end server advertises the OSA MAC address for the server IP, but when a packet arrives with the ethernet destination address of the server's OSA and the IP of the cluster, the server's OSA card does not know which of its hosts, if any, should receive that packet. The same principles that permit OSA-to-ethernet MAC-forwarding to work out of one shared OSA do not hold when trying to forward into a shared OSA.
Workaround:
If the servers in the Load Balancer configuration are on the same zSeries or S/390 platform type, you can define point-to-point (CTC or IUCV) connections between Load Balancer and each server. Set up the endpoints with private IP addresses. The point-to-point connection is used for Load Balancer-to-server traffic only. Then add the servers with the IP address of the server endpoint of the tunnel. With this configuration, the cluster traffic comes through the Load Balancer OSA card and is forwarded across the point-to-point connection where the server responds through its own default route. The response uses the server's OSA card to leave, which might or might not be the same card.
If the servers in the Load Balancer configuration are not on the same zSeries or S/390 platform type, or if it is not possible to define a point-to-point connection between Load Balancer and each server, it is recommended that you use Load Balancer's encapsulation feature, which is a protocol that permits Load Balancer to forward across routers.
When using encapsulation, the client->cluster IP packet is received by Load Balancer, encapsulated, and sent to the server. At the server, the original client->cluster IP packet is excapsulated, and the server responds directly to the client. The advantage with using GRE is that Load Balancer sees only the client-to-server traffic, not the server-to-client traffic. The disadvantage is that it lowers the maximum segment size (MSS) of the TCP connection due to encapsulation overhead.
Refer to the topic Use encapsulation forwarding to forward traffic across network segments for more information on how to configure Load Balancer to forward with encapsulation.
Linux iptables can interfere with load balancing of traffic and must be disabled on the Dispatcher machine.
lsmod | grep ip_tables
The
output from the preceding command might be similar to this: ip_tables 22400 3
iptable_mangle,iptable_nat,iptable_filter
Issue the following
command for each iptable listed in the output to display the rules
for the tables: iptables -t <short_name> -L
For
example: iptables -t mangle -L
iptables -t nat -L
iptables -t filter -L
If iptable_nat is loaded, it
must be unloaded. Because iptable_nat has a dependency on iptable_conntrack,
iptable_conntrack also must be removed. Issue the following command
to unload these two iptables: rmmod iptable_nat iptable_conntrack
On Solaris systems, when you try to configure an IPv6 server on a Websphere Load Balancer for IPv4 and IPv6 installation, you might receive the message unable to add server. This can be caused by the way the Solaris operating system handles the ping request for an IPv6 address.
When adding a server to the configuration on Solaris systems, Load Balancer tries to ping the server to obtain the MAC address of the server. The Solaris machine might choose a configured cluster address as the source address of the ping request, instead of using the NFA address of the machine. If the cluster address is configured on the server loopback, the ping response is not received at the Load Balancer machine; therefore, it does not add the server to the configuration.
The solution is to configure another IPv6 address on the Load Balancer machine either before or after configuring the IPv6 cluster address. This address must be an address that is not aliased on the loopback of the backend server on which you are trying to add to the Load Balancer configuration. Then add the server to the Load Balancer configuration.
During the Load Balancer installation process, a Java file set also gets installed. Load Balancer will be the only application that uses the Java version which installs with the product.You should not upgrade this version of the Java file set on your own. If there are problem which requires an upgrade for the Java file set, you should report the problem to IBM Service so the Java file set which is shipped within Load Balancer will be upgraded with an official fix level.
After setting up Load Balancer for IPv6 on the HP-UX operating system, client requests to the cluster address fail. This error is a result of the interaction between the neighbor discovery function for the operating system and the Load Balancer.
When a back-end server is added to the configuration, Load Balancer tries to ping the new server for the MAC address. The HP-UX server might choose a configured cluster address as the source address of the ping request, instead of using the nonforwarding address (NFA) of the machine. In this case, a new entry is created for the cluster address in the routing table of the back-end HP-UX server in addition to the local loopback entry. The new entry has a higher routing priority than the local loopback. Thus, the client requests that reach the back-end server are then redirected back to the Load Balancer server, which does not respond.
To work around this problem, after Load Balancer is set up completely, configure the loopback alias on the back-end server as a final step. If the cluster address is aliased on the loopback when the Load Balancer configuration is loaded, remove the cluster loopback alias on the back-end server completely and then re-alias it. To bring down the loopback alias, use the ifconfig lo0:1 inet6 command from the terminal window. Re-alias the loopback alias.
If you are using Load Balancer with IP security (IPsec) enabled, output packets might be incorrect and dispatcher configuration information might display incorrectly in the command line interface and administrative console for WebSphere Application Server. Load Balancer reports that it is forwarding connections, but clients do not receive responses.
If you are using Load Balancer function and IP security on the same host, there might be communication problems between Load Balancer and the application server. The Load Balancer component is not fully compatible with IPsec features and it transmits data from both sides of the security layer. Load Balancer receives packets below IPsec and, as a result, receives encrypted packets that it does not decrypt. When sending data, Load Balancer transmits them above IPsec, so it sends unencrypted packets to the application server that are encrypted on the other end by IPsec. The application server, therefore, receives encrypted data that cannot be used.
dscontrol manager quiesce server
dscontrol manager unquiesce server
If
the unquiesce command occurs after the manager has retrieved the weights,
the executor function overwrites the weight that is used to determine
if the server state has changed. This process does not cause any side
effects unless the server is also quiesced.The chances of experiencing this problem increase with larger configurations because the manager cycle takes longer to run. Also, there is a higher probability that the manager cycle will be in progress when the unquiesce command is issued.