Troubleshooting Content Based Router

Use the information that is provided to help you solve problems that can occur in Content Based Routing.

Click a link in the table to go to a full description and possible solution for the problem that you are experiencing.
Table 1. CBR Troubleshooting table
Symptom Possible cause
CBR not running correctly Conflicting port numbers
The cbrcontrol or lbadmin command fails with ‘Server not responding' or ‘unable to access RMI server' message Commands fail due to socksified stack. Or commands fail due to not starting cbrserver
Requests are not being load balanced Caching Proxy was started before the executor was started
On Solaris, the cbrcontrol executor start command fails with ‘Error: Executor was not started.' message Command fails because the system IPC defaults may need to be modified, or link to library is incorrect.
URL rule does not work Syntactical or configuration error
Unexpected GUI behavior when using Windows systems paired with Matrox AGP video card Problem occurs when using Matrox AGP video cards while running the Load Balancer GUI
GUI hangs (or unexpected behavior) when trying to load a large configuration file. Java does not have access to enough memory to handle such a large change to the GUI
On Windows platform, corrupted Latin-1 national characters appear in command prompt Change font properties of command prompt window
On HP-UX platform, the following message occurs: java.lang.OutOfMemoryError unable to create new native thread Some HP-UX installations by default allow 64 threads per process. This is insufficient.
On Windows platform, advisors and reach targets mark all servers down Task offloading is not disabled or may need to enable icmp.
On Windows platform, problem resolving IP address to host name when more than one address is configured to an adapter The IP address you want as your hostname must appear first in the registry.
On Solaris systems, Load Balancer processes end when you exit the terminal session window from which they started Use the nohup command to prevent the processes that you started from receiving a hangup signal when you exit the terminal session.

CBR will not run

This problem can occur when another application is using one of the ports used by CBR. For more information, go to Checking CBR port numbers.

cbrcontrol or lbadmin command fails

  1. The cbrcontrol command returns: Error: Server not responding. Or, the lbadmin command returns: Error: unable to access RMI server. These errors result when your machine has a socksified stack. To correct this problem, edit the socks.cnf file to contain the following lines:
    EXCLUDE-MODULE java
    EXCLUDE-MODULE javaw
  2. The administration consoles for Load Balancer interfaces (command line, graphical user interface, and wizards) communicate with cbrserver using remote method invocation (RMI). The default communication uses three ports; each port is set in the cbrserver start script:
    • ◦11099 to receive commands from cbrcontrol
    • ◦10004 to send metric queries to Metric Server
    • ◦11199 for the RMI server port

    This can cause problems when one of the administration consoles runs on the same machine as a firewall or through a firewall. For example, when Load Balancer runs on the same machine as a firewall, and you issue cbrcontrol commands, you might see errors such as Error: Server not responding.

    To avoid this problem, edit the cbrserver script file to set the port used by RMI for the firewall (or other application). Change the line: LB_RMISERVERPORT=11199 to LB_RMISERVERPORT=yourPort. Where yourPort is a different port.

    When complete, restart cbrserver and open traffic for ports 11099, 10004, 11199, and 11100, or for the chosen port for the host address from which the administration console runs.

  3. These errors can also occur if you have not already started cbrserver.

Requests not being load balanced

Caching Proxy and CBR have been started, but requests are not being load balanced. This error can occur if you start Caching Proxy before starting the executor. If this happens, the stderr log for Caching Proxy will contain the following error message: "ndServerInit: Could not attach to executor." To avoid this problem, start the executor before starting Caching Proxy.

[Solaris]

On Solaris systems, cbrcontrol executor start command fails

On Solaris systems, the cbrcontrol executor start command returns: "Error: Executor was not started." This error occurs if you do not configure the IPC (Inter-process Communication) for the system so that the maximum size of a shared memory segment and semaphore IDs are bigger than the operating system's default. In order to increase the size of the shared memory segment and semaphore IDs, you must edit the /etc/system file. For more information on how to configure this file, see the section on modifying the system defaults for IPCs (Inter-process Communication).

Syntactical or configuration error

If the URL rule does not work, this can be a result of either a syntactical or configuration error. For this problem check the following:
  • Verify the rule is configured correctly. See Appendix B. Content rule (pattern) syntax, for details.
  • Issue a cbrcontrol rule report for this rule, and check the ‘Times Fired' column to see if it has incremented according to the number of requests made. If it has incremented correctly, recheck the server configuration.
  • If the rule is not being fired, add an ‘always true' rule. Issue a cbrcontrol rule report on the ‘always true' rule to verify that it is getting fired.
[Windows]

On Windows platform, unexpected GUI behavior when using Matrox AGP video cards

On Windows platform when using a Matrox AGP card, unexpected behavior can occur in the Load Balancer GUI. When clicking the mouse, a block of space slightly larger than the mouse pointer can become corrupted causing possible highlighting reversal or images to shift out of place on the screen. Older Matrox cards have not shown this behavior. There is no known fix when using Matrox AGP cards.

Disconnect from host occurs when resize Netscape browser window while using Web administration

If you are using remote Web administration to configure Load Balancer, do not resize (Minimize, Maximize, Restore Down, and so on) the Netscape browser window in which the Load Balancer GUI appears. Because Netscape reloads a page every time browser windows are resized, this will cause a disconnect from host. You will need to reconnect to host each time you resize the window. If you are performing remote Web administration on a Windows platform, use Internet Explorer.

[Windows]

On Windows platform, corrupted Latin-1 national characters appear in command prompt window

In a command prompt window on the Windows operating system, some national characters of the Latin-1 family might appear corrupted. For example, the letter "a" with a tilde may display as a pi symbol. To fix this, you must change the font properties of the command prompt window. To change the font, do the following:
  1. Click the icon in the upper left corner of the command prompt window
  2. Select properties, then click the font tab
  3. The default font is raster fonts; change this to Lucida Console and click OK

On HP-UX, Java out of memory, thread error occurs

Some HP-UX 11i installations are pre-configured to allow only 64 threads per process. However, some Load Balancer configurations require more than this amount. For HP-UX systems, set the threads per process to at least 256. To increase this value, use the sam utility to set the max_thread_proc kernel parameter. If heavy use is expected, you might need to increase max_thread_proc beyond 256.

Increase the max_thread_proc parameter, as follows:
  1. Type sam from the command line
  2. Select Kernel Configuration > Configurable Parameters
  3. Select max_thread_proc
  4. Press the spacebar to highlight max_thread_proc
  5. Press Tab one time, then press the right-arrow key until you select Actions
  6. Press Enter to display the Actions menu, then press M to select Modify Configurable Parameter. (If you do not see this option, highlight max_thread_proc)
  7. Press Tab until you select the Formula/Value field
  8. Type a value of 256 or greater.
  9. Click OK
  10. Press Tab one time, then select Actions
  11. Press K for Process New Kernel.
  12. Select Yes
  13. Restart your system
[Windows]

On Windows systems, advisors and reach targets mark all servers down

When configuring your adapter on a Load Balancer machine, you must ensure that the following two settings are correct for the advisor to work:
  • Disable Task Offloading, which is most commonly used on 3Com adapter cards, as follows:
    1. Go to Start > Settings > Control Panel > Network and Dial-up Connections, then select the adapter.
    2. In the pop-up window, click Properties.
    3. Click Configure, then select the Advanced tab.
    4. In the property pane, select the Task Offload property, then select disable in the value field.
  • Enable Protocol 1 (ICMP) for IP protocols if you are enabling TCP/IP filtering. If ICMP is not enabled, the ping test to the backend server will not succeed. Check if ICMP is enabled, as follows:
    1. Go to Start > Settings > Control Panel > Network and Dial-up Connections, then select the adapter.
    2. In the pop-up window, click Properties.
    3. From the components pane, select Internet Protocol (TCP/IP), then click Properties.
    4. Click Advanced, then select the Options tab.
    5. Select TCP/IP filtering in the options pane, then click Properties.
    6. If you have selected Enable TCP/IP Filtering and permit only for IP protocols, you must add IP Protocol 1 in addition to the existing TCP and UDP ports that you enabled.
[Windows]

On Windows systems, resolving IP address to host name when more than one address is configured to an adapter

On Windows platform, when configuring an adapter with more than one IP address, configure the IP address that you want affiliated to the host name first in the registry.

Because Load Balancer is dependent on InetAddress.getLocalHost() in many instances (for example, lbkeys create), multiple IP addresses aliased to a single adapter might cause problems. To avoid this problem, list the IP address to which you want your host name to resolve first in the registry.

To address this issue, reorder the adapters in the Advanced Settings for the Control Panel’s Network Connections option. For example:
  1. Open the Control Panel.
  2. Open the Network Connections option.
  3. From the menu bar, select Advanced > Advanced Settings...
  4. Reorder the adapters that are listed in the Advanced Settings panel.
Concept topic    

Terms and conditions for information centers | Feedback

Last updated: April 16, 2014 11:59 AM EDT
File name: ctrb_cbr.html