Tuning Java virtual machines

The application server is a Java based process and requires a Java virtual machine (JVM) environment to run and support the Java applications running on the application server. You can configure the Java runtime environment to tune performance and system resource usage. This topic applies to the IBM Technology for Java Virtual Machine. Refer to the topic Tuning the Classic JVM if you are using the IBM Developer Kit for Java that is provided with the i5/OS product.

Before you begin

About this task

A Java runtime environment provides the execution environment for Java based applications and servers such as WebSphere Application Server. Therefore the Java configuration plays a significant role in determining performance and system resource consumption for the product, and the applications that you are running.

Supported JVMs are available from different JVM providers. This includes
  • IBM virtual machine for Javas.

    The IBM Java 5.0 and newer versions include major improvements in virtual machine technology to provide significant performance and serviceability enhancements over IBM's earlier Java execution technology. Refer to the Web site http://www.ibm.com/software/webservers/appserv/was/performance.html for more information about this new technology.

Even though JVM tuning is dependent on the JVM provider you use, there are some general tuning concepts that apply to all JVMs. These general concepts include:

The following steps provide specific instructions on how to perform the following types of tuning for each JVM. The steps do not have to be performed in any specific order.

Procedure

  1. Optimize the startup and runtime performance

    In some environments, such as a development environment, it is more important to optimize the startup performance of your application server rather than the runtime performance. In other environments, it is more important to optimize the runtime performance. By default, IBM virtual machines for Java are optimized for runtime performance, while HotSpot based JVMs are optimized for startup performance.

    The Java Just-In-Time (JIT) compiler has a big impact on whether startup or runtime performance is optimized. The initial optimization level that the compiler uses influences the length of time it takes to compile a class method, and the length of time it takes to start the server. For faster startups, you should reduce the initial optimization level that the compiler uses. However if you reduce the initial optimization level, the runtime performance of your applications might be degraded because the class methods are now compiled at a lower optimization level.

    • -Xquickstart

      This setting influences how the IBM virtual machine for Java uses a lower optimization level for class method compiles. A lower optimization level provides for faster server startups, but lowers runtime performance. If this parameter is not specified, the IBM virtual machine for Java defaults to starting with a high initial optimization level for compiles, which results in faster runtime performance, but slower server starts.

      Default: High initial compiler optimization level
      Recommended: High initial compiler optimization level
      Usage: -Xquickstart provides faster server startup.
  2. Configure the heap size

    The heap size settings control garbage collection in the JVM that is provide with i5/OS. The initial heap size is a threshold that triggers new garbage collection cycles. For example, if the initial heap size is 10 MB, a new collection cycle is triggered as soon as the JVM detects that since the last collection cycle, 10 MB are allocated.

    Smaller heap sizes result in more frequent garbage collections than larger heap sizes. If the maximum heap size is reached, the garbage collector stops operating asynchronously, and user threads are forced to wait for collection cycles to complete.

    The maximum heap size can affect application performance. The maximum heap size specifies the maximum amount of object space the garbage collected heap can consume. If the maximum heap size is too small, performance might degrade significantly, or the application might receive out of memory errors when the maximum heap size is reached.

    The JVM has thresholds it uses to manage the JVM's storage. When the thresholds are reached, the garbage collector gets invoked to free up unused storage. Therefore, garbage collection can cause significant degradation of Java performance. Before changing the initial and maximum heap sizes, you should consider the following information:
    • In the majority of cases you should set the maximum JVM heap size to value higher than the initial JVM heap size. This allows for the JVM to operate efficiently during normal, steady state periods within the confines of the initial heap but also to operate effectively during periods of high transaction volume by expanding the heap up to the maximum JVM heap size. In some rare cases where absolute optimal performance is required you might want to specify the same value for both the initial and maximum heap size. This will eliminate some overhead that occurs when the JVM needs to expand or contract the size of the JVM heap. Make sure the region is large enough to hold the specified JVM heap.
    • Beware of making the Initial Heap Size too large. While a large heap size initially improves performance by delaying garbage collection, a large heap size ultimately affects response time when garbage collection eventually kicks in because the collection process takes more time.

    The IBM Developer Kit and Runtime Environment, Java2 Technology Edition, Version 5.0 Diagnostics Guide, that is available on the developerWorks Web site, provides additional information on tuning the heap size.

    To use the administrative console to configure the heap size:

    1. In the administrative console, click Servers > Application Servers > server.
    2. in the Server Infrastructure section , click Java and Process Management > Process Definition > Java Virtual Machine.
    3. Specify a new value in either the Initial heap size or the Maximum heap size field.

      You can also specify values for both fields if you need to adjust both settings.

      bestprac: For performance analysis, the initial and maximum heap sizes should be equal.

      The Initial heap size setting specifies, in megabytes, the amount of storage that is allocated for the JVM heap when the JVM starts. The Maximum heap size setting specifies, in megabytes, the maximum amount of storage that can be allocated to the JVM heap. Both of these settings have a significant effect on performance.



      The illustration represents three CPU profiles, each running a fixed workload with varying Java heap settings. In the middle profile, the initial and maximum heap sizes are set to 128MB. Four garbage collections occur. The total time in garbage collection is about 15% of the total run. When the heap parameters are doubled to 256MB, as in the top profile, the length of the work time increases between garbage collections. Only three garbage collections occur, but the length of each garbage collection is also increased. In the third profile, the heap size is reduced to 64MB and exhibits the opposite effect. With a smaller heap size, both the time between garbage collections and the time for each garbage collection are shorter. For all three configurations, the total time in garbage collection is approximately 15%. This example illustrates an important concept about the Java heap and its relationship to object utilization. There is always a cost for garbage collection in Java applications.

      Run a series of test experiments that vary the Java heap settings. For example, run experiments with 128MB, 192MB, 256MB, and 320MB. During each experiment, monitor the total memory usage. If you expand the heap too aggressively, paging can occur. If paging occurs, reduce the size of the heap or add more memory to the system. When all the runs are finished, compare the following statistics:
      • Number of garbage collection calls
      • Average duration of a single garbage collection call
      • Ratio between the length of a single garbage collection call and the average time between calls
      If the application is not over utilizing objects and has no memory leaks, the state of steady memory utilization is reached. Garbage collection also occurs less frequently and for short duration.
    4. Click Apply or OK.
    5. Save your changes to the master configuration.
    6. Stop and restart the application server.

    You can also use the following command line parameters to adjust these settings. These parameters apply to all supported JVMs and are used to adjust the minimum and maximum heap size for each application server or application server instance.

    • -Xms

      This setting controls the initial size of the Java heap. Properly tuning this parameter reduces the overhead of garbage collection, which improves server response time and throughput. For some applications, the default setting for this option might be too low, which causes a high number of minor garbage collections.

      Default: 50MB. This default value applies for both 31-bit and 64-bit configurations.
      Recommended: Workload specific, but higher than the default.
      Usage: -Xms256m sets the initial heap size to 256 megabytes.
    • -Xmx

      This setting controls the maximum size of the Java heap. Increasing this parameter increases the memory available to the application server, and reduces the frequency of garbage collection. Increasing this setting can improve server response time and throughput. However, increasing this setting also increases the duration of a garbage collection when it does occur. This setting should never be increased above the system memory available for the application server instance. Increasing the setting above the available system memory can cause system paging and a significant decrease in performance.

      Default: 256MB. This default value applies for both 31-bit and 64-bit configurations.
      Recommended: Workload specific, but higher than the default, depending on the amount of available physical memory.
      Usage: -Xmx512m sets the maximum heap size to 512 megabytes.
    • –Xlp64k

      This parameter can be used to allocate the heap using medium size pages, such as 64 KB. Using this virtual memory page size for the memory that an application requires can improve the performance and throughput of the application because of hardware efficiencies that are associated with a larger page size.

      [Updated in September 2011] i5/OS and AIX® provide rich support around 64 KB pages because 64 KB pages are intended to be general purpose pages. 64 KB pages are easy to enable, and applications might receive performance benefits when 64 KB pages are used. Starting with Java 6 SR 7, the Java heap is allocated with 64K pages by default. For Java 6 SR 6 or earlier, 4K pages is the default setting, This setting can be changed without changing the operating system configuration. However, it is recommended that you run your application servers in a separate storage pool if you use of 64KB pages. [Updated in September 2011]

      sep2011

      Recommended Use 64 KB page size whenever possible.

      i5/OS POWER5+ systems, and i5/OS Version 6, Release 1, support a 64 KB page size.

    • [Updated in September 2011] –Xlp4k

      This parameter can be used to allocate the heap using 4 KB pages. Using this virtual memory page size for the memory that an application requires, instead of 64 KB, might negatively impact performance and throughput of the application because of hardware inefficiencies that are associated with a smaller page size.

      [Updated in September 2011] Starting with Java 6 SR 7, the Java heap is allocated with 64K pages by default. For Java 6 SR 6 or earlier, 4K pages is the default setting, This setting can be changed without changing the operating system configuration. However, it is recommended that you run your application servers in a separate storage pool if you use of 64KB pages. [Updated in September 2011]

      sep2011

      Recommended Use -Xlp64k instead of -Xlp4k whenever possible.
      [Updated in September 2011]
      sep2011
  3. Tune Java memory
    Enterprise applications written in the Java language involve complex object relationships and utilize large numbers of objects. Although, the Java language automatically manages memory associated with object life cycles, understanding the application usage patterns for objects is important. In particular, you should verify that:
    • The application is not over utilizing objects.
    • The application is not leaking objects.
    • The Java heap parameters are set properly to handle a given object usage pattern.
    1. Check for over-utilization of objects.

      You can use the Tivoli Performance Viewer to check You can use the Tivoli Performance Viewer to observe the counters for the JVM runtime. This information indicates whether the application is overusing objects. Refer to the topic Enabling the Java virtual machine profiler data for more information JVMPI counters.

      You can also use the following tools to monitor JVM object creation:
      • The WRKJVMJOB command. WRKJVMJOB (Work JVM Jobs) command allows the user to list and monitor Java Virtual Machines running in active jobs. This command is available in i5/OS Version 6, Release 1, and higher.
      • The GENJVMDMP command. The (Generate JVM Dump) command generates JVM dumps for a specific job. This command is available in i5/OS Version 6, Release 1, and higher.

      The best result for the average time between garbage collections is at least 5-6 times the average duration of a single garbage collection. If you do not achieve this number, the application is spending more than 15 percent of its time in garbage collection.

      If the information indicates a garbage collection bottleneck, there are two ways to clear the bottleneck. The most cost-effective way to optimize the application is to implement object caches and pools. Use a Java profiler to determine which objects to target. If you can not optimize the application, adding memory, processors and clones might help. Additional memory allows each clone to maintain a reasonable heap size. Additional processors allow the clones to run in parallel.

    2. Test for memory leaks

      Memory leaks in the Java language are a dangerous contributor to garbage collection bottlenecks. Memory leaks are more damaging than memory overuse, because a memory leak ultimately leads to system instability. Over time, garbage collection occurs more frequently until the heap is exhausted and the Java code fails with a fatal out-of-memory exception. Memory leaks occur when an unused object has references that are never freed. Memory leaks most commonly occur in collection classes, such as Hashtable because the table always has a reference to the object, even after real references are deleted.

      High workload often causes applications to crash immediately after deployment in the production environment. This is especially true for leaking applications where the high workload accelerates the magnification of the leakage and a memory allocation failure occurs.

      The goal of memory leak testing is to magnify numbers. Memory leaks are measured in terms of the amount of bytes or kilobytes that cannot be garbage collected. The delicate task is to differentiate these amounts between expected sizes of useful and unusable memory. This task is achieved more easily if the numbers are magnified, resulting in larger gaps and easier identification of inconsistencies. The following list contains important conclusions about memory leaks:
      • Long-running test

        Memory leak problems can manifest only after a period of time, therefore, memory leaks are found easily during long-running tests. Short running tests can lead to false alarms. It is sometimes difficult to know when a memory leak is occurring in the Java language, especially when memory usage has seemingly increased either abruptly or monotonically in a given period of time. The reason it is hard to detect a memory leak is that these kinds of increases can be valid or might be the intention of the developer. You can learn how to differentiate the delayed use of objects from completely unused objects by running applications for a longer period of time. Long-running application testing gives you higher confidence for whether the delayed use of objects is actually occurring.

      • Repetitive test

        In many cases, memory leak problems occur by successive repetitions of the same test case. The goal of memory leak testing is to establish a big gap between unusable memory and used memory in terms of their relative sizes. By repeating the same scenario over and over again, the gap is multiplied in a very progressive way. This testing helps if the number of leaks caused by the execution of a test case is so minimal that it is hardly noticeable in one run.

        You can use repetitive tests at the system level or module level. The advantage with modular testing is better control. When a module is designed to keep the private module without creating external side effects such as memory usage, testing for memory leaks is easier. First, the memory usage before running the module is recorded. Then, a fixed set of test cases are run repeatedly. At the end of the test run, the current memory usage is recorded and checked for significant changes. Remember, garbage collection must be suggested when recording the actual memory usage by inserting System.gc() in the module where you want garbage collection to occur, or using a profiling tool, to force the event to occur.

      • Concurrency test

        Some memory leak problems can occur only when there are several threads running in the application. Unfortunately, synchronization points are very susceptible to memory leaks because of the added complication in the program logic. Careless programming can lead to kept or unreleased references. The incident of memory leaks is often facilitated or accelerated by increased concurrency in the system. The most common way to increase concurrency is to increase the number of clients in the test driver.

        Consider the following points when choosing which test cases to use for memory leak testing:
        • A good test case exercises areas of the application where objects are created. Most of the time, knowledge of the application is required. A description of the scenario can suggest creation of data spaces, such as adding a new record, creating an HTTP session, performing a transaction and searching a record.
        • Look at areas where collections of objects are used. Typically, memory leaks are composed of objects within the same class. Also, collection classes such as Vector and Hashtable are common places where references to objects are implicitly stored by calling corresponding insertion methods. For example, the get method of a Hashtable object does not remove its reference to the retrieved object.
      You can use these tools to detect memory leaks:
      • Tivoli Performance Viewer. Refer to the topic Enabling the Java virtual machine profiler data for more information about how to use this tool.
      • The WRKJVMJOB command. WRKJVMJOB (Work JVM Jobs) command allows the user to list and monitor Java Virtual Machines running in active jobs. This command is available in i5/OS Version 6, Release 1, and higher.
      • The GENJVMDMP command. The (Generate JVM Dump) command generates JVM dumps for a specific job. This command is available in i5/OS Version 6, Release 1, and higher.
      For the best results, repeat experiments with increasing duration, like 1000, 2000, and 4000 page requests. The Tivoli Performance Viewer graph of used memory should have a sawtooth shape. Each drop on the graph corresponds to a garbage collection. There is a memory leak if one of the following occurs:
      • The amount of memory used immediately after each garbage collection increases significantly. The sawtooth pattern looks more like a staircase.
      • The jagged pattern has an irregular shape.

      Also, look at the difference between the number of objects allocated and the number of objects freed. If the gap between the two increases over time, there is a memory leak.

      Heap consumption indicating a possible leak during a heavy workload (the application server is consistently near 100% CPU utilization), yet appearing to recover during a subsequent lighter or near-idle workload, is an indication of heap fragmentation. Heap fragmentation can occur when the JVM can free sufficient objects to satisfy memory allocation requests during garbage collection cycles, but the JVM does not have the time to compact small free memory areas in the heap to larger contiguous spaces.

      Another form of heap fragmentation occurs when small objects (less than 512 bytes) are freed. The objects are freed, but the storage is not recovered, resulting in memory fragmentation until a heap compaction has been run.

  4. Tune garbage collection

    Examining Java garbage collection gives insight to how the application is utilizing memory. Garbage collection is a Java strength. By taking the burden of memory management away from the application writer, Java applications are more robust than applications written in languages that do not provide garbage collection. This robustness applies as long as the application is not abusing objects. Garbage collection normally consumes from 5% to 20% of total execution time of a properly functioning application. If not managed, garbage collection is one of the biggest bottlenecks for an application.

    Monitoring garbage collection during the execution of a fixed workload, enables you to gain insight as to whether the application is over-utilizing objects. Garbage collection can even detect the presence of memory leaks.

    You can use JVM settings to configure the type and behavior of garbage collection. When the JVM cannot allocate an object from the current heap because of lack of contiguous space, the garbage collector is invoked to reclaim memory from Java objects that are no longer being used. Each JVM vendor provides unique garbage collector policies and tuning parameters.

    You can use the Verbose garbage collection setting in the administrative console to enable garbage collection monitoring. The output from this setting includes class garbage collection statistics. The format of the generated report is not standardized between different JVMs or release levels.

    To ensure meaningful statistics, run a fixed workload until the application state is steady. It usually takes several minutes to reach a steady state.

    You can also use object statistics in the Tivoli Performance Viewer to monitor garbage collection statistics.

    For more information about monitoring garbage collection, refer to the following documentation:
    • Performance: Resources for learning for a description of the IBM verbose:gc output
    • The description of the WRKJVMJOB command in the i5/OS Information Center. The WRKJVMJOB (Work JVM Jobs) command allows the user to list and monitor Java Virtual Machines running in active jobs. This command is available in i5/OS Version 6, Release 1 and higher.
    • The description of the GENJVMDMP command in the i5/OS Information Center. The GENJVMDMP (Generate JVM Dump) command generates JVM dumps for a specific job. This command is available in i5/OS Version 6, Release 1 and higher.

    To adjust your JVM garbage collection settings:

    1. In the administrative console, click Servers > Application Servers > server.
    2. Under Server Infrastructure, click Java and Process Management > Process Definition > Java Virtual Machine.
    3. Enter the –X option you want to change in the Generic JVM arguments field.
    4. Click OK.
    5. Save your changes to the master configuration.
    6. Stop and restart the application server.

    For more information about the –X options for the different JVM garbage collectors refer to the following:

    The IBM virtual machine for Java garbage collector.
    A complete guide to the IBM Java garbage collector is provided in the IBM Developer Kit and Runtime Environment, Java2 Technology Edition, Version 5.0 Diagnostics Guide. This document is available on the developerWorks Web site.
    • -Xnoclassgc

      By default, the JVM unloads a class from memory whenever there are no live instances of that class left. The overhead of loading and unloading the same class multiple times, can decrease performance.

      Avoid trouble Avoid trouble: You can use the -Xnoclassgc argument to disable class garbage collection. However, the performance impact of class garbage collection is typically minimal, and turning off class garbage collection in a Java Platform, Enterprise Edition (Java EE) based system, with its heavy use of application class loaders, might effectively create a memory leak of class data, and cause the JVM to throw an Out-of-Memory Exception.gotcha

      When this option is used, if you have to redeploy an application, you should always restart the application server to clear the classes and static data from the pervious version of the application.

      Default: Class garbage collection is enabled.
      Recommended:

      Do not disable class garbage collection.

      Usage: Xnoclassgc disables class garbage collection.
  5. [Updated in March 2012] Enable localhost name caching By default in the IBM® SDK for Java, the static method java/net/InetAddress.getLocalHost does not cache its result. This method is used throughout WebSphere® Application Server, but particularly in administrative agents such as the deployment manager and node agent. If the localhost address of a process will not change while it is running, then it is advised to use a built-in cache for the localhost lookup by setting the com.ibm.cacheLocalHost system property to the value true. Refer to the Java virtual machine custom properties topic in the information center for instructions on setting JVM custom properties on the various types of processes.
    Note: The address for servers configured using DHCP change over time. Do not set this property unless you are using statically assigned IP addresses for your server.
    Information Value
    Default com.ibm.cacheLocalHost = false
    Recommended com.ibm.cacheLocalHost = true (see description)
    Usage [Updated in September 2012] Specifying -Dcom.ibm.cacheLocalHost=true enables the getLocalHost cache [Updated in September 2012]
    sep2012
    [Updated in March 2012]
    mar2012
  6. Tune the configuration update process for a large cell configuration.
    In a large cell configuration, you might need to determine whether configuration update performance or consistency checking is more important. The deployment manager maintains a master configuration repository for the entire cell. By default, when the configuration changes, the product compares the configuration in the workspace with the master repository to maintain workspace consistency. However, the consistency verification process can cause an increase in the amount of time to save a configuration change or to deploy a large number of applications. The following factors influence how much time is required:
    • The more application servers or clusters there are defined in cell, the longer it takes to save a configuration change.
    • The more applications there are deployed in a cell, the longer it takes to save a configuration change.
    If the amount of time required to change a configuration change is unsatisfactory, you can add the config_consistency_check custom property to your JVM settings and set the value of this property to false. To set this custom property, complete the following steps:
    1. In the administrative console, click System administration > Deployment manager.
    2. Under Server Infrastructure, select Java and Process Management, and then click Process Definition.
    3. Under Additional Properties, click Java Virtual Machine > Custom Properties > New.
    4. Enter config_consistency_check in the Name field and false in the Value field.
    5. Click OK and then save these changes to the master configuration.
    6. Restart the server.
    Supported configurations Supported configurations: The config_consistency_check custom property affects the deployment manager process only. It does not affect other processes including the node agent and application server processes. The consistency check is not performed on these processes. However, within the SystemOut.log files for these processes, you might see a note that the consistency check is disabled. For these non-deployment manager processes, you can ignore this message.sptcfg

    If you are using the wsadmin command wsadmin -conntype none in local mode, you must set the config_consistency_check property to false before issuing this command.

What to do next

Each Java vendor provides detailed information on performance and tuning for their JVM. Use the following Web sites to obtain additional tuning information for a specific Java runtime environments:

If you use DB2, consider disabling SafepointPolling technology in the HP virtual machine for Java for HP-UX. Developed to ensure safepoints for Java threads, SafepointPolling technology generates a signal that can interfere with the signal between WebSphere Application Server and a DB2 database. When this interference occurs, database deadlocks often result. Prevent the interference by starting the JVM with the -XX:-SafepointPolling option, which disables SafepointPolling during runtime.




In this information ...


IBM Redbooks, demos, education, and more

(Index)

Use IBM Suggests to retrieve related content from ibm.com and beyond, identified for your convenience.

This feature requires Internet access.

Task topic Task topic    

Terms and conditions for information centers | Feedback

Last updatedLast updated: Aug 30, 2013 10:47:11 PM CDT
http://www14.software.ibm.com/webapp/wsbroker/redirect?version=pix&product=was-nd-iseries&topic=tprf_tunejvm_v61
File name: tprf_tunejvm_v61.html