[8.5.5.9 or later]
This topic applies to WebSphere Application Server Liberty V8.5.5.9 and earlier. For the latest Liberty topics, see the WebSphere Application Server Liberty documentation.

Using the Logstash collector

Use the Logstash collector feature in Liberty to collect log and other events from your Liberty servers and send them to a remote Logstash server. The collected events can be used for log analysis and troubleshooting purposes.

Before you begin

The logstashCollector-1.0 feature was tested with Logstash V2.x, Elasticsearch V2.x, and Kibana V4.x. You can use the logstashCollector-1.0 feature with a Logstash server that runs with any of the available output plug-ins from Logstash. However, many users choose to use Logstash V2.x with Elasticsearch V2.x and Kibana V4.x to provide a complete log consolidation and analysis facility. For more information, see Elasticsearch.

Procedure

  1. Set up Logstash V2.x by following the instructions from Elasticsearch.
  2. Create or acquire certificate and key pair files for SSL for Logstash. The following example is the command for openSSL that can be used for generating a certificate and key pair. Customize the number of days the keys are valid as required.
    openssl req -x509 -newkey rsa:2048 -keyout logstash.key -out logstash.crt -days 365 -nodes
  3. For Logstash V2.x and Elasticsearch users, copy the sample into a liberty_logstash_template.json file. See the repository for a sample Logstash index template. Customize the _ttl defaults as required to indicate the number of milliseconds to keep records of each event type.
  4. For Logstash V2.x and Elasticsearch users, copy the sample into a liberty_logstash.conf file. See the repository for sample Logstash filters. Customize lumberjack ssl_certificate path, ssl_key path, and port number as required. Customize Elasticsearch hosts and template path as required.
  5. Complete the following steps for each of the Liberty servers that you want to collect events from:
    1. Acquire or create a keystore for the Liberty server. To create a self-signed certificate use the following command. Customize the server name, password, and subject as required.
      d:\wlp\bin\securityUtility createSSLCertificate --server=myServerName --password="Liberty" --subject=CN=myHostname,OU=defaultServer,O=ibm,C=us
    2. Import the logstash.crt file from step 2 into your server's key.jks file. Customize the wlp_install_dir and server name as required. When prompted for a password, use the certificate password from step 5a.
      d:\java\bin\keytool -import -noprompt -alias logstash -file logstash.crt -keystore wlp_install_dir\usr\servers\myServerName\resources\security\key.jks
    3. Run the following command to install the logstashcollector-1.0 feature:
      d:\wlp\bin\installUtility install logstashcollector-1.0
    4. Configure Logstash collector in the server.xml file in Liberty by adding the following content. Customize the logstashCollector list of sources, host name, and port as required.
      <featureManager>
          <feature>logstashCollector-1.0</feature>
      </featureManager>
      
      <keyStore id="defaultKeyStore" password="Liberty" />
      
      <ssl id="mySSLConfig" trustStoreRef="defaultKeyStore" keyStoreRef="defaultKeyStore" />
      
      <logstashCollector 
          source="message,trace,garbageCollection,ffdc,accessLog" 
          hostName="localhost" 
          port="5043" 
          sslRef="mySSLConfig"
      />
      Note: Trace and access logs are usually high volume logs and require more network, CPU, and storage resources to collect.
  6. For users of Elasticsearch and Kibana V4.x, import the Kibana dashboard as follows:
    1. Save the Kibana dashboard JSON to a file on your local file system. For Elasticsearch and Kibana V4.x users, see the repository for a sample Kibana dashboard.
    2. Import the dashboard into Kibana by clicking Settings > Objects > Import.. When prompted provide the path to the file you saved in the previous step.
  7. Save the dashboard using the save (disk) icon. Enter "Liberty" in the text box that is provided and click the save icon near the text box. The next time that you visit Kibana from any browser you can reload this dashboard using the load icon and clicking Liberty.

Results

Your Liberty servers are configured to send events to your Logstash server, and you can now view your events in the Liberty dashboard using Kibana.

Icon that indicates the type of topic Task topic



Timestamp icon Last updated: Tuesday, 12 December 2017
http://www14.software.ibm.com/webapp/wsbroker/redirect?version=phil&product=was-libcore-mp&topic=twlp_analytics_logstash
File name: twlp_analytics_logstash.html