Use the Logstash collector feature in Liberty to collect log and other events from your
Liberty servers and send them to a remote Logstash server. The collected events can be used for log
analysis and troubleshooting purposes.
Before you begin
The logstashCollector-1.0 feature was tested with Logstash V2.x, Elasticsearch V2.x, and Kibana
V4.x. You can use the logstashCollector-1.0 feature with a Logstash server that runs with any of the
available output plug-ins from Logstash. However, many users choose to use Logstash V2.x with
Elasticsearch V2.x and Kibana V4.x to provide a complete log consolidation and analysis facility.
For more information, see Elasticsearch.
Procedure
- Set up Logstash V2.x by following the instructions from Elasticsearch.
- Create or acquire certificate and key pair files for SSL for Logstash. The following example is
the command for openSSL that can be used for generating a certificate and key pair. Customize the
number of days the keys are valid as required.
openssl req -x509 -newkey rsa:2048 -keyout logstash.key -out logstash.crt -days 365 -nodes
- For Logstash V2.x and Elasticsearch users, copy the sample into a
liberty_logstash_template.json file. See the repository for a sample Logstash index template. Customize the _ttl
defaults as required to indicate the number of milliseconds to keep records of each event
type.
- For Logstash V2.x and Elasticsearch users, copy the sample into a
liberty_logstash.conf file. See the repository for sample Logstash filters. Customize lumberjack
ssl_certificate path, ssl_key path, and port number as
required. Customize Elasticsearch hosts and template path as required.
- Complete the following steps for each of the Liberty servers that you want to collect events
from:
- Acquire or create a keystore for the Liberty server. To create a self-signed certificate use
the following command. Customize the server name, password, and subject as required.
d:\wlp\bin\securityUtility createSSLCertificate --server=myServerName --password="Liberty" --subject=CN=myHostname,OU=defaultServer,O=ibm,C=us
- Import the logstash.crt file from step 2 into your server's key.jks file.
Customize the wlp_install_dir and server name as required. When prompted for a password, use the
certificate password from step 5a.
d:\java\bin\keytool -import -noprompt -alias logstash -file logstash.crt -keystore wlp_install_dir\usr\servers\myServerName\resources\security\key.jks
- Run the following command to install the logstashcollector-1.0 feature:
d:\wlp\bin\installUtility install logstashcollector-1.0
- Configure Logstash collector in the server.xml file in Liberty by adding
the following content. Customize the logstashCollector list of sources, host name, and port as
required.
<featureManager>
<feature>logstashCollector-1.0</feature>
</featureManager>
<keyStore id="defaultKeyStore" password="Liberty" />
<ssl id="mySSLConfig" trustStoreRef="defaultKeyStore" keyStoreRef="defaultKeyStore" />
<logstashCollector
source="message,trace,garbageCollection,ffdc,accessLog"
hostName="localhost"
port="5043"
sslRef="mySSLConfig"
/>
Note: Trace and access logs are usually high volume logs and require more network, CPU, and storage
resources to collect.
- For users of Elasticsearch and Kibana V4.x, import the Kibana dashboard as follows:
- Save the Kibana dashboard JSON to a file on your local file system. For Elasticsearch and
Kibana V4.x users, see the repository for a sample Kibana dashboard.
- Import the dashboard into Kibana by clicking . When prompted provide the path to the file you saved in the previous step.
- Save the dashboard using the save (disk) icon. Enter "Liberty" in the text box that is provided
and click the save icon near the text box. The next time that you visit Kibana from any browser you
can reload this dashboard using the load icon and clicking Liberty.
Results
Your Liberty servers are configured to send events to your Logstash server, and you can now
view your events in the Liberty dashboard using Kibana.