[8.5.5.9 or later]
This topic applies to WebSphere Application Server Liberty V8.5.5.9 and earlier. For the latest Liberty topics, see the WebSphere Application Server Liberty documentation.

Logstash collector

Liberty generates various events at runtime, such as log events, trace events, first failure data capture (ffdc) events, access log events, and garbage collection events. It is helpful to consolidate events from all servers so the events can be searched, filtered, and analyzed, particularly when you are managing many servers, or when you are running servers on different platforms (for example on dedicated hardware and in the cloud). It can also be helpful to store events on a separate server in cases where you use Liberty in environments that lack persistent file storage for problem determination data. Liberty now provides the Logstash collector feature to help you remotely consolidate events. You can avoid running the agents on your Liberty server machine to collect your events using this feature. The collector captures in-flight events, breaks them into fields, and securely forwards the events to the configured Logstash server.

Logstash collector

The Logstash collector feature (logstashCollector-1.0) sends events to a Logstash server that you provide.
The logstash collector feature offers a flexible way to choose one or more of the following supported sources of data that needs to be sent to logstash:
  • message - messages log events
  • trace - traces log events
  • accesslog - Http access log events
  • ffdc - FFDC log events
  • garbageCollection - GarbageCollection events
Logstash can be used with the Elasticsearch search server and Kibana dashboard, all of which you provide, set up, and manage, to give a consolidated view of logs or other events from across your enterprise. Note that there are no separate processes or agents to be set up on the Liberty server machine when using the Logstash collector to forward events.
Figure 1. Logstash collector
This diagram shows the flow of events to the Elasticsearch server through the logstash event pipeline.

Event structure

The collectors send each event as a set of field name-value pairs. Each different type of event has its own set of fields. Knowing which fields each event has is useful when creating your own Kibana dashboards.
The following fields are common and present in all events:
type – a string that identifies the type of event
datetime – time at which the event occurred
hostName – host name of the server that was the source of the event
wlpUserDir – user directory of the server that was the source of the event, for example, D:\wlp\usr
serverName – server name of the server that was the source of the event
sequence – sequence number of event (useful for sorting records with the same time stamp)
Besides the common fields, each of the event types also has its own unique fields:
Message events (type:"liberty_message")
severity – 1 letter severity indicator (F = Fatal, E = Error, W = Warning, A = Audit, I = Info, O = SystemOut, R = SystemErr)
messageId – message ID in the log line, which can be used to find out specific types of errors, for example, SRVE0250I
methodName – method name from log record
className – class name from log record
loggerName – logger name from log record
threadId – thread ID in the log line, for example, 00000015. Note that the thread ID is a string and not a number
message – the message, starting with the message ID
Trace events (type:"liberty_trace")
severity – 1 letter severity indicator (1 = Fine, 2 = Finer, 3 = Finest, > = Entry, < = Exit)
methodName – method name from log record
className – class name from log record
loggerName – logger name from log record
threadId – thread ID in the log line, for example, 00000015. Note that the thread ID is a string and not a number
message – the message
HTTP access log events (type:"liberty_accesslog")
uriPath – Path information for the requested URL. This does NOT contain the query parameters, for example, /pushworksserver/ push/apps/tags
requestMethod – HTTP verb used, for example, GET
remoteHost – remote host IP address in the log line, for example, 127.0.0.1
userAgent – userAgent value in the request.
requestProtocol – Protocol information in the log line, for example, HTTP/1.1
queryString – string representing query string from the HTTP request, for example, color=blue&size=large
bytesReceived – bytes received in the URL, for example, 94 in the sample 1
responseCode – HTTP response code, for example, 200
elapsedTime – time that is taken to serve the request
requestHost – request host IP address in the log line, for example, 127.0.0.1
requestPort – port number of the request
FFDC events (type:"liberty_ffdc")
className – the class that emitted the FFDC entry
exceptionName – the exception that was reported in the FFDC entry
probeID – the unique identifier of the FFDC point within the class
stackTrace – the stack trace of the FFDC incident
objectDetails – the incident details for the FFDC incident
threadId – the thread ID of the ffdc incident
Garbage Collection events (type:"liberty_gc")
heap – the total heap available
usedHeap – the amount of heap used
duration – the duration for which GarbageCollection was run
gcType – the type of garbage collection event (for example: Nursery, Global, etc.)
reason – the reason for garbage collection

Icon that indicates the type of topic Concept topic



Timestamp icon Last updated: Tuesday, 12 December 2017
http://www14.software.ibm.com/webapp/wsbroker/redirect?version=phil&product=was-libcore-mp&topic=cwlp_analytics_collectors
File name: cwlp_analytics_collectors.html