XD Custom Log – 6.1.0.3/6.1.0.4 feature interim fix

Configuration 

Users will be able to specify: 
   •	data to include
   •	order of data
   •	conditional logic to filter which requests are included
   •	number of log files
   •	names of log files

There will be a ruleset, which is a collection of rules and log actions. 

Rules will be used to determine the conditions upon which log actions will be executed. An expression field will be used to determine the requests/responses to log. This expression uses the standard HTTP language operands, plus operand "response.code" to filter by HTTP response codes, i.e.404, 503, etc.  For example, “response.code = 200” would get all responses of response code 200.  Rules will have priorities to determine the order in which they should be evaluated. 

Finally, there are log actions which will be done when rules are matched. These log actions will have a 'continue' field. When continue=true, subsequent log actions may still be executed. However if continue=false, once this log action is done, no subsequent log actions will be done.  Log actions have a priority attribute, but it is not used at this time. 

Custom property objects will be allowed at the ruleset, rule, and action level.

The configuration objects and attributes needed in the model are below (Note: "ruleset" is used instead of "rules" to avoid ambiguity with the already existing rules.xml.): 

ruleset 
   •	name required, String (to allow multiple rulesets) 
   •	type required, String (protocol identifier, ie. "HTTP", "SOAP")  
   •	continueDefault required, boolean (if set determines the default value for 'continue') 
   •	properties optional
   •	rule optional

rule 
   •	name required, String (unique within ruleset) 
   •	priority required, int (unique within ruleset) 
   •	expression required, String 
   •	action required
   •	properties optional

action 
   •	name required, String (unique within rule)
   •	priority optional, int (unique within rule) - not used at this time
   •	type required, String, log/foo1/foo2 
   •	value required, String (contains a series of logFormats delimited by ‘@’, for example: value="custom01.log %a %d@ custom02.log %A %D")
   •	continue optional, boolean (if not set, defaults to "continueDefault" value - true/false) 
   •	properties optional

customProperties 
   •	name required, String 
   •	value required, String 

The ruleset data object may be created on the ODR server, or on a cluster of ODRs. Ruleset.xml defined at the ODR will take precedence over ruleset.xml on the cluster. If no log rules are configured at all, then there will be no custom log - there is no default. 

Log Action Format 

The action's value attribute will contain a set of arguments to indicate the filename and format of the log. To create multiple logs in the value attribute, delimit the arguments with a "@".

The first argument for the log action is the name of the log file, relative to the ODR 's logs directory. 

The following arguments will be parameters to specify what information to place into the log. Also, the data will appear in the order that the parameters are specified. 

The HTTP request's data is logged by placing " % " parameters in the format string, which are replaced in the log entry by the values as follows: 
   •	%a : Remote IP -address 
   •	%A : Local IP -address 
   •	%B : Bytes sent, excluding HTTP headers. 
   •	%b : Bytes sent, excluding HTTP headers. In CLF format i.e. a '-' rather than a 0 when no bytes are sent 
   •	%{FOOBAR }C : The contents of cookie in the request sent to the server. 
   •	%{FOOBAR}e : The contents of the debug argument ("-D") FOOBAR specified for the JVM of the ODR process. 
   •	%h : Remote host 
   •	%H : The request protocol HTTP or HTTPS 
   •	%{FOOBAR}i : The contents of Foobar: header line(s) in the request sent to the server. 
   •	% I: Bytes received, including request and headers, cannot be zero 
   •	%m : The request method 
   •	%{FOOBAR}o : The contents of Foobar: header line(s) in the reply. 
   •	% O: Bytes sent, including headers, cannot be zero. 
   •	%p : The canonical Port of the server serving the request 
   •	%q : The query string (prepended with a ? if a query string exists, otherwise an empty string) 
   •	%r : First line of request 
   •	%s : Status, HTTP response code, i.e. 503, 404, 200
   •	%t : Time, in common log format time format (standard english format) 
   •	%{format}t : The time, in the form given by format, which should be in strftime(3) format. (potentially localized). See "Supported Strftime Format Conversions " below for details on the format. 
   •	%U : The URL path requested, not including any query string 
   •	%v : The canonical ServerName of the server serving the request 
   •	%z : Destination IP -address 
   •	%Z : Destination server name (cell/node/server) 

A "-" will be used when a field is blank. A " " will be used as the delimiter between fields. 


Administration 

Configuration of custom logs will be handled via the ruleset configuration objects. This can be done via wsadmin with the $AdminConfig create, modify, and remove commands. Validation here is limited to type checking. More thorough validation will be done during ODR startup, by issuing runtime exceptions to SystemOut. 

wsadmin 

The custom log must be configured at the wsadmin command line.  
Examples for configuring rulesets (bold is user input) follow : 


****Attributes lists the attributes for the config object: ****

wsadmin> $AdminConfig attributes Ruleset 
"properties Property (DiscoverableDescriptiveProperty, TypedProperty, DescriptiveProperty)*"
"defaultActions Action*"
"defaultContinue Boolean "
"name String "
"rules Rule*"
"type String " 

wsadmin> $AdminConfig attributes Rule "actions Action*"
"properties Property (DiscoverableDescriptiveProperty, TypedProperty, DescriptiveProperty)*"
"expression String "
"name String "
"priority Integer " 

wsadmin> $AdminConfig attributes Action "continue Boolean "
"properties Property (DiscoverableDescriptiveProperty, TypedProperty, DescriptiveProperty)*"
"name String "
"priority Integer "
"type String "
"value String "

wsadmin> 


*****Creates: *****

wsadmin> $AdminConfig create Ruleset (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1|server.xml#Server_1222124132765) { {name MyRuleset } {type HTTP } {defaultContinue true}} MyRuleset (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Ruleset_1222125392906) 

wsadmin> $AdminConfig create Rule (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Ruleset _1222125392906) {{name MyRule } {priority 0} {expression "virtualhost <> 'badvhost'"}} MyRule (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Rule_1222125775828) 

wsadmin> $AdminConfig create Action (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Rule_ 1222125775828) {{name MyAction } {type log} {continue false} {value "MyCustom.log %r %T %z %Z"}} MyAction (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Action_1222126008890)

wsadmin> $AdminConfig save 


****Modify would be something like: ****

$AdminConfig modify (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Rule_1222125775828) {{priority 1}} 


****Remove would be something like ****

$AdminConfig remove (cells/dabtcCell06/nodes/dabtcNode07/servers/odr1/ruleset/MyRuleset|ruleset.xml#Rule_1222125775828) 


Multiple Log Files 

Multiple log files can be created using separate log formats specifying different log file names. This can be done in one log action by separating log format with the "@" delimiter. For example, "custom1.log %a %b@custom2.log %Z %T %A". Also, users can create logs filtered by separate rules, such as multiple rules with expressions checking virtual hosts. 


StrfTime Format Conversions 

The format used when using the %{format}t log parameter is based on the non-extended BSD strftime(3) time conversion functions. The parameters which are specifically supported, along with the sample output, are listed below. Note that selected locale-specific parameters are supported as American English only -- specifically, the order in which month, day, and year appear, and perhaps the ordering of other items, will remain American English in all locales. The spelling of words such as the day of the week, or the name of the month, and the timezone will be locale-correct. 

Semantics of conversion characters in format string: 

Conversion	  Description            
String 
%A		Replaced by national representation of the full weekday name. 	E.g. 'Thursday' 
%a 		Replaced by national representation of the abbreviated weekday. 	E.g. 'Thu' 
%B 		Replaced by national representation of the full month name. 	E.g. 'September' 
%b 		Replaced by national representation of the abbreviated month. 	E.g. 'Sep' 
%C 		Year, divided by 100 (i.e. the century). 	E.g. '20' 
%c 		Replaced by national representation of time and date. 	E.g. 'Thu Sep 25 22:32:00 EDT 2008' 
%D 		Equivalent to %m/%d/%y 	E.g. '09/25/08' 
%d 		Replaced by the day of the month as a decimal number (01-31). 	E.g. '25' 
%e 		Replaced by the day of the month as a decimal number (1-31). 	E.g. '25' 
%F 		Equivalent to %Y-%m-%d. 	E.g. '2008-09-25' 
%G 		Replaced by a year as a decimal number with century. This year is the one that contains the greater part of the week (Monday as the first day of the week). 	E.g. '2008' 
%g 		Replaced by the same year as in %G, but as a decimal number without century (00-99). 	E.g. '08' 
%H 		Replaced by the hour (24-hour clock) as a decimal number (00-23). 	E.g. '22' 
%h 		Equivalent to %b. 	E.g. 'Sep' 
%I 		Replaced by the hour (12-hour clock) as a decimal number (01-12). 	E.g. '10' 
%j 		Replaced by the day of the year as a decimal number (001-366). 	E.g. '269' 
%k 		Replaced by the hour (24-hour clock) as a decimal number (0-23); single digits are preceded by a blank. 	E.g. '22' (or ' 1' for 1 in the morning). 
%l 		Replaced by the hour (12-hour clock) as a decimal number (1-12); single digits are preceded by a blank. 	E.g. '10' 
%M 		Replaced by the minute as a decimal number (00-59). 	E.g. '32' 
%m 		Replaced by the month as a decimal number (01-12). 	E.g. '09' 
%n 		Replaced by a newline character. 	  
%P 		Replaced by AM or PM. 	E.g. 'PM' 
%p 		Same as %P but lower-case. 	E.g. 'pm' 
%R 		Equivalent to %H:%M. 	E.g. '22:32' 
%r 		Equivalent to %I:%M:%S %p 	E.g. '10:32:00 pm' 
%S 		Replaced by the second as a decimal number (00-60). 	E.g. '00' 
%s 		Replaced by the number of seconds since the Epoch, UTC (see mktime(3)). 	E.g. '1222396320' 
%T 		Equivalent to %H:%M:%S 	E.g. '22:32:00' 
%t 		Replaced by a tab character. 	  
%U 		Replaced by the week number of the year (Sunday as the first day of the week) as a decimal number (00-53). 	E.g. '38' 
%u 		Replaced by the weekday (Monday as the first day of the week) as a decimal number (1-7). 	E.g. '4' 
%V 		Replaced by the week number of the year (Monday as the first day of the week) as a decimal number (01-53). If the week containing January 1 has four or more days in the new year, then it is week 1; otherwise it is the last week of the previous year, and the next week is week 1. 	E.g. '39' 
%v 		Equivalent to %e-%b-%Y 	E.g. '25-Sep-2008' 
%W 		Replaced by the week number of the year (Monday as the first day of the week) as a decimal number (00-53). 	E.g. '38' 
%w 		Replaced by the weekday (Sunday as the first day of the week) as a decimal number (0-6). 	E.g. '4' 
%X 		Replaced by national representation of the time. 	E.g. '22:32:00' 
%x 		Replaced by national representation of the date. 	E.g. '09/25/08' 
%Y 		Replaced by the year with century as a decimal number. 	E.g. '2008' 
%y 		Replaced by the year without century as a decimal number. 	E.g. '08' 
%Z 		Replaced by the time zone name. 	E.g. 'EDT' 
%z 		Replaced by the time zone offset from UTC; a leading plus sign stands for east of UTC, a minus sign or west of UTC, hours and minutes follow with two digits each and no delimiter between them (common form for RFC 822 date headers). 	E.g. '-0500' 
%+ 		Replaced by national representation of the date and time (the format is similar to that produced by date(1)). 	E.g. 'Thu Sep 25 22:32:00 EDT 2008' 
%1 		Milliseconds, rounded to three places, with leading zeros. Note: this is a departure from the strftime format, which does not represent milliseconds. 	E.g. '000' 
%% 		Replaced by '%' 	% 


Example: 

%{The time is: %X.%1%nThe date is: %x}t 

yields the following output: 

   The time is: 22:32:00.000
   The date is: 09/25/08