Use this documentation with care! It describes
the heavily outdated version 5, which was actively
developed around 2010 and is considered dead by the
rsyslog team for many years now.
This documentation reflects the latest update of the
previously existing (now removed) v5-stable branch.
It describes the 5.10.2 version, which was never
released. As such, it contains some content that
does not apply to any released version.
To obtain the doc that properly matches your installed
v5 version, obtain the doc set from your distro. Each
version of rsyslog contained the version that exactly
matches it.
As general advise, it is strongly suggested to
upgrade to the current version supported by the rsyslog
project. The current version can always be found on
the right-hand side info box on the
rsyslog web site.
Note that there is no rsyslog community support available
for this heavily outdated version. If you need to stick
with it, please ask your distribution for support.
The property replacer is a core component in rsyslogd’s string template system. A syslog message has a number of well-defined properties. Each of this properties can be accessed and manipulated by the property replacer. With it, it is easy to use only part of a property value or manipulate the value, e.g. by converting all characters to lower case.
Syslog message properties are used inside templates. They are accessed by putting them between percent signs. Properties can be modified by the property replacer. The full syntax is as follows:
%property:fromChar:toChar:options%
The property replacer can use all rsyslog properties.
FromChar and toChar are used to build substrings. They specify the offset within the string that should be copied. Offset counting starts at 1, so if you need to obtain the first 2 characters of the message text, you can use this syntax: “%msg:1:2%”. If you do not whish to specify from and to, but you want to specify options, you still need to include the colons. For example, if you would like to convert the full message text to lower case, use “%msg:::lowercase%”. If you would like to extract from a position until the end of the string, you can place a dollar-sign (“$”) in toChar (e.g. %msg:10:$%, which will extract from position 10 to the end of the string).
There is also support for regular expressions. To use them, you need to place a “R” into FromChar. This tells rsyslog that a regular expression instead of position-based extraction is desired. The actual regular expression must then be provided in toChar. The regular expression must be followed by the string “–end”. It denotes the end of the regular expression and will not become part of it. If you are using regular expressions, the property replacer will return the part of the property text that matches the regular expression. An example for a property replacer sequence with a regular expression is: “%msg:R:.*Sev:. \(.*\) \[.*–end%”
It is possible to specify some parametes after the “R”. These are comma-separated. They are:
R,<regexp-type>,<submatch>,<nomatch>,<match-number>
regexp-type is either “BRE” for Posix basic regular expressions or “ERE” for extended ones. The string must be given in upper case. The default is “BRE” to be consistent with earlier versions of rsyslog that did not support ERE. The submatch identifies the submatch to be used with the result. A single digit is supported. Match 0 is the full match, while 1 to 9 are the acutal submatches. The match-number identifies which match to use, if the expression occurs more than once inside the string. Please note that the first match is number 0, the second 1 and so on. Up to 10 matches (up to number 9) are supported. Please note that it would be more natural to have the match-number in front of submatch, but this would break backward-compatibility. So the match-number must be specified after “nomatch”.
nomatch specifies what should be used in case no match is found.
The following is a sample of an ERE expression that takes the first submatch from the message string and replaces the expression with the full field if no match is found:
%msg:R,ERE,1,FIELD:for (vlan[0-9]\*):--end%
and this takes the first submatch of the second match of said expression:
%msg:R,ERE,1,FIELD,1:for (vlan[0-9]\*):--end%
Please note: there is also a rsyslog regular expression checker/generator online tool available. With that tool, you can check your regular expressions and also generate a valid property replacer sequence. Usage of this tool is recommended. Depending on the version offered, the tool may not cover all subleties that can be done with the property replacer. It concentrates on the most often used cases. So it is still useful to hand-craft expressions for demanding environments.
Also, extraction can be done based on so-called “fields”. To do so, place a “F” into FromChar. A field in its current definition is anything that is delimited by a delimiter character. The delimiter by default is TAB (US-ASCII value 9). However, if can be changed to any other US-ASCII character by specifying a comma and the decimal US-ASCII value of the delimiter immediately after the “F”. For example, to use comma (”,”) as a delimiter, use this field specifier: “F,44”. If your syslog data is delimited, this is a quicker way to extract than via regular expressions (actually, a much quicker way). Field counting starts at 1. Field zero is accepted, but will always lead to a “field not found” error. The same happens if a field number higher than the number of fields in the property is requested. The field number must be placed in the “ToChar” parameter. An example where the 3rd field (delimited by TAB) from the msg property is extracted is as follows: “%msg:F:3%”. The same example with semicolon as delimiter is “%msg:F,59:3%”.
The use of fields does not permit to select substrings, what is rather unfortunate. To solve this issue, starting with 6.3.9, fromPos and toPos can be specified for strings as well. However, the syntax is quite ugly, but it was the only way to integrate this functonality into the already-existing system. To do so, use ”,fromPos” and ”,toPos” during field extraction. Let’s assume you want to extract the substring from position 5 to 9 in the previous example. Then, the syntax is as follows: “%msg:F,59,5:3,9%”. As you can see, “F,59” means field-mode, with semicolon delimiter and ”,5” means starting at position 5. Then “3,9” means field 3 and string extraction to position 9.
Please note that the special characters “F” and “R” are case-sensitive. Only upper case works, lower case will return an error. There are no white spaces permitted inside the sequence (that will lead to error messages and will NOT provide the intended result).
Each occurence of the field delimiter starts a new field. However, if you add a plus sign (“+”) after the field delimiter, multiple delimiters, one immediately after the others, are treated as separate fields. This can be useful in cases where the syslog message contains such sequences. A frequent case may be with code that is written as follows:
int n, m;
...
syslog(LOG_ERR, "%d test %6d", n, m);
This will result into things like this in syslog messages: “1 test 2”, “1 test 23”, “1 test 234567”
As you can see, the fields are delimited by space characters, but their exact number is unknown. They can properly be extracted as follows:
"%msg:F,32:2%" to "%msg:F,32+:2%".
This feature was suggested by Zhuang Yuyao and implemented by him. It is modeled after perl compatible regular expressions.
Property options are case-insensitive. Currently, the following options are defined:
similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands that a space is written for single-digit days. With this option, a zero is written instead. This format seems to be used by syslog-ng and the date-rfc3164-buggyday option can be used in migration scenarios where otherwise lots of scripts would need to be adjusted. It is recommended not to use this option when forwarding to remote hosts - they may treat the date as invalid (especially when parsing strictly according to RFC 3164).
This feature was introduced in rsyslog 4.6.2 and v4 versions above and 5.5.3 and all versions above.
escape-cc
replace control characters (ASCII value 127 and values less then 32) with an escape sequence. The sequnce is “#<charval>” where charval is the 3-digit decimal value of the control character. For example, a tabulator would be replaced by “#009”. Note: using this option requires that $EscapeControlCharactersOnReceive is set to off.
space-cc
replace control characters by spaces Note: using this option requires that $EscapeControlCharactersOnReceive is set to off.
drop-cc
drop control characters - the resulting string will neither contain control characters, escape sequences nor any other replacement character like space. Note: using this option requires that $EscapeControlCharactersOnReceive is set to off.
To use multiple options, simply place them one after each other with a comma delmimiting them. For example “escape-cc,sp-if-no-1st-sp”. If you use conflicting options together, the last one will override the previous one. For example, using “escape-cc,drop-cc” will use drop-cc and “drop-cc,escape-cc” will use escape-cc mode.
This documentation is part of the rsyslog project. Copyright © 2008-2014 by Rainer Gerhards and Adiscon. Released under the GNU GPL version 2 or higher.