Pattern Development Guide

SpectX pattern language allows to describe patterns using matchers. A matcher is a “mini-pattern” which matches certain type of data. For example INTEGER (or INT for short) matches integral numbers, IPADDR matches IPv4 or IPv6 addresses. There are matchers available for handling all kind of data types. Their names are related to the type and are easy to remember. Look them up in reference manual or in on-screen help (press CTRL+SPACE when you type in Pattern Developer window).

A written pattern is interpreted from left to right, ignoring extra whitespaces, line breaks and comments in between. Hence you can write the pattern describing an integral number followed by single space, ip-address and line break as one-liner:

1
INT ' ' IPADDR EOL

Or you can write the same pattern in more explanatory way:

1
2
3
4
5
6
7
/* this pattern expects an integer number and an ip-address
    separated by single space in each line */

INT       //an integer
' '       //followed by single space
IPADDR    //followed by ipv4 or ipv6 address
EOL       //line is terminated with line feed character

Matching vs Parsing

You don’t necessarily need all data elements in log record for analysis. For instance field separators, end of record markers are useful only for parsing but we don’t need them in when we run the queries. Hence all the matchers in a defined pattern must match but only a subset of them may also extract data (also called parsing).

A matcher will extract data only when it has been assigned an export name - this is an arbitrary name of your choice, which becomes the name of field you use in query statements. In following example, the pattern has in total five matchers, of which only three are extracting data:

../../_images/virtualStructure.png

Here we have chosen to ignore characters between ip-address and timestamp, and also end-of-line character.

The data which does not match to defined pattern is captured by SpectX engine in the system field _unmatched. It is hidden by default, hence you need to include it explicitly in the select statement to access unmatched data. You’ll need it when Finding Unmatched Corner Cases.