Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Query Syntax

Here is how you search through your log events

Log event structure

In Scanner, a log event is a collection of key-value pairs called fields. In a field, the key is always a string, and the value may be either a string or a number.

For example, a log event from application logs might look like this:

{
  "message": "INFO - Successfully added item. item_id=817343 shopping_cart_id=1842101",
  "message_kv": {
    "item_id": 817343,
    "shopping_cart_id": 1842101
  },
  "elapsed_ms": 79,
  "status_code": 200,
  "kubernetes": {
    "container_name": "shopping_cart_api",
    "pod_name": "app-3"
  },
  "@scnr": {
    "context_fields": "container_name,pod_name"
  }
}

And the resulting Scanner log event would look like this:

Text queries

Type in free-form text to search for hits. By default, search is case insensitive for ASCII characters, so these match the same lines.

By default, tokens are matched separately, so these match the same lines.

Search terms only match full tokens by default (see Token Boundaries). Wildcards (see below) can be used for subtoken matches.

Bare (unquoted) strings must begin with an alphabetical or *@%_ character, and cannot include whitespace or any of the following characters: :()"'<>=|,~{}!#`. They also can't be any reserved keywords (see Reserved Keywords). Use single-quotes ' if you need to match whitespace, reserved characters, or reserved keywords.

Use double-quotes " for exact, case-sensitive matching.

Quoted strings support escape sequences. See Escape Sequences for Strings for a comprehensive list.

Use * for wildcard searches. You can use \* to match the actual asterisk character instead.

Note: Leading wildcards like *value are slow because they cannot use the index and must scan all data directly. Prefer trailing wildcards (value*) when possible. See Wildcard Performance for details.

Use r" for a raw string. Raw strings do not support escape sequences or wildcards and perform case-sensitive matching.

To include double quote characters " in a raw string, add a # character outside each quote. Use multiple (up to 4) if you need to include # characters inside the string:

Column Queries

Use column: value to search for a column that contains value.

Like in simple text queries, the : operator only matches full tokens by default; see Token Boundaries. Wildcards * can be used for subtoken matching.

Use column = value to search for a column that is exactly value.

Use column: * or column = * if you just want to check if a column exists at all.

Like with string values, bare (unquoted) column names must begin with an alphabetical or *@%_ character, and cannot include whitespace or any of the following characters: :()"'<>=|,~{}!`. They also can't be any reserved keywords (see Reserved Keywords).

Use backticks `` to denote columns that contain spaces or other disallowed characters.

Quoted column names support escape sequences. See Escape Sequences for Strings for a comprehensive list.

Use * or ** as a wildcard in column names. * matches any character other than .[]; ** matches any character. You can use \* to match the actual asterisk character instead.

Number queries

If your log events have number fields, you can look for exact matches or inequalities.

Boolean queries

Scanner supports boolean queries using and, or, and not. These are case-insensitive.

You can use parentheses to specify order of operations.

If parentheses aren't used, then not has highest precedence, then and, then or, so these two queries are identical.

If omitted, the default operator is and; i.e. any two query terms without a boolean operator will be assumed to be using and, so the following two queries are identical.

Boolean operators can be used inside of column filters for the : and = operators, in which case the column filter distributes. Hence, these queries are identical.

Inside of a column filter, the default operator is or rather than and, so the following queries are identical.

Additional Details

Token Boundaries

Scanner breaks field values into tokens at boundaries. Tokens consist of alphanumeric characters and underscores; everything else (spaces, hyphens, dots, etc.) is a boundary.

A query match will always start and stop on a whole token. Wildcards * can span token boundaries and be used for subtoken matching.

  • al matches "Al Sharpton" but not "Walt Whitman" or "Alan Turing"

  • al* matches "Al Sharpton" and "Alan Turing" but not "Walt Whitman"

  • *al* matches all of the above

See Understanding Tokens and Query Performance for details on how tokenization affects query efficiency.

Escape Sequences for Strings

You can use escape sequences for certain characters. These work in all strings, including column name strings.

Escape sequence
Character

\"

double quote "

\'

single quote '

\`

backtick `

\*

asterisk *

\\

backslash \

\/

forward slash /

\b

backspace U+0008

\f

form feed U+000C

\n

line feed U+000A

\r

carriage return U+000D

\t

horizontal tab U+0009

\uXXXX

unicode character U+XXXX

Reserved Keywords

The following keywords are reserved in filters: and, or, not, let, from, in. Use quotes if you need to search for them as strings, and backticks if you need to use them as column names.

PreviousQuerying & AnalysisNextUnderstanding Tokens and Query Performance

Last updated

Was this helpful?