Aggregations

Scanner supports some functions for basic aggregations.

Functions are invoked with the parentheses (), e.g. count(), although top-level parentheses are optional. E.g.:

# These are all OK
* | stats countdistinct(foo, bar)
* | stats(countdistinct(foo, bar))
* | countdistinct foo, bar
* | countdistinct(foo, bar)
# This is a parse error
* | stats countdistinct foo, bar

Functions are chained onto text query filter clauses with the | (vertical pipe) operator, and can be attached to any legal query:

# get how many error responses occurred for each kubernetes pod
status_code >= 400 
| stats count() by kubernetes.container_name, kubernetes.pod_name

# get how many different emails there are in the whole dataset
* | countdistinct email

# get how many AWS API calls were made to S3 or DynamoDB and returned an error
(eventSource: "s3" or eventSource: "dynamodb") and errorCode: *
| stats count() as errorCount

# get the average, median, and 90th percentile S3 request counts by IAM user
userIdentity.type: "IAMUser" and eventSource: "s3.amazonaws.com"
| stats count() as numReqs, userIdentity.arn by userIdentity.arn
| stats avg(numReqs), percentile(50, numReqs), percentile(90, numReqs)

Function arguments can support either numeric or string values. Strings in function arguments may be specified as either bare words, or quoted strings. However, if a string begins with a number, it must be quoted.

# These are all OK
* | max "num_events"
* | max num_events
* | max "3d_objects"
# This is a parse error
* | max 3d_objects

Quoted strings in function arguments may be singly- or doubly-quoted, and support the same escape sequences as in the filter query. Note that here, the * character is simply treated as a character, since function argument strings don't support wildcarding.

Reserved Keywords

The following keywords are reserved in function arguments: by, as, true, false, null. Bare words are also prohibited from beginning with the following characters: +-*/0123456789. Use quotes if you need to pass any of these values as strings to a function.