Writing Detection Rules

File requirements

  • Each detection rule should have its own YAML file. The file should validate against our Schema. Detection rules will not sync if there are errors parsing or validating the YAML.

  • The file name should end with .yaml or .yml extension. Files with other extensions will be ignored.

  • The file should have # schema: https://scanner.dev/schema/scanner-detection-rule.v1.json as its first line. This is how we determine whether to a YAML file is a detection rule. YAML files without this comment will be ignored.

Schema

The schema for Scanner detection rules is at https://scanner.dev/schema/scanner-detection-rule.v1.json.

The schema contains information on the properties, including descriptions, requirements, and valid values. This can be used as a reference to write your detection rules and to validate your YAML documents.

Detection Rule

A detection rule YAML consists of:

  • name - detection rule name

  • description - description of detection rule

  • enabled - whether the detection rule is enabled

  • severity - severity of the detection rule. Scanner uses OCSF Severity ID standards for severity tags. Refer to the schema for more details.

  • query_text - the query for the detection rule

  • time_range_s - the lookback period (in seconds). Refer to the schema for more details.

  • run_frequency_s - how frequently to run the detection rule (in seconds). Refer to the schema for more details.

  • event_sink_keys - keys for event sinks to send detection events to. See Detection Rule Event Sinks, below.

  • tests - a list of test cases and expected results to apply to this detection rule. See Detection Rule Tests, below.

Example

Below is an example YAML for a detection rule. It looks for updates to AWS SecurityHub findings, which may indicate evasion of security measures.

It is configured to run every 60 seconds and look for possible threat activity in the last 5 minutes of logs. It will trigger alerts with High severity level and send them to the alert event sinks that are associated with the key high_severity_alerts.

# schema: https://scanner.dev/schema/scanner-detection-rule.v1.json
name: AWS SecurityHub Findings Evasion
enabled: true
description: |
  Detects the modification of the findings on SecurityHub.

  References:
  * https://docs.aws.amazon.com/cli/latest/reference/securityhub/

  False Positives:
  * System or Network administrator behaviors
  * DEV, UAT, SAT environment. You should apply this rule with PROD environment only.
severity: High
query_text: |
  %ingest.source_type="aws:cloudtrail"
  eventSource="securityhub.amazonaws.com"
  eventName=(
    BatchUpdateFindings
    DeleteInsight
    UpdateFindings
    UpdateInsight
  )
  | stats
    min(timestamp) as firstTime,
    max(timestamp) as lastTime,
    count() as eventCount
    by
    userIdentity.arn,
    eventSource,
    eventName,
    awsRegion
time_range_s: 300
run_frequency_s: 60
event_sink_keys:
- high_severity_alerts
tests:
  - name: Test alert is triggered when SecurityHub findings modified
    now_timestamp: "2024-08-21T00:03:00.000Z"
    dataset_inline: |
      {"timestamp":"2024-08-21T00:02:30.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"securityhub.amazonaws.com","eventName":"BatchUpdateFindings","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"awsRegion":"us-west-2"}
      {"timestamp":"2024-08-21T00:02:00.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"s3.amazonaws.com","eventName":"GetObject","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"requestParameters":{"bucketName":"mybucket","key":"exampleobject"}}
      {"timestamp":"2024-08-21T00:01:50.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"securityhub.amazonaws.com","eventName":"UpdateFindings","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"awsRegion":"us-west-2"}
      {"timestamp":"2024-08-21T00:01:30.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"securityhub.amazonaws.com","eventName":"DeleteInsight","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"awsRegion":"us-west-2"}
      {"timestamp":"2024-08-21T00:01:20.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"lambda.amazonaws.com","eventName":"InvokeFunction","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"requestParameters":{"functionName":"my-function"}}
      {"timestamp":"2024-08-21T00:01:10.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"securityhub.amazonaws.com","eventName":"UpdateInsight","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"awsRegion":"us-west-2"}
    expected_detection_result: true
  - name: Test no alert is triggered when SecurityHub findings not modified
    now_timestamp: "2024-08-21T00:03:00.000Z"
    dataset_inline: |
      {"timestamp":"2024-08-21T00:02:30.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"s3.amazonaws.com","eventName":"GetObject","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"requestParameters":{"bucketName":"mybucket","key":"exampleobject"}}
      {"timestamp":"2024-08-21T00:02:00.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"lambda.amazonaws.com","eventName":"InvokeFunction","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"requestParameters":{"functionName":"my-function"}}
      {"timestamp":"2024-08-21T00:01:50.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"dynamodb.amazonaws.com","eventName":"PutItem","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"requestParameters":{"tableName":"my-table"}}
      {"timestamp":"2024-08-21T00:01:20.000Z","%ingest.source_type":"aws:cloudtrail","eventSource":"rds.amazonaws.com","eventName":"CreateDBInstance","userIdentity":{"arn":"arn:aws:iam::123456789012:user/JohnDoe"},"requestParameters":{"dBInstanceIdentifier":"mydb"}}
    expected_detection_result: false

Index Specifiers

To include an index specifier in your query text, you must use the full index specifier format, which is @index={ UUID | "alias" }. While @index=alias works on Scanner web and API, this format cannot be used for detection rules because aliases can change.

To get the full index specifier format, tab-to-complete an index specifier in the Scanner search box and copy-paste it into your detection rule file.

Below is a valid index specifier:

query_text: |
  @index={ 00000000-0000-0000-0000-000000000000 | "cloudtrail_logs" }

Below are examples of invalid index specifiers. These will result in QueryParseErrors.

query_text: |
  @index=00000000-0000-0000-0000-000000000000
query_text: |
  @index=cloudtrail_logs

Using index rather than @index will search for a column called index rather than filter a Scanner index.

Detection Rule Event Sinks

You can specify keys that identify which event sinks a detection rule should log events to. You can assign one or more actual event sinks to each key in the settings for the sync source; see Getting Started.

Event sink keys can just be specified as an array in YAML. Each event sink key must begin with an ASCII letter, and contain only ASCII alphanumeric characters, -, or _.

Example

Below is an example of a detection rule's event sink keys. This should be in the same file as your detection rule.

event_sink_keys:
    - high_severity_alerts
    - soar_response_flow

Detection Rule Tests

You can specify tests in your YAML file to run against the detection rule.

A detection rule test consists of:

  • name - test name

  • now_timestamp - timestamp to start the test (optional)

  • dataset_inline - a list of JSON log events for the test

  • expected_detection_result - whether the given dataset triggers a detection event

Detection rules will not sync if there are failing tests.

Rules will be validated and tests run on sync.

Dataset

Test datasets are a list of JSON log events. There are a few ways to write test datasets:

  1. You can export results from Scanner as JSON lines and use the results as a test dataset. To export results, run a query in Scanner and click the arrow next to Run and select JSON Lines under Export Results.

  2. You can copy-paste raw_events from Scanner search results. The copy-pasted result is not guaranteed to be valid JSON because values may not be properly escaped.

  3. You can write your own JSON log events. A valid JSON log event must contain a timestamp in RFC 3339 format. It can contain any number of key/value pairs. An example log event:

{"timestamp":"2024-07-05T10:12:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}

Example

Below is an example of a detection rule test. This should be in the same file as your detection rule.

tests:
  - name: Test unauthorized event present
    now_timestamp: 2024-07-05T11:00:00.000Z
    dataset_inline: |
      {"timestamp":"2024-07-05T10:12:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
      {"timestamp":"2024-07-05T10:12:45.123Z","eventSource":"lambda.amazonaws.com"}
      {"timestamp":"2024-07-05T10:13:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
      {"timestamp":"2024-07-05T10:14:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
      {"timestamp":"2024-07-05T10:15:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
      {"timestamp":"2024-07-05T10:16:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
      {"timestamp":"2024-07-05T10:17:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
      {"timestamp":"2024-07-05T10:18:34.123Z","errorMessage":"unauthorized","eventSource":"s3.amazonaws.com"}
    expected_detection_result: true

Offline Validation

While working on your detection rules, it can be valuable to test them as-is without needing to push to Github and sync with Scanner. We provide tools to do so:

  • A JSON schema that describes detection rules can be found at https://scanner.dev/schema/scanner-detection-rule.v1.json, which can be used to validate the YAML files directly.

  • Scanner provides a Python CLI and public API endpoints that can be used to validate the rule files, and also to run the provided tests. For more information:

PreviousGetting StartedNextCLI

Last updated