scanner
  • About Scanner
  • When to use it
  • Architecture
  • Getting Started
  • Playground Guide
    • Overview
    • Part 1: Search and Analysis
    • Part 2: Detection Rules
    • Wrapping Up
  • Log Data Sources
    • Overview
    • List
      • AWS
        • AWS Aurora
        • AWS CloudTrail
        • AWS CloudWatch
        • AWS ECS
        • AWS EKS
        • AWS GuardDuty
        • AWS Lambda
        • AWS Route53 Resolver
        • AWS VPC Flow
        • AWS VPC Transit Gateway Flow
        • AWS WAF
      • Cloudflare
        • Audit Logs
        • Firewall Events
        • HTTP Requests
        • Other Datasets
      • Crowdstrike
      • Custom via Fluentd
      • Fastly
      • GitHub
      • Jamf
      • Lacework
      • Osquery
      • OSSEC
      • Sophos
      • Sublime Security
      • Suricata
      • Syslog
      • Teleport
      • Windows Defender
      • Windows Sysmon
      • Zeek
  • Indexing Your Logs in S3
    • Linking AWS Accounts
      • Manual setup
        • AWS CloudShell
      • Infra-as-code
        • AWS CloudFormation
        • Terraform
        • Pulumi
    • Creating S3 Import Rules
      • Configuration - Basic
      • Configuration - Transformations
      • Previewing Imports
      • Regular Expressions in Import Rules
  • Using Scanner
    • Query Syntax
    • Aggregation Functions
      • avg()
      • count()
      • countdistinct()
      • eval()
      • groupbycount()
      • max()
      • min()
      • percentile()
      • rename()
      • stats()
      • sum()
      • table()
      • var()
      • where()
    • Detection Rules
      • Event Sinks
      • Out-of-the-Box Detection Rules
      • MITRE Tags
    • API
      • Ad hoc queries
      • Detection Rules
      • Event Sinks
      • Validating YAML files
    • Built-in Indexes
      • _audit
    • Role-Based Access Control (RBAC)
    • Beta features
      • Scanner for Splunk
        • Getting Started
        • Using Scanner Search Commands
        • Dashboards
        • Creating Custom Content in Splunk Security Essentials
      • Scanner for Grafana
        • Getting Started
      • Jupyter Notebooks
        • Getting Started with Jupyter Notebooks
        • Scanner Notebooks on Github
      • Detection Rules as Code
        • Getting Started
        • Writing Detection Rules
        • CLI
        • Managing Synced Detection Rules
      • Detection Alert Formatting
        • Customizing PagerDuty Alerts
      • Scalar Functions and Operators
        • coalesce()
        • if()
        • arr.join()
        • math.abs()
        • math.round()
        • str.uriencode()
  • Single Sign On (SSO)
    • Overview
    • Okta
      • Okta Workforce
      • SAML
  • Self-Hosted Scanner
    • Overview
Powered by GitBook
On this page
  • Usage
  • Validate detection rule
  • Example
  • Run detection rule tests
  • Example
  • Failing tests

Was this helpful?

  1. Using Scanner
  2. Beta features
  3. Detection Rules as Code

CLI

Usage

Scanner provides a Python CLI for validating and running tests on YAML files to aid in writing detection rules.

To install the Scanner CLI:

pip install scanner-cli

You will need to provide the API URL of your Scanner instance and an API key. Go to Settings > API Keys to find your API URL and API key.

You can either set these values as environment variables:

export SCANNER_API_URL=<Scanner API URL>
export SCANNER_API_KEY=<Scanner API key>

or provide them as arguments to the CLI:

scanner-cli <command> --api-url=<Scanner API URL> --api-key=<Scanner API key>

Validate detection rule

To validate files:

scanner-cli validate -f detections/some_detection.yaml

To validate directories:

scanner-cli validate -d detections

Only YAML files with the correct schema header will be validated. Multiple files or directories can be provided. To recursively search through directories, use the recursive flag -r.

validate checks that the specified files are valid Scanner detection rule files. After validating files, use run-teststo check that all tests pass before syncing to Scanner.

Example

Run detection rule tests

To run detection rule tests on files:

scanner-cli run-tests -f detections/some_detection.yaml

To run detection rule tests on directories:

scanner-cli run-tests -d detections

This will only run tests on YAML files with the correct schema header. Multiple files or directories can be provided. To recursively search through directories, use the recursive flag -r.

Example

Failing tests

Tests fail if:

  • The expected_detection_resultis true and the provided log events do not trigger a detection in the specified time window

  • The expected_detection_resultis false and the provided log events trigger a detection in the specified time window

Tips for debugging failed tests:

  • Check that the log events are hits for query_text

  • Check that the log events have timestamps that fall in the time window specified by now_timestamp, time_range_s , and run_frequency_s

    • If now_timestampis not provided, the timestamp of the latest log event is used. In both cases, the timestamps are rounded to the next run_frequency_s.

    • Example: if time_range_sis 600 seconds, run_frequency_sis 60 seconds, and now_timestampis 2024-07-05T10:15:00.000Z, the time window specified is 2024-07-05T10:05:00.000Z to 2024-07-05T10:15:00.000Z. Log events must have timestamps in this time window in order to trigger an alert.

PreviousWriting Detection RulesNextManaging Synced Detection Rules

Last updated 3 months ago

Was this helpful?

Validate detection rules
Run tests