scanner
  • About Scanner
  • When to use it
  • Architecture
  • Getting Started
  • Playground Guide
    • Overview
    • Part 1: Search and Analysis
    • Part 2: Detection Rules
    • Wrapping Up
  • Log Data Sources
    • Overview
    • List
      • AWS
        • AWS Aurora
        • AWS CloudTrail
        • AWS CloudWatch
        • AWS ECS
        • AWS EKS
        • AWS GuardDuty
        • AWS Lambda
        • AWS Route53 Resolver
        • AWS VPC Flow
        • AWS VPC Transit Gateway Flow
        • AWS WAF
      • Cloudflare
        • Audit Logs
        • Firewall Events
        • HTTP Requests
        • Other Datasets
      • Crowdstrike
      • Custom via Fluentd
      • Fastly
      • GitHub
      • Jamf
      • Lacework
      • Osquery
      • OSSEC
      • Sophos
      • Sublime Security
      • Suricata
      • Syslog
      • Teleport
      • Windows Defender
      • Windows Sysmon
      • Zeek
  • Indexing Your Logs in S3
    • Linking AWS Accounts
      • Manual setup
        • AWS CloudShell
      • Infra-as-code
        • AWS CloudFormation
        • Terraform
        • Pulumi
    • Creating S3 Import Rules
      • Configuration - Basic
      • Configuration - Transformations
      • Previewing Imports
      • Regular Expressions in Import Rules
  • Using Scanner
    • Query Syntax
    • Aggregation Functions
      • avg()
      • count()
      • countdistinct()
      • eval()
      • groupbycount()
      • max()
      • min()
      • percentile()
      • rename()
      • stats()
      • sum()
      • table()
      • var()
      • where()
    • Detection Rules
      • Event Sinks
      • Out-of-the-Box Detection Rules
      • MITRE Tags
    • API
      • Ad hoc queries
      • Detection Rules
      • Event Sinks
      • Validating YAML files
    • Built-in Indexes
      • _audit
    • Role-Based Access Control (RBAC)
    • Beta features
      • Scanner for Splunk
        • Getting Started
        • Using Scanner Search Commands
        • Dashboards
        • Creating Custom Content in Splunk Security Essentials
      • Scanner for Grafana
        • Getting Started
      • Jupyter Notebooks
        • Getting Started with Jupyter Notebooks
        • Scanner Notebooks on Github
      • Detection Rules as Code
        • Getting Started
        • Writing Detection Rules
        • CLI
        • Managing Synced Detection Rules
      • Detection Alert Formatting
        • Customizing PagerDuty Alerts
      • Scalar Functions and Operators
        • coalesce()
        • if()
        • arr.join()
        • math.abs()
        • math.round()
        • str.uriencode()
  • Single Sign On (SSO)
    • Overview
    • Okta
      • Okta Workforce
      • SAML
  • Self-Hosted Scanner
    • Overview
Powered by GitBook
On this page

Was this helpful?

  1. Indexing Your Logs in S3
  2. Linking AWS Accounts
  3. Infra-as-code

Terraform

Getting started with Terraform

You can use this Terraform file to set up the IAM role, IAM policies, an SNS topic, and an S3 bucket to integrate with Scanner. You provide the value for s3_buckets_to_index, and Scanner provides the values for scanner_aws_account_id and scanner_external_id.

It will also create a new SNS topic and update your S3 bucket to send s3:ObjectCreated notifications to this topic. The SNS topic will then relay those notifications to the SQS queue in your Scanner instance.

variable "scanner_external_id" {
  description = "Scanner provides an External ID to use here."
  type        = string
}

variable "scanner_aws_account_id" {
  description = "Scanner provides its AWS Account ID to use here."
  type        = string
}

variable "s3_buckets_to_index" {
  description = "Enter the names of the S3 buckets that you would like Scanner to index."
  type        = list(string)
}

variable "s3_bucket_kms_key_arns" {
  description = "Enter the ARNs of the KMS keys used to encrypt buckets in s3_buckets_to_index. Not needed if the buckets do not use KMS keys."
  type        = list(string)
  default     = []
}

# The IAM role used by Scanner to access the S3 files
resource "aws_iam_role" "scanner_role" {
  name = "scnr-ScannerRole"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          AWS = var.scanner_aws_account_id
        }
        Action = "sts:AssumeRole"
        Condition = {
          StringEquals = {
            "sts:ExternalId" = var.scanner_external_id
          }
        }
      }
    ]
  })
}


# The S3 bucket where Scanner will store all index files
resource "aws_s3_bucket" "scanner_index_files_bucket" {
  bucket = "scanner-index-files-${var.scanner_external_id}"
  // NOTE: With this flag, Terraform will disallow the deletion of the entire
  // stack. Unfortunately Terraform does not yet provide a way to protect only
  // some resources while deleting the rest.
  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_public_access_block" "scanner_index_files_bucket_public_access_block" {
  bucket = aws_s3_bucket.scanner_index_files_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_s3_bucket_server_side_encryption_configuration" "scanner_index_files_bucket_encryption_config" {
  bucket = aws_s3_bucket.scanner_index_files_bucket.id

  rule {
    bucket_key_enabled = true
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "scanner_index_files_bucket_lifecycle_configuration" {
  bucket = aws_s3_bucket.scanner_index_files_bucket.id

  rule {
    id     = "ExpireTagging"
    status = "Enabled"
    filter {
      tag {
        key   = "Scnr-Lifecycle"
        value = "expire"
      }
    }
    expiration {
      days = 1
    }
  }

  rule {
    id     = "AbortIncompleteMultiPartUploads"
    status = "Enabled"
    abort_incomplete_multipart_upload {
      days_after_initiation = 1
    }
  }
}


# The SNS topic to notify Scanner when the log files buckets have new files
resource "aws_sns_topic" "scnr_logs_bucket_event_notification_topic" {
  name = "scnr-LogsBucketEventNotificationTopic"

  policy = data.aws_iam_policy_document.scnr_logs_bucket_event_notification_topic_policy.json
}

data "aws_iam_policy_document" "scnr_logs_bucket_event_notification_topic_policy" {
  statement {
    principals {
      type        = "Service"
      identifiers = ["s3.amazonaws.com"]
    }

    effect    = "Allow"
    actions   = ["SNS:Publish"]
    resources = ["*"]
  }
}

resource "aws_sns_topic_subscription" "scnr_logs_bucket_event_notification_topic_subscription" {
  topic_arn            = aws_sns_topic.scnr_logs_bucket_event_notification_topic.arn
  protocol             = "sqs"
  endpoint             = "arn:aws:sqs:${data.aws_region.current.name}:${var.scanner_aws_account_id}:scnr-S3ObjectCreatedNotificationsQueue"
  raw_message_delivery = true
}

# Create one notification configuration for each bucket
resource "aws_s3_bucket_notification" "logs_bucket_event_notification" {
  for_each = toset(var.s3_buckets_to_index)
  bucket = each.value

  topic {
    topic_arn = aws_sns_topic.scnr_logs_bucket_event_notification_topic.arn
    events    = ["s3:ObjectCreated:*"]
  }
}


# The IAM policy allowing the Scanner IAM role to access the S3 buckets
resource "aws_iam_role_policy" "scanner_policy" {
  name   = "ScannerPolicy"
  role   = aws_iam_role.scanner_role.name
  policy = data.aws_iam_policy_document.scanner_policy_document.json
}

data "aws_iam_policy_document" "scanner_policy_document" {
  statement {
    actions = [
      "s3:GetBucketLocation",
      "s3:GetBucketTagging",
      "s3:ListAllMyBuckets",
    ]
    resources = ["*"]
  }

  statement {
    actions = [
      "s3:GetBucketEncryption",
      "s3:GetBucketNotification",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket",
    ]
    # Generate ARNs for each bucket in the list
    resources = flatten([
      for bucket in var.s3_buckets_to_index : [
        "arn:aws:s3:::${bucket}",
        "arn:aws:s3:::${bucket}/*"
      ]
    ])
  }

  statement {
    actions = [
      "s3:DeleteObject",
      "s3:DeleteObjectTagging",
      "s3:DeleteObjectVersion",
      "s3:DeleteObjectVersionTagging",
      "s3:GetLifecycleConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket",
      "s3:PutObject",
      "s3:PutObjectTagging",
    ]
    resources = [
      "arn:aws:s3:::scanner-index-files-${var.scanner_external_id}",
      "arn:aws:s3:::scanner-index-files-${var.scanner_external_id}/*"
    ]
  }

  # This statement is only necessary if KMS keys are provided for encrypted buckets
  dynamic "statement" {
    for_each = toset(var.s3_bucket_kms_key_arns)
    content {
      actions   = [
        "kms:Decrypt",
        "kms:DescribeKey",
      ]
      resources = var.s3_bucket_kms_key_arns
    }
  }
}

output "scanner_role_arn" {
  description = "The ARN of the new Scanner IAM Role"
  value       = aws_iam_role.scanner_role.arn
}

data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

If your S3 buckets are in multiple regions

If the S3 buckets that you want to index are in multiple regions, edit this Terraform file to do the following:

  • Create one aws_sns_topic per region.

  • Create an aws_sns_topic_subscription for each SNS topic, all pointing to the same SQS queue in your Scanner instance.

  • Create an aws_s3_bucket_notification for each S3 bucket, and point it to the SNS topic that is in the same region as the S3 bucket.

PreviousAWS CloudFormationNextPulumi

Last updated 2 months ago

Was this helpful?