AWS CloudFormation

Getting started with CloudFormation

You can use this CloudFormation file to set up the IAM role, IAM policies, S3 bucket, and SNS topic to integrate with Scanner.

You can get the ScannerAWSAccountID and ScannerExternalID from your Scanner support engineer or from the Scanner UI.

The CloudFormation file is available here for copy-paste, or you can download it from this link:

cloudformation/2024-07/scanner.yml

---
AWSTemplateFormatVersion: "2010-09-09"
Description: Scanner S3 Indexing Integration
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Scanner Auth Parameters
        Parameters:
          - ScannerExternalID
    ParameterLabels:
      ScannerExternalID:
        default: Scanner External ID
      ScannerAWSAccountID:
        default: Scanner AWS Account ID
      S3BucketToIndex:
        default: S3 Bucket to Index
Parameters:
  ScannerExternalID:
    Description: Scanner provides an External ID to use here.
    Type: String
  ScannerAWSAccountID:
    Description: Scanner provides its AWS Account ID to use here.
    Type: String
  S3BucketToIndex:
    Description: Enter the name of the S3 bucket that you would like Scanner to index.
    Type: String
Resources:
  ScannerRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              AWS:
                Ref: ScannerAWSAccountID
            Action: sts:AssumeRole
            Condition:
              StringEquals:
                sts:ExternalId:
                  Ref: ScannerExternalID
  ScannerIndexFilesBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: "Retain"
    Properties:
      BucketName: !Sub
        - "scanner-index-files-${Suffix}"
        - Suffix: !Ref ScannerExternalID
      AccessControl: Private
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - BucketKeyEnabled: true
            ServerSideEncryptionByDefault:
              SSEAlgorithm: "aws:kms"
      LifecycleConfiguration:
        Rules:
          - Id: ExpireTagging
            Status: Enabled
            TagFilters:
              - Key: "Scnr-Lifecycle"
                Value: "expire"
            ExpirationInDays: 1
          - Id: AbortIncompleteMultiPartUploads
            Status: Enabled
            AbortIncompleteMultipartUpload:
              DaysAfterInitiation: 1
  LogsBucketEventNotificationTopic:
    Type: AWS::SNS::Topic
  LogsBucketEventNotificationTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: sns:Publish
            Resource: "*"
      Topics:
        - !Ref LogsBucketEventNotificationTopic
  LogsBucketEventNotificationTopicSubscription:
    Type: AWS::SNS::Subscription
    Properties:
      TopicArn: !Ref LogsBucketEventNotificationTopic
      Endpoint: !Sub
        - "arn:aws:sqs:${InstanceRegion}:${ScannerAWSAccountID}:scnr-S3ObjectCreatedNotificationsQueue"
        - InstanceRegion: !Ref "AWS::Region"
          ScannerAWSAccountID: !Ref ScannerAWSAccountID
      Protocol: "sqs"
      RawMessageDelivery: true
  ScannerPolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: ScannerPolicy
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - s3:ListAllMyBuckets
              - s3:GetBucketLocation
              - s3:GetBucketTagging
            Resource: "*"
          - Effect: Allow
            Action:
              - s3:GetBucketNotification
              - s3:ListBucket
              - s3:GetObject
              - s3:GetObjectTagging
            Resource:
              - Fn::Join:
                  - ""
                  - - "arn:aws:s3:::"
                    - Ref: S3BucketToIndex
              - Fn::Join:
                  - ""
                  - - "arn:aws:s3:::"
                    - Ref: S3BucketToIndex
                    - "/*"
          - Effect: Allow
            Action:
              - s3:GetLifecycleConfiguration
              - s3:ListBucket
              - s3:GetObject
              - s3:GetObjectTagging
              - s3:PutObject
              - s3:PutObjectTagging
              - s3:DeleteObject
              - s3:DeleteObjectTagging
              - s3:DeleteObjectVersion
              - s3:DeleteObjectVersionTagging
            Resource:
              - Fn::Join:
                  - ""
                  - - "arn:aws:s3:::scanner-index-files-"
                    - Ref: ScannerExternalID
              - Fn::Join:
                  - ""
                  - - "arn:aws:s3:::scanner-index-files-"
                    - Ref: ScannerExternalID
                    - "/*"
      Roles:
        - Ref: ScannerRole
Outputs:
  ScannerRoleARN:
    Description: The ARN of the new Scanner IAM Role
    Value:
      Fn::GetAtt:
        - ScannerRole
        - Arn
  ScannerIndexFilesBucketName:
    Description: The name of the new S3 bucket that will store Scanner Index Files
    Value: !Ref ScannerIndexFilesBucket

Permissions needed to launch the CloudFormation template

To launch (and rollback) the template successfully, your IAM role will need the following IAM permissions.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "sid1",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:DeleteBucket",
                "s3:PutEncryptionConfiguration",
                "s3:PutLifecycleConfiguration",
                "s3:PutBucketPublicAccessBlock"
            ],
            "Resource": "arn:aws:s3:::scanner-index-files-*"
        },
        {
            "Sid": "sid2",
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:GetRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:PutRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DetachRolePolicy",
            ],
            "Resource": [
                "arn:aws:iam::*:role/*-ScannerRole-*"
            ]
        },
        {
            "Sid": "sid3",
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateChangeSet",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DescribeStacks",
                "cloudformation:DescribeStackEvents",
                "cloudformation:ListStacks",
                "cloudformation:CreateStack",
                "cloudformation:DeleteStack",
                "cloudformation:DescribeChangeSet",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:GetTemplateSummary"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "sid4",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketNotification",
                "s3:PutBucketNotification"
            ],
            "Resource": [
                "arn:aws:s3:::<s3_bucket_you_want_to_index>"
            ]
        }
    ]
}

If your S3 buckets are in multiple regions

If the S3 buckets that you want to index are in multiple regions, edit the CloudFormation file to do the following:

• Create one AWS::SNS::Topic per region.

• Create an AWS::SNS::Subscription for each SNS topic, all pointing to the same SQS queue in your Scanner instance.

Setting up bucket notifications

To allow Scanner to index log files continuously, you will need to configure your S3 buckets to send "object created" notifications to an SNS topic created by CloudFormation earlier, which will forward the notifications to an SQS queue running in your Scanner instance. Setting up bucket notifications cannot be done directly in CloudFormation without the CloudFormation stack taking full ownership of the S3 bucket, which we don't necessarily want.

Here is how to set up bucket event notifications manually using the AWS console.

Navigate to S3 > (Click on your bucket) > Properties.

Scroll down to Event notifications.

Click Create event notification

Give it an Event name. Optionally provide a Prefix and Suffix to filter down to a specific set of files.

Select the checkbox next to All object create events - s3:ObjectCreated:*.

Scroll down to Destination. Select SNS topic. Under Specify SNS topic, select Choose from your SNS topics. Select the SNS topic created by Scanner's CloudFormation template earlier:

scnr-LogsBucketEventNotificationTopic

This SNS topic is already configured with a subscription to push bucket event notifications to Scanner's SQS queue.

Click Save changes.

Object creation notifications should now be sent to your Scanner instance.

What if I already have a conflicting destination for bucket event notifications?

You might be sending bucket notifications to a different destination already, like to the SQS queue of another vendor, to a different SNS topic, or to a Lambda function.

To resolve this situation, we recommend the following configuration:

• Configure the bucket to push event notifications to an SNS topic.

• Configure the SNS topic to relay messages to multiple destinations by creating subscriptions:

  • Create a topic subscription for Scanner

    • Protocol: SQS

    • Endpoint: ARN of your Scanner instance's SQS queue. Your contacts on the Scanner team can give this to you.

    • Enable raw message delivery.

  • Create topic subscriptions for your other vendors and destinations.