Manual Setup - AWS CloudShell

If you choose the Manual setup option, Scanner will walk you through a series of AWS CloudShell commands you can run to create resources in your AWS account. CloudShell is a browser-based terminal in the AWS Management Console. You can run CLI commands to set up your resources across different services.

You can follow the steps directly in the Scanner UI, or you can follow the AWS CloudShell guide independently on your own.

Log in to your AWS Management Console, open CloudShell and follow the instructions below to set up your resources.

1. Set shell variables

Replace the values below with the Scanner-provided values and the names of buckets in your account.

Command

# These values will be provided by Scanner
REGION="<INSERT_VALUE_HERE>"
SCANNER_AWS_ACCOUNT_ID="<INSERT_VALUE_HERE>"
STS_EXTERNAL_ID="<INSERT_VALUE_HERE>"

# Insert your AWS account ID here
YOUR_AWS_ACCOUNT_ID="<INSERT_VALUE_HERE>"

# S3 buckets used as destinations for Collect Rules (Scanner will write raw logs into these).
# Leave empty if not using Collect Rules: S3_COLLECT_RULE_DESTINATION_BUCKETS=()
S3_COLLECT_RULE_DESTINATION_BUCKETS=("<BUCKET_1>" "<BUCKET_2>")

# S3 buckets used as the source of Index Rules (raw logs will be ingested from these).
# Include the bucket in both if it is also a Collect Rule destination.
# Leave empty if not using Index Rules: S3_INDEX_RULE_SOURCE_BUCKETS=()
S3_INDEX_RULE_SOURCE_BUCKETS=("<BUCKET_1>" "<BUCKET_2>")

# KMS key ARNs for Collect Rule destination buckets (only if using customer-managed KMS keys).
# Leave empty if not using KMS keys: S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS=()
S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS=()

# KMS key ARNs for Index Rule source buckets (only if using customer-managed KMS keys).
# Leave empty if not using KMS keys: S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS=()
S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS=()

# These values are derived from values above
S3_INDEX_FILES_BUCKET_NAME=scnr-index-files-$STS_EXTERNAL_ID

# These are default names for resources to be created
IAM_SCANNER_ROLE_NAME="scnr-ScannerRole"
IAM_SCANNER_ROLE_POLICY_NAME="scnr-ScannerRolePolicy"
EVENT_RULE_EXEC_ROLE_NAME="scnr-LogsBucketsEventRuleExecRole"
EVENT_RULE_NAME="scnr-LogsBucketsObjectCreatedRule"

2. Create S3 index files bucket

This bucket is where Scanner stores index files, keeping all log data within your AWS account.

Please ensure this bucket is used exclusively for Scanner indexing. Avoid adding any unrelated files to maintain optimal performance.

Command

# Create bucket
aws s3api create-bucket \
  --region $REGION \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --acl "private" \
  --create-bucket-configuration "LocationConstraint=$REGION"

Output

Command

# Set public access block
aws s3api put-public-access-block \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

# Check public access block
aws s3api get-public-access-block --bucket $S3_INDEX_FILES_BUCKET_NAME

Output

Command

# Set bucket encryption
aws s3api put-bucket-encryption \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --server-side-encryption-configuration '{
    "Rules": [
      {
        "BucketKeyEnabled": true,
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "aws:kms"
        }
      }
    ]
   }'

# Check bucket encryption
aws s3api get-bucket-encryption --bucket $S3_INDEX_FILES_BUCKET_NAME

Output

Command

# Set lifecycle configuration
aws s3api put-bucket-lifecycle-configuration \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --lifecycle-configuration '{
    "Rules": [
      {
        "ID": "ExpireTagging",
        "Filter": {
          "Tag": {
            "Key": "Scnr-Lifecycle",
            "Value": "expire"
          }
        },
        "Status": "Enabled",
        "Expiration": {
          "Days": 1
        }
      },
      {
        "ID": "AbortIncompleteMultiPartUploads",
        "Filter": {},
        "Status": "Enabled",
        "AbortIncompleteMultipartUpload": {
          "DaysAfterInitiation": 1
        }
      }
    ]
  }'

# Check lifecycle configuration
aws s3api get-bucket-lifecycle-configuration --bucket $S3_INDEX_FILES_BUCKET_NAME

Output

3. Create EventBridge IAM role

When new log files appear in your S3 buckets, S3 sends object-created events to Amazon EventBridge. An EventBridge rule forwards those events to Scanner's event bus. This IAM role allows EventBridge to put events onto Scanner's event bus.

Command

# Create IAM role for EventBridge to send events to Scanner's event bus
aws iam create-role \
  --role-name $EVENT_RULE_EXEC_ROLE_NAME \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": { "Service": "events.amazonaws.com" },
        "Action": "sts:AssumeRole"
      }
    ]
  }'

Output

Command

# Attach inline policy to allow PutEvents to Scanner's event bus
SCANNER_EVENT_BUS_ARN="arn:aws:events:${REGION}:${SCANNER_AWS_ACCOUNT_ID}:event-bus/scnr-IndexingEventBus"

aws iam put-role-policy \
  --role-name $EVENT_RULE_EXEC_ROLE_NAME \
  --policy-name "PutEventsToScannerEventBus" \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": "events:PutEvents",
        "Resource": "'${SCANNER_EVENT_BUS_ARN}'"
      }
    ]
  }'

# Check policy
aws iam get-role-policy \
  --role-name $EVENT_RULE_EXEC_ROLE_NAME \
  --policy-name "PutEventsToScannerEventBus"

Output

4. Create EventBridge rule

This rule matches S3 object-created events only from your Index Rule source buckets (S3_INDEX_RULE_SOURCE_BUCKETS) and forwards them to Scanner's event bus.

EventBridge rules only receive events from S3 buckets in the same region as the rule. If your Index Rule source buckets are in multiple regions, create this role and rule in each region (re-run the commands in sections 3 and 4 with the appropriate region).

Command

# Get the role ARN for the rule target
EVENT_RULE_EXEC_ROLE_ARN=$(aws iam get-role --role-name $EVENT_RULE_EXEC_ROLE_NAME --query "Role.Arn" --output text)
SCANNER_EVENT_BUS_ARN="arn:aws:events:${REGION}:${SCANNER_AWS_ACCOUNT_ID}:event-bus/scnr-IndexingEventBus"

# Build event pattern filtered to Index Rule source buckets only
BUCKET_NAMES_JSON=$(printf '"%s",' "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}" | sed 's/,$//')
EVENT_PATTERN='{"source":["aws.s3"],"detail-type":["Object Created"],"detail":{"bucket":{"name":['$BUCKET_NAMES_JSON']}}}'

# Create EventBridge rule
aws events put-rule \
  --region $REGION \
  --name $EVENT_RULE_NAME \
  --event-pattern "$EVENT_PATTERN" \
  --state ENABLED

# Add target (Scanner's event bus)
aws events put-targets \
  --region $REGION \
  --rule $EVENT_RULE_NAME \
  --targets '[{"Id":"ScannerIndexingEventBus","Arn":"'${SCANNER_EVENT_BUS_ARN}'","RoleArn":"'${EVENT_RULE_EXEC_ROLE_ARN}'"}]'

# Check rule
aws events describe-rule --region $REGION --name $EVENT_RULE_NAME
aws events list-targets-by-rule --region $REGION --rule $EVENT_RULE_NAME

Output

5. Enable EventBridge on S3 buckets

Enable Amazon EventBridge notifications on each Index Rule source bucket so that object-created events are sent to EventBridge. The rule you created will forward them to Scanner's event bus.

Enabling EventBridge on a bucket does not conflict with existing S3 event notification configurations (SNS, SQS, or Lambda). Both can operate simultaneously.

Command

# Enable EventBridge notifications for each Index Rule source bucket
for BUCKET in "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}"
do
  aws s3api put-bucket-notification-configuration \
    --bucket "$BUCKET" \
    --notification-configuration '{"EventBridgeConfiguration": {}}'
done

# Check notification configuration (for each bucket)
for BUCKET in "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}"
do
  aws s3api get-bucket-notification-configuration --bucket "$BUCKET"
done

Output

6. Create IAM Scanner Role

Scanner will assume this IAM role to perform actions in your AWS account, such as reading and writing log files.

Command

# Create role
aws iam create-role \
  --role-name $IAM_SCANNER_ROLE_NAME \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::'${SCANNER_AWS_ACCOUNT_ID}':root"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
          "StringEquals": {
            "sts:ExternalId": "'${STS_EXTERNAL_ID}'"
          }
        }
      }
    ]
  }'

Output

Command

# Build the policy document dynamically based on which buckets are configured.
# The policy includes:
# - Basic S3 permissions for all buckets
# - Scanner will write the processed index files to the Index Files Bucket.
#   Delete permission is needed as Scanner compacts and expires files in this
#   bucket periodically.
POLICY_STATEMENTS='[
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetBucketLocation",
      "s3:GetBucketTagging",
      "s3:ListAllMyBuckets"
    ],
    "Resource": "*"
  },
  {
    "Effect": "Allow",
    "Action": [
      "s3:DeleteObject",
      "s3:DeleteObjectTagging",
      "s3:DeleteObjectVersion",
      "s3:DeleteObjectVersionTagging",
      "s3:GetEncryptionConfiguration",
      "s3:GetLifecycleConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket",
      "s3:PutObject",
      "s3:PutObjectTagging"
    ],
    "Resource": [
      "arn:aws:s3:::'$S3_INDEX_FILES_BUCKET_NAME'",
      "arn:aws:s3:::'$S3_INDEX_FILES_BUCKET_NAME'/*"
    ]
  }'

# As destinations for Collect Rules, Scanner will write raw log files into
# these buckets.
if [ ${#S3_COLLECT_RULE_DESTINATION_BUCKETS[@]} -gt 0 ]; then
  COLLECT_DEST_ARNS=$(printf "%s\n" "${S3_COLLECT_RULE_DESTINATION_BUCKETS[@]}" | awk '{print "\"arn:aws:s3:::" $0 "\",\"arn:aws:s3:::" $0 "/*\""}' | paste -sd, -)
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetBucketNotification",
      "s3:GetEncryptionConfiguration",
      "s3:GetLifecycleConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket",
      "s3:PutObject",
      "s3:PutObjectTagging"
    ],
    "Resource": ['$COLLECT_DEST_ARNS']
  }'

  # Scanner will test the integration by writing/deleting test files into/from
  # Collect Rule destination buckets.
  COLLECT_DEST_TEST_ARNS=$(printf "%s\n" "${S3_COLLECT_RULE_DESTINATION_BUCKETS[@]}" | awk '{print "\"arn:aws:s3:::" $0 "/*ScannerTestFile\""}' | paste -sd, -)
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "s3:DeleteObject",
      "s3:DeleteObjectTagging",
      "s3:DeleteObjectVersion",
      "s3:DeleteObjectVersionTagging"
    ],
    "Resource": ['$COLLECT_DEST_TEST_ARNS']
  }'
fi

# As sources for Index Rules, Scanner will read the raw log files from these
# buckets.
if [ ${#S3_INDEX_RULE_SOURCE_BUCKETS[@]} -gt 0 ]; then
  INDEX_SOURCE_ARNS=$(printf "%s\n" "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}" | awk '{print "\"arn:aws:s3:::" $0 "\",\"arn:aws:s3:::" $0 "/*\""}' | paste -sd, -)
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetBucketNotification",
      "s3:GetEncryptionConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket"
    ],
    "Resource": ['$INDEX_SOURCE_ARNS']
  }'
fi

# If a Collect Rule destination bucket is encrypted by a Customer Managed KMS
# key, the Scanner Role will need permission to encrypt with the key when
# writing raw log files into the bucket.
if [ ${#S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS[@]} -gt 0 ]; then
  COLLECT_KMS_ARNS=$(printf "\"%s\"," "${S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS[@]}" | sed 's/,$//')
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "kms:DescribeKey",
      "kms:GenerateDataKey"
    ],
    "Resource": ['$COLLECT_KMS_ARNS']
  }'
fi

# If an Index Rule source bucket is encrypted by a Customer Managed KMS key,
# the Scanner Role will need permission to decrypt with the key when
# reading/ingesting the raw log files from the bucket.
if [ ${#S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS[@]} -gt 0 ]; then
  INDEX_KMS_ARNS=$(printf "\"%s\"," "${S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS[@]}" | sed 's/,$//')
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "kms:Decrypt",
      "kms:DescribeKey"
    ],
    "Resource": ['$INDEX_KMS_ARNS']
  }'
fi

POLICY_STATEMENTS+=']'

# Create policy
aws iam put-role-policy \
  --role-name $IAM_SCANNER_ROLE_NAME \
  --policy-name $IAM_SCANNER_ROLE_POLICY_NAME \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": '$POLICY_STATEMENTS'
  }'

# Check policy
aws iam get-role-policy \
  --role-name $IAM_SCANNER_ROLE_NAME \
  --policy-name $IAM_SCANNER_ROLE_POLICY_NAME

Output

Adding more S3 buckets

If you need to add more S3 buckets from an existing account after the initial setup, you need to do the following:

For Index Rule Source Buckets

If you need to add more S3 log files buckets from an existing account after the initial setup, you need to do the following:

• Enable EventBridge on each new bucket (same as in step 5: put-bucket-notification-configuration with EventBridgeConfiguration: {}).

• Update the EventBridge rule so it forwards events from the new bucket: re-run the put-rule command from step 4 with S3_INDEX_RULE_SOURCE_BUCKETS updated to include the new bucket name(s). The rule's event pattern filters by bucket name, so new buckets must be added to the pattern.

• Update the Scanner IAM role policy:

  1. Go to AWS Console -> IAM -> <Scanner Role>

  2. Permissions -> <Scanner Role Policy> -> Edit

  3. For each bucket, add two rows under the Resource array containing the log files buckets: one for bucket_arn and one for bucket_arn/*.

For Collect Rule Destination Buckets

• Update the Scanner IAM role policy:

  1. Go to AWS Console -> IAM -> <Scanner Role>

  2. Permissions -> <Scanner Role Policy> -> Edit

  3. In the Collect Rule Destination Buckets statement, add two rows under Resource: one for arn:aws:s3:::bucket_name and one for arn:aws:s3:::bucket_name/*.

  4. In the Collect Rule Destination Buckets (test files) statement, add one row under Resource: arn:aws:s3:::bucket_name/*ScannerTestFile.