# Manual Setup - AWS CloudShell

If you choose the **Manual setup** option, Scanner will walk you through a series of **AWS CloudShell** commands you can run to create resources in your AWS account. CloudShell is a browser-based terminal in the AWS Management Console. You can run CLI commands to set up your resources across different services.

You can follow the steps directly in the Scanner UI, or you can follow the **AWS CloudShell** guide independently on your own.

Log in to your AWS Management Console, [open CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html#how-to-get-started) and follow the instructions below to set up your resources.

### 1. Set shell variables

Replace the values below with the Scanner-provided values and the names of buckets in your account.

{% tabs %}
{% tab title="Command" %}

```bash
# These values will be provided by Scanner
REGION="<INSERT_VALUE_HERE>"
SCANNER_AWS_ACCOUNT_ID="<INSERT_VALUE_HERE>"
STS_EXTERNAL_ID="<INSERT_VALUE_HERE>"

# Insert your AWS account ID here
YOUR_AWS_ACCOUNT_ID="<INSERT_VALUE_HERE>"

# S3 buckets used as destinations for Collect Rules (Scanner will write raw logs into these).
# Leave empty if not using Collect Rules: S3_COLLECT_RULE_DESTINATION_BUCKETS=()
S3_COLLECT_RULE_DESTINATION_BUCKETS=("<BUCKET_1>" "<BUCKET_2>")

# S3 buckets used as the source of Index Rules (raw logs will be ingested from these).
# Include the bucket in both if it is also a Collect Rule destination.
# Leave empty if not using Index Rules: S3_INDEX_RULE_SOURCE_BUCKETS=()
S3_INDEX_RULE_SOURCE_BUCKETS=("<BUCKET_1>" "<BUCKET_2>")

# KMS key ARNs for Collect Rule destination buckets (only if using customer-managed KMS keys).
# Leave empty if not using KMS keys: S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS=()
S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS=()

# KMS key ARNs for Index Rule source buckets (only if using customer-managed KMS keys).
# Leave empty if not using KMS keys: S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS=()
S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS=()

# These values are derived from values above
S3_INDEX_FILES_BUCKET_NAME=scnr-index-files-$STS_EXTERNAL_ID

# These are default names for resources to be created
IAM_SCANNER_ROLE_NAME="scnr-ScannerRole"
IAM_SCANNER_ROLE_POLICY_NAME="scnr-ScannerRolePolicy"
EVENT_RULE_EXEC_ROLE_NAME="scnr-LogsBucketsEventRuleExecRole"
EVENT_RULE_NAME="scnr-LogsBucketsObjectCreatedRule"
```

{% endtab %}
{% endtabs %}

### 2. Create S3 index files bucket

This bucket is where Scanner stores index files, keeping all log data within your AWS account.

{% hint style="info" %}
Please ensure this bucket is used exclusively for Scanner indexing. Avoid adding any unrelated files to maintain optimal performance.
{% endhint %}

{% tabs %}
{% tab title="Command" %}

```bash
# Create bucket
aws s3api create-bucket \
  --region $REGION \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --acl "private" \
  --create-bucket-configuration "LocationConstraint=$REGION"

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "Location": "<S3_INDEX_FILES_BUCKET_NAME>.s3.amazonaws.com/"
}
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Command" %}

```bash
# Set public access block
aws s3api put-public-access-block \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

# Check public access block
aws s3api get-public-access-block --bucket $S3_INDEX_FILES_BUCKET_NAME

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "PublicAccessBlockConfiguration": {
    "BlockPublicAcls": true,
    "IgnorePublicAcls": true,
    "BlockPublicPolicy": true,
    "RestrictPublicBuckets": true
  }
}
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Command" %}

```bash
# Set bucket encryption
aws s3api put-bucket-encryption \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --server-side-encryption-configuration '{
    "Rules": [
      {
        "BucketKeyEnabled": true,
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "aws:kms"
        }
      }
    ]
   }'

# Check bucket encryption
aws s3api get-bucket-encryption --bucket $S3_INDEX_FILES_BUCKET_NAME

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "ServerSideEncryptionConfiguration": {
    "Rules": [
      {
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "aws:kms"
        },
        "BucketKeyEnabled": true
      }
    ]
  }
}
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Command" %}

```bash
# Set lifecycle configuration
aws s3api put-bucket-lifecycle-configuration \
  --bucket $S3_INDEX_FILES_BUCKET_NAME \
  --lifecycle-configuration '{
    "Rules": [
      {
        "ID": "ExpireTagging",
        "Filter": {
          "Tag": {
            "Key": "Scnr-Lifecycle",
            "Value": "expire"
          }
        },
        "Status": "Enabled",
        "Expiration": {
          "Days": 1
        }
      },
      {
        "ID": "AbortIncompleteMultiPartUploads",
        "Filter": {},
        "Status": "Enabled",
        "AbortIncompleteMultipartUpload": {
          "DaysAfterInitiation": 1
        }
      }
    ]
  }'

# Check lifecycle configuration
aws s3api get-bucket-lifecycle-configuration --bucket $S3_INDEX_FILES_BUCKET_NAME

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "Rules": [
    {
      "Expiration": {
        "Days": 1
      },
      "ID": "ExpireTagging",
        "Filter": {
          "Tag": {
            "Key": "Scnr-Lifecycle",
            "Value": "expire"
          }
        },
        "Status": "Enabled"
      },
      {
        "ID": "AbortIncompleteMultiPartUploads",
        "Filter": {},
        "Status": "Enabled",
        "AbortIncompleteMultipartUpload": {
          "DaysAfterInitiation": 1
        }
      }
  ]
}
```

{% endtab %}
{% endtabs %}

### 3. Create EventBridge IAM role

When new log files appear in your S3 buckets, S3 sends object-created events to Amazon EventBridge. An EventBridge rule forwards those events to Scanner's event bus. This IAM role allows EventBridge to put events onto Scanner's event bus.

{% tabs %}
{% tab title="Command" %}

```bash
# Create IAM role for EventBridge to send events to Scanner's event bus
aws iam create-role \
  --role-name $EVENT_RULE_EXEC_ROLE_NAME \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": { "Service": "events.amazonaws.com" },
        "Action": "sts:AssumeRole"
      }
    ]
  }'

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "Role": {
    "Path": "/",
    "RoleName": "<EVENT_RULE_EXEC_ROLE_NAME>",
    "Arn": "arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/<EVENT_RULE_EXEC_ROLE_NAME>",
    ...
  }
}
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Command" %}

```bash
# Attach inline policy to allow PutEvents to Scanner's event bus
SCANNER_EVENT_BUS_ARN="arn:aws:events:${REGION}:${SCANNER_AWS_ACCOUNT_ID}:event-bus/scnr-IndexingEventBus"

aws iam put-role-policy \
  --role-name $EVENT_RULE_EXEC_ROLE_NAME \
  --policy-name "PutEventsToScannerEventBus" \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": "events:PutEvents",
        "Resource": "'${SCANNER_EVENT_BUS_ARN}'"
      }
    ]
  }'

# Check policy
aws iam get-role-policy \
  --role-name $EVENT_RULE_EXEC_ROLE_NAME \
  --policy-name "PutEventsToScannerEventBus"

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "RoleName": "<EVENT_RULE_EXEC_ROLE_NAME>",
  "PolicyName": "PutEventsToScannerEventBus",
  "PolicyDocument": { ... }
}
```

{% endtab %}
{% endtabs %}

### 4. Create EventBridge rule

This rule matches S3 object-created events only from your Index Rule source buckets (`S3_INDEX_RULE_SOURCE_BUCKETS`) and forwards them to Scanner's event bus.

{% hint style="info" %}
EventBridge rules only receive events from S3 buckets in the same region as the rule. If your Index Rule source buckets are in multiple regions, create this role and rule in each region (re-run the commands in sections 3 and 4 with the appropriate region).
{% endhint %}

{% tabs %}
{% tab title="Command" %}

```bash
# Get the role ARN for the rule target
EVENT_RULE_EXEC_ROLE_ARN=$(aws iam get-role --role-name $EVENT_RULE_EXEC_ROLE_NAME --query "Role.Arn" --output text)
SCANNER_EVENT_BUS_ARN="arn:aws:events:${REGION}:${SCANNER_AWS_ACCOUNT_ID}:event-bus/scnr-IndexingEventBus"

# Build event pattern filtered to Index Rule source buckets only
BUCKET_NAMES_JSON=$(printf '"%s",' "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}" | sed 's/,$//')
EVENT_PATTERN='{"source":["aws.s3"],"detail-type":["Object Created"],"detail":{"bucket":{"name":['$BUCKET_NAMES_JSON']}}}'

# Create EventBridge rule
aws events put-rule \
  --region $REGION \
  --name $EVENT_RULE_NAME \
  --event-pattern "$EVENT_PATTERN" \
  --state ENABLED

# Add target (Scanner's event bus)
aws events put-targets \
  --region $REGION \
  --rule $EVENT_RULE_NAME \
  --targets '[{"Id":"ScannerIndexingEventBus","Arn":"'${SCANNER_EVENT_BUS_ARN}'","RoleArn":"'${EVENT_RULE_EXEC_ROLE_ARN}'"}]'

# Check rule
aws events describe-rule --region $REGION --name $EVENT_RULE_NAME
aws events list-targets-by-rule --region $REGION --rule $EVENT_RULE_NAME

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output (describe-rule)
{
  "Name": "<EVENT_RULE_NAME>",
  "EventPattern": "{\"source\":[\"aws.s3\"],\"detail-type\":[\"Object Created\"],\"detail\":{\"bucket\":{\"name\":[\"<BUCKET_1>\",\"<BUCKET_2>\"]}}}",
  "State": "ENABLED",
  ...
}
```

{% endtab %}
{% endtabs %}

### 5. Enable EventBridge on S3 buckets

Enable Amazon EventBridge notifications on each Index Rule source bucket so that object-created events are sent to EventBridge. The rule you created will forward them to Scanner's event bus.

{% hint style="info" %}
Enabling EventBridge on a bucket does not conflict with existing S3 event notification configurations (SNS, SQS, or Lambda). Both can operate simultaneously.
{% endhint %}

{% tabs %}
{% tab title="Command" %}

```bash
# Enable EventBridge notifications for each Index Rule source bucket
for BUCKET in "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}"
do
  aws s3api put-bucket-notification-configuration \
    --bucket "$BUCKET" \
    --notification-configuration '{"EventBridgeConfiguration": {}}'
done

# Check notification configuration (for each bucket)
for BUCKET in "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}"
do
  aws s3api get-bucket-notification-configuration --bucket "$BUCKET"
done

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output (for each bucket)
{
  "EventBridgeConfiguration": {}
}
```

{% endtab %}
{% endtabs %}

### 6. Create IAM Scanner Role

Scanner will assume this IAM role to perform actions in your AWS account, such as reading and writing log files.

{% tabs %}
{% tab title="Command" %}

```bash
# Create role
aws iam create-role \
  --role-name $IAM_SCANNER_ROLE_NAME \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::'${SCANNER_AWS_ACCOUNT_ID}':root"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
          "StringEquals": {
            "sts:ExternalId": "'${STS_EXTERNAL_ID}'"
          }
        }
      }
    ]
  }'

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "Role": {
    "Path": "/",
    "RoleName": "<IAM_SCANNER_ROLE_NAME>",
    "RoleId": "<id>",
    "Arn": "arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/<IAM_SCANNER_ROLE_NAME>",
    "CreateDate": "<timestamp>",
    "AssumeRolePolicyDocument": {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::<SCANNER_AWS_ACCOUNT_ID>:root"
          },
          "Action": "sts:AssumeRole",
          "Condition": {
            "StringEquals": {
              "sts:ExternalId": "<STS_EXTERNAL_ID>"
            }
          }
        }
      ]
    }
  }
}
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Command" %}

```bash
# Build the policy document dynamically based on which buckets are configured.
# The policy includes:
# - Basic S3 permissions for all buckets
# - Scanner will write the processed index files to the Index Files Bucket.
#   Delete permission is needed as Scanner compacts and expires files in this
#   bucket periodically.
POLICY_STATEMENTS='[
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetBucketLocation",
      "s3:GetBucketTagging",
      "s3:ListAllMyBuckets"
    ],
    "Resource": "*"
  },
  {
    "Effect": "Allow",
    "Action": [
      "s3:DeleteObject",
      "s3:DeleteObjectTagging",
      "s3:DeleteObjectVersion",
      "s3:DeleteObjectVersionTagging",
      "s3:GetEncryptionConfiguration",
      "s3:GetLifecycleConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket",
      "s3:PutObject",
      "s3:PutObjectTagging"
    ],
    "Resource": [
      "arn:aws:s3:::'$S3_INDEX_FILES_BUCKET_NAME'",
      "arn:aws:s3:::'$S3_INDEX_FILES_BUCKET_NAME'/*"
    ]
  }'

# As destinations for Collect Rules, Scanner will write raw log files into
# these buckets.
if [ ${#S3_COLLECT_RULE_DESTINATION_BUCKETS[@]} -gt 0 ]; then
  COLLECT_DEST_ARNS=$(printf "%s\n" "${S3_COLLECT_RULE_DESTINATION_BUCKETS[@]}" | awk '{print "\"arn:aws:s3:::" $0 "\",\"arn:aws:s3:::" $0 "/*\""}' | paste -sd, -)
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetBucketNotification",
      "s3:GetEncryptionConfiguration",
      "s3:GetLifecycleConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket",
      "s3:PutObject",
      "s3:PutObjectTagging"
    ],
    "Resource": ['$COLLECT_DEST_ARNS']
  }'

  # Scanner will test the integration by writing/deleting test files into/from
  # Collect Rule destination buckets.
  COLLECT_DEST_TEST_ARNS=$(printf "%s\n" "${S3_COLLECT_RULE_DESTINATION_BUCKETS[@]}" | awk '{print "\"arn:aws:s3:::" $0 "/*ScannerTestFile\""}' | paste -sd, -)
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "s3:DeleteObject",
      "s3:DeleteObjectTagging",
      "s3:DeleteObjectVersion",
      "s3:DeleteObjectVersionTagging"
    ],
    "Resource": ['$COLLECT_DEST_TEST_ARNS']
  }'
fi

# As sources for Index Rules, Scanner will read the raw log files from these
# buckets.
if [ ${#S3_INDEX_RULE_SOURCE_BUCKETS[@]} -gt 0 ]; then
  INDEX_SOURCE_ARNS=$(printf "%s\n" "${S3_INDEX_RULE_SOURCE_BUCKETS[@]}" | awk '{print "\"arn:aws:s3:::" $0 "\",\"arn:aws:s3:::" $0 "/*\""}' | paste -sd, -)
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetBucketNotification",
      "s3:GetEncryptionConfiguration",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:ListBucket"
    ],
    "Resource": ['$INDEX_SOURCE_ARNS']
  }'
fi

# If a Collect Rule destination bucket is encrypted by a Customer Managed KMS
# key, the Scanner Role will need permission to encrypt with the key when
# writing raw log files into the bucket.
if [ ${#S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS[@]} -gt 0 ]; then
  COLLECT_KMS_ARNS=$(printf "\"%s\"," "${S3_COLLECT_RULE_DESTINATION_BUCKETS_KMS_KEY_ARNS[@]}" | sed 's/,$//')
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "kms:DescribeKey",
      "kms:GenerateDataKey"
    ],
    "Resource": ['$COLLECT_KMS_ARNS']
  }'
fi

# If an Index Rule source bucket is encrypted by a Customer Managed KMS key,
# the Scanner Role will need permission to decrypt with the key when
# reading/ingesting the raw log files from the bucket.
if [ ${#S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS[@]} -gt 0 ]; then
  INDEX_KMS_ARNS=$(printf "\"%s\"," "${S3_INDEX_RULE_SOURCE_BUCKETS_KMS_KEY_ARNS[@]}" | sed 's/,$//')
  POLICY_STATEMENTS+=',
  {
    "Effect": "Allow",
    "Action": [
      "kms:Decrypt",
      "kms:DescribeKey"
    ],
    "Resource": ['$INDEX_KMS_ARNS']
  }'
fi

POLICY_STATEMENTS+=']'

# Create policy
aws iam put-role-policy \
  --role-name $IAM_SCANNER_ROLE_NAME \
  --policy-name $IAM_SCANNER_ROLE_POLICY_NAME \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": '$POLICY_STATEMENTS'
  }'

# Check policy
aws iam get-role-policy \
  --role-name $IAM_SCANNER_ROLE_NAME \
  --policy-name $IAM_SCANNER_ROLE_POLICY_NAME

```

{% endtab %}

{% tab title="Output" %}

```json
# Expected output
{
  "RoleName": <IAM_SCANNER_ROLE_NAME>,
  "PolicyName": <IAM_SCANNER_ROLE_POLICY_NAME>,
  "PolicyDocument": <policy document>
}
```

{% endtab %}
{% endtabs %}

## Adding more S3 buckets

If you need to add more S3 buckets from an existing account after the initial setup, you need to do the following:

### For Index Rule Source Buckets

If you need to add more S3 log files buckets from an existing account after the initial setup, you need to do the following:

* [Enable EventBridge on each new bucket](#5-enable-eventbridge-on-s3-buckets) (same as in step 5: `put-bucket-notification-configuration` with `EventBridgeConfiguration: {}`).
* Update the EventBridge rule so it forwards events from the new bucket: re-run the `put-rule` command from [step 4](#4-create-eventbridge-rule) with `S3_INDEX_RULE_SOURCE_BUCKETS` updated to include the new bucket name(s). The rule's event pattern filters by bucket name, so new buckets must be added to the pattern.
* Update the Scanner IAM role policy:
  1. Go to AWS Console -> IAM -> *\<Scanner Role>*
  2. Permissions -> *\<Scanner Role Policy>* -> Edit
  3. For **each** bucket, add **two** rows under the `Resource` array containing the log files buckets: one for `bucket_arn` and one for `bucket_arn/*`.

### For Collect Rule Destination Buckets

* Update the Scanner IAM role policy:
  1. Go to AWS Console -> IAM -> *\<Scanner Role>*
  2. Permissions -> *\<Scanner Role Policy>* -> Edit
  3. In the Collect Rule Destination Buckets statement, add **two** rows under `Resource`: one for `arn:aws:s3:::bucket_name` and one for `arn:aws:s3:::bucket_name/*`.
  4. In the Collect Rule Destination Buckets (test files) statement, add one row under `Resource`: `arn:aws:s3:::bucket_name/*ScannerTestFile`.