AWS CloudTrail

Scanner supports AWS CloudTrail logs, which is cloud audit log data that describes activity across your AWS accounts. This is one of the most valuable data sources for running detections in AWS. In order for Scanner to see these logs, you can configure CloudTrail to publish them to S3.

Step 1: Create a CloudTrail trail

When you create a new CloudTrail trail, it will stream logs to an S3 bucket of your choice. You can follow the AWS documentation to perform this action. See: Creating a trail for your AWS account.

Step 2: Link the trail S3 bucket to Scanner

If you haven't done so already, link the S3 bucket containing your CloudTrail logs to Scanner using the Linking AWS Accounts guide.

Step 3: Set up an S3 Import Rule in Scanner

1. Within Scanner, navigate to Settings > S3 Import Rules.

2. Click Create Rule.

3. For Rule name, type a name like my_team_name_aws_cloudtrail_logs.

4. For Destination Index, choose the index where you want these logs to be searchable in Scanner.

5. For Status, set to Active if you want to start indexing the data immediately.

6. For Source Type, we recommend aws:cloudtrail, but you are free to choose any name. However, out-of-the-box detection rules will expect aws:cloudtrail.

7. For AWS Account, choose the account that contains the S3 bucket containing CloudTrail logs.

8. For S3 Bucket, choose the S3 bucket containing CloudTrail logs.

9. For S3 Key Prefix, type the prefix (i.e. directory path) where the CloudTrail trail is writing logs. This is usually just AWSLogs/. We will use Additional Regex to refine the selection further.

10. Click + Additional Regex, and type: .*/CloudTrail/.*\.json\.gz

    1. This will ensure that we only index files gzipped JSON files in the directory /CloudTrail/, and skip any files in directories like /CloudTrail-Digest/ or any others.

11. For File type, choose CloudTrailJson with Gzip compression.

12. For Timestamp extractors, under Column name, type eventTime. This is the field in each log event that contains the timestamp information.

13. Click Preview rule to try it out. Check that the S3 keys you expect are appearing, and check that the log events inside are being parsed properly with the timestamp detected properly.

14. When you're ready, click Create.