Cloudflare

Cloudflare logs provide visibility into your edge security and network traffic. DNS logs reveal DNS query patterns and potential malicious domains, while HTTP logs capture detailed request metadata including headers, status codes, and security events. These logs are useful for detecting threats, investigating security incidents, and understanding your organization's internet activity.

Cloudflare Logpush allows you to push your Cloudflare logs directly to Amazon S3. This guide walks through how to set up Cloudflare as a log source in Scanner Collect, so that logs can be ingested from S3, normalized, and indexed for search and detection.

Scanner Collect focuses on two Cloudflare zone-scoped datasets: DNS and HTTP. For all other Cloudflare log types, including account-scoped datasets, teams are welcome to ingest them via a Custom: AWS S3 integration.

Prerequisites

Before setting up Cloudflare in Scanner, you must:

1. Have a Cloudflare Enterprise subscription - Logpush is only available for Enterprise customers

2. Set up S3 bucket and Logpush in Cloudflare - Follow the Cloudflare Logpush setup guide to configure your S3 bucket and enable Logpush

3. Configure Logpush format - When setting up Logpush, ensure that files are written as newline delimited JSON and gzipped

Once Cloudflare is pushing logs to your S3 bucket, you can proceed with configuring the source in Scanner Collect.

Step 1: Create a New Source

Navigate to the Collect tab in the Scanner UI.

• Click Create New Source.

• Click Select a Source Type.

• In the Cloudflare category, select either Cloudflare: DNS or Cloudflare: HTTP depending on which logs you want to ingest.

You'll see that:

• Ingest Method is set to AWS S3

• Destination is set to Scanner

Click Next.

Step 2: Configure the Source

• Set a Display Name, such as my-org-cloudflare.

• Leave File Type as JsonLines.

• Leave Compression as Gzip.

Click Next.

Step 3: Set the Origin (S3 Bucket)

• Select the S3 bucket where your Cloudflare Logpush logs are stored.

• (Optional) Enter a Bucket Prefix if logs are stored under a specific key path.

• No additional File Regex configuration is needed.

Click Next.

Step 4: Set the Destination

• Choose the Scanner index where Cloudflare logs should be stored for search and detection.

• Leave the Source Label set to cloudflare.

Click Next.

Step 5: Transform and Enrich

• Keep the default enrichment settings:

  • Parse JSON Columns (automatically parses stringified JSON if present)

• (Optional) Add additional transformation or enrichment steps if desired.

Click Next.

Step 6: Timestamp Extraction

The Timestamp Field will be automatically set to the correct default:

• For Cloudflare: DNS logs, this will be Timestamp

• For Cloudflare: HTTP logs, this will be EdgeStartTimestamp

Click Next.

Step 7: Review and Create

• Review your configuration.

• (Optional) Use the preview feature to confirm how Scanner will match S3 keys and parse your log files.

When everything looks correct, click Create Source.

Once created, Scanner will begin monitoring your S3 bucket for new Cloudflare logs, index them into your selected destination, and make them available for search and detection.