Getting Started

Here is how to get started with Scanner for Splunk, allowing you to index your logs in S3 at low cost and search them at high speed directly from within Splunk.

1. Ensure that you have logs in an S3 bucket

Using tools like Vector.dev, Cribl, or other log pipeline tools, you can store your logs in S3 instead of sending them directly to Splunk.

Many tools, like Crowdstrike Falcon Data Replicator and the Github Audit system can write logs directly to your S3 buckets.

Once you have logs in your S3 buckets, you can start to index them with Scanner. We support JSON, CSV, Parquet, and plaintext log files. No need to transform them first. Just point Scanner at your raw log files.

2. Configure Scanner to index these logs in your S3 bucket

Following the S3 integration guide, configure Scanner to index these logs in S3. This allows search queries to execute at high speed even as data volumes reach hundreds of terabytes or petabytes.

3. Install the "Scanner for Splunk" app into your Splunk instance.

Option 1: Install directly from within Splunk

1. In your Splunk instance, navigate to Apps > Manage apps.

2. Click Browse more apps.

3. Search for Scanner for Splunk.

4. Click on the Install button.

Option 2: Download from Splunkbase and upload the file to your Splunk instance

1. Visit the Scanner for Splunk page in Splunkbase, and download the app. It will be downloaded to your desktop as a .tgz file.

2. In you Splunk instance, navigate to Apps > Manage apps.

3. Click Install app from file.

4. Choose the .tgz file you downloaded earlier.

4. Configure the "Scanner for Splunk" app

After you install the Scanner for Splunk app, you will be prompted to set up the app for the first time. If you proceed, you will be directed to the configuration page.

If you need to view the configuration page again later, simply click on Scanner for Splunk in the app sidebar. Also, you can navigate to Apps > Manage apps, find the Scanner for Splunk row, and click the Set up link there.

Set the API URL and assign API Keys to roles

Within the Scanner for Splunk configuration page in Splunk, add the API URL of your Scanner instance, and add API keys. You can find your API URL and API keys by visiting Scanner and navigating to Settings > API.

Each API key is assigned to a Splunk role. Within Scanner, you can configure the RBAC permissions of API keys, giving them varying permissions to indexes and detection rules in Scanner.

When a user runs a Scanner command, it will run with the combined permissions of all of the API keys of the user's roles.

API keys are stored in Splunk's storage passwords feature, as required by Splunk Cloud. As a result, each Splunk role must have the list_storage_passwords capability. Otherwise, the role will not be able to view the API keys stored in Splunk's storage passwords feature.

4. Execute scanner and scannertable search commands from within Splunk

Start executing search queries against your high-volume logs in S3 by using the scanner and scannertable custom search commands. These commands are available system-wide.

The commands take a parameter q, which must be a query written in Scanner's query language.

The query is executed against Scanner's ad hoc queries API. By default, the API returns the most recent 1000 results in descending timestamp order.

Follow along for more information about running Scanner search commands in Splunk.