> For the complete documentation index, see [llms.txt](https://docs.scanner.dev/scanner/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/backfills.md).

# Backfills

You can backfill historical data using an existing [Index Rule](https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/create-an-index-rule.md).

## Permissions

Creating a backfill requires the **Backfill any Index Rule** permission, found in the **Collect** group of role settings.

## Start a Backfill

In the Scanner UI:

* Navigate to the Collect tab.
* Click the '+' icon in the upper right and select 'Backfill historical data'.

You can also start a backfill from an Index Rule's "more" menu.

Each backfill has two phases:

1. **Backfill scan** — Lists the files that match and estimates their total size. Nothing is indexed yet.
2. **Backfill run** — Ingests the files found by a completed scan.

{% hint style="warning" %}
Backfilled data counts toward your indexing usage. Scans and backfill runs cannot be canceled once started.
{% endhint %}

To create a backfill:

1. **Select Index Rule** — Choose the Index Rule to backfill.
2. **Filters** — (Optional) Enter a **pattern** to narrow which files are backfilled. Leave it empty to backfill every file the Index Rule matches. Use the file browser to see which files will be backfilled. See [Backfill Pattern Syntax](#backfill-pattern-syntax) below.
3. **Scan Bucket** — Review the filters and the Index Rule, then click **Begin scan**. You can review the Files and Est. size on the **Jobs** page as the scan progresses.
4. **Review the Scan** — When the scan completes, open it on the **Jobs** page. You can review the Files and Est. size.
5. **Run the Backfill** — Click **Run backfill** on the completed scan and type the confirmation text to start it. Progress appears on the **Jobs** page.

Note: It may take time for logs to appear in your index after backfill run reaches **Ingest enqueued**, especially for large backfills.

## Backfill Pattern Syntax

You can append a **pattern** to your Index Rule's prefix to narrow which files will be backfilled.

### Examples

These examples use an Index Rule with key prefix `logs/` and keys like `logs/us-east-1/2026/08/01/file.json.gz`.

| Pattern                 | Backfills                                  |
| ----------------------- | ------------------------------------------ |
| *(empty)*               | Every file the Index Rule matches          |
| `us-east-1/202`         | `us-east-1`, 2020 through 2029             |
| `us-east-1/2026`        | `us-east-1`, all of 2026                   |
| `us-*/2026`             | All `us-` regions, all of 2026             |
| `*/2026/08/*/*.json.gz` | All regions, August 2026, `.json.gz` files |

### Notes

* `*` can be used as a wildcard, and can match a part of a path segment or a whole path segment.
* Use `\` to escape characters. `**`, `?`, `[`, `]`, `{`, `}` are reserved.
* `/` can't be escaped.

## Related

* [Create an Index Rule](https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/create-an-index-rule.md)
* [Index Organization](https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/index-organization.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/backfills.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
