> For the complete documentation index, see [llms.txt](https://docs.scanner.dev/scanner/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/sources/streams/confluent-cloud.md).

# Confluent Cloud

Scanner can ingest data from Confluent Cloud topics using Confluent's fully-managed [Amazon S3 Sink connector](https://docs.confluent.io/cloud/current/connectors/cc-s3-sink/cc-s3-sink.html). The connector runs inside Confluent Cloud, so there is no Kafka Connect infrastructure for you to operate, and writes topic data to an S3 collect buffer bucket that Scanner is linked to.

## Step 1: Set up the collect buffer bucket

Follow the [Streams overview](/scanner/using-scanner-complete-feature-reference/data-ingestion/sources/streams.md) to create an S3 collect buffer bucket with a 7-day lifecycle policy and link it to Scanner.

## Step 2: Create an Amazon S3 Sink connector in Confluent Cloud

In the Confluent Cloud console, add the **Amazon S3 Sink** connector to your cluster and select the topics to deliver. Recommended settings:

* **Output message format: JSON**. This writes one JSON record per line (newline-delimited JSON).
* **Compression: gzip.**
* **Time interval: HOURLY** (or DAILY). This groups output files into time-partitioned S3 prefixes.
* **Flush size**: the default is fine; files are also rotated on the time interval, so low-volume topics still flush regularly.
* **AWS credentials**: use a Confluent **provider integration** (IAM assume-role) if available, since it avoids storing long-lived AWS access keys in Confluent. Otherwise, create an IAM user with write access to the collect buffer bucket and supply its access key.

Note that the fully-managed connector is a paid Confluent Cloud feature, billed per task-hour plus per GB transferred.

## Step 3: Ingest via Scanner Collect

Follow the [Create an Index Rule](/scanner/using-scanner-complete-feature-reference/data-ingestion/create-an-index-rule.md) guide to ingest the collect buffer bucket via Scanner Collect, using File Type `JsonLines` and Compression `Gzip`.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.scanner.dev/scanner/using-scanner-complete-feature-reference/data-ingestion/sources/streams/confluent-cloud.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.