Apache Kafka

Discover Kafka topics and sample messages for detection.

Category: Streaming
Source type: KAFKA
Produces: topic
Links: Apache KafkaProduct Kafka security (SASL and TLS)Credentials Confluent REST ProxyReference

Event streams carry the same personal data as the databases they feed, often with less scrutiny — a topic is a pipe, and nobody reads what goes through a pipe.

What you need to connect

A bootstrap host and port, and one of four connection modes:

Mode	What you supply
None	Nothing — a plaintext broker, typically internal
SASL	Username, password, mechanism, and a CA certificate
Client certificate	An access key, certificate and CA bundle (mTLS)
REST Proxy	Username and password for a Kafka REST Proxy over HTTP(S)

The REST Proxy mode needs no broker-protocol access at all, which is often the only way in from outside a cluster’s network.

One asset per topic, whose content is a sample of messages from it. Scope by topic allow- and denylist, cap the topic count, and choose whether internal topics (__consumer_offsets and friends) are included.

Metadata on every asset

Asset kind · topic

Field	Type	Always present	What it is
partition_count	integer	Yes	Number of partitions
replication_factor	integer	No	Replication factor
earliest_offset	integer	No	Sum of earliest offsets across partitions
latest_offset	integer	No	Sum of latest offsets across partitions
retention_ms	integer	No	Topic retention in milliseconds
cleanup_policy	string	No	Topic cleanup policy (delete/compact)

Lineage

This source records no lineage. Nothing in the system it reads describes data moving from one place to another, so no FLOW edges are produced. Related items are still linked — see Lineage & Relationships for what those links mean and how they differ from lineage.

Worth knowing

Reading does not disturb consumers. Messages are sampled without committing offsets on your consumer groups.
A dedicated read-only principal is the right credential — it needs describe and read on the topics you want.

Configuration

Beyond the fields below, every source also has the settings shared by all of them: the sampling strategy, the detectors to run, the scan schedule, and the compute limits for its scan jobs.

Required

Without these, the source will not save.

This section depends on which authentication method you pick — one of the following applies.

No Authentication

Field	Type	Required	What it does	Default
auth_mode	"NONE"	Yes	—	—
host	string	Yes	Kafka broker host name or IP (e.g. kafka.example.com)	—
port	integer	Yes	Kafka broker port	9092

SASL (Username/Password)

Field	Type	Required	What it does	Default
auth_mode	"SASL"	Yes	—	—
host	string	Yes	Kafka broker host name or IP (e.g. kafka.example.com)	—
port	integer	Yes	Kafka broker port	9092

Client Certificate (mTLS)

Field	Type	Required	What it does	Default
auth_mode	"CLIENT_CERT"	Yes	—	—
host	string	Yes	Kafka broker host name or IP (e.g. kafka.example.com)	—
port	integer	Yes	Kafka broker port	9092

Kafka REST Proxy (Username/Password)

Field	Type	Required	What it does	Default
auth_mode	"REST"	Yes	—	—
host	string	Yes	Kafka REST proxy host name or IP (e.g. kafka-rest.example.com)	—
port	integer	Yes	Kafka REST proxy port	8082

Secrets

Stored encrypted and never shown again after you save them. See Configuration & Fields.