Audit Logging in Kubernetes

This feature requires an enterprise license. To get a trial license key or extend your trial period, generate a new trial license key. To purchase a license, contact Redpanda Sales.

If Redpanda has enterprise features enabled and it cannot find a valid license, restrictions apply.

Many scenarios for streaming data include the need for fine-grained auditing of user activity related to the system. This is especially true for regulated industries such as finance, healthcare, and the public sector. Complying with PCI DSS v4 standards, for example, requires verbose and detailed activity auditing, alerting, and analysis capabilities.

Redpanda’s auditing capabilities support recording both administrative and operational interactions with topics and with users. Redpanda complies with the Open Cybersecurity Schema Framework (OCSF), providing a predictable and extensible solution that works seamlessly with industry standard tools.

With audit logging enabled, there should be no noticeable changes in performance other than slightly elevated CPU usage.

Audit logging is configured at the cluster level. Redpanda supports excluding specific topics or principals from auditing to help reduce noise in the log. Audit logging is disabled by default.

Audit log flow

The Redpanda audit log mechanism functions similar to the Kafka flow. When a user interacts with another user or with a topic, Redpanda writes an event to a specialized audit topic. The audit topic is immutable. Only Redpanda can write to it. Users are prevented from writing to the audit topic directly and the Kafka API cannot create or delete it.

Audit log flow

By default, any management and authentication actions performed on the cluster yield messages written to the audit log topic that are retained for seven days. Interactions with all topics by all principals are audited. Actions performed using the Kafka API and Admin API are all audited, as are actions performed directly through rpk.

Messages recorded to the audit log topic comply with the open cybersecurity schema framework. Any number of analytics frameworks, such as Splunk or Sumo Logic, can receive and process these messages. Using an open standard ensures Redpanda’s audit logs coexist with those produced by other IT assets, powering holistic monitoring and analysis of your assets.

Audit log configuration options

Redpanda’s audit logging mechanism supports several options to control the volume and availability of audit records. Configuration is applied at the cluster level.

You can configure these options directly in either the Helm values or the Redpanda resource.

auditLogging.enabled: Sets the value of the audit_enabled cluster property to enable audit logging. When you set this to true, Redpanda checks for an existing topic named _redpanda.audit_log. If none is found, Redpanda automatically creates one for you. Default: false.
auditLogging.partitions: Sets the value of the audit_log_num_partitions cluster property to define the number of partitions used by a newly created audit topic. This configuration applies only to the audit log topic and may be different from the cluster or other topic configurations. This cannot be altered for an existing audit log topic. Default: 12.
auditLogging.replicationFactor: Sets the value of the audit_log_replication_factor cluster property to define the replication factor for a newly created audit log topic. This configuration applies only to the audit log topic and may be different from the cluster or other topic configurations. This cannot be altered for existing audit log topics. If a value is not provided, Redpanda uses the internal_topic_replication_factor cluster property value. Default: null.
auditLogging.enabledEventTypes: Sets the value of the audit_enabled_event_types cluster property. This option is a list of JSON strings identifying the event types to include in the audit log. Valid values include any of the following: management, produce, consume, describe, heartbeat, authenticate, schema_registry, admin. Default: '["management","authenticate","admin"]'.
auditLogging.excludedTopics: Sets the value of the audit_excluded_topics cluster property. This option is a list of JSON strings identifying the topics the audit logging system should ignore. This list cannot include the _redpanda.audit_log topic. Redpanda rejects the command if you do attempt to include that topic. Default: null.
auditLogging.excludedPrincipals: Sets the value of the audit_excluded_principals cluster property. This option is a list of JSON strings identifying the principals the audit logging system should ignore. Principals can be listed as User:name or name, both are accepted. Default: null.
auditLogging.clientMaxBufferSize: Sets the value of the audit_client_max_buffer_size cluster property to define the number of bytes allocated by the internal audit client for audit messages. When changing this, you must disable audit logging and then re-enable it for the change to take effect. Consider increasing this if your system generates a very large number of audit records in a short amount of time. Default: 16777216.
auditLogging.queueDrainIntervalMs: Sets the value of the audit_queue_drain_interval_ms cluster property. Internally, Redpanda batches audit log messages in memory and periodically writes them to the audit log topic. This option defines the period in milliseconds between draining this queue to the audit log topic. Longer intervals may help prevent duplicate messages, especially in high throughput scenarios, but they also increase the risk of data loss during hard shutdowns where the queue is lost. Default: 500.
auditLogging.queueMaxBufferSizePerShard: Sets the value of the audit_queue_max_buffer_size_per_shard cluster property to define the maximum amount of memory in bytes used by the audit buffer in each shard. When this size is reached, requests to log additional audit messages return a non-retryable error. Default: 1048576.

Even though audited event messages are stored to a specialized immutable topic, standard topic settings still apply. For example, you can apply the same Tiered Storage, retention time, and replication settings available to normal topics. These particular options are important for controlling the amount of disk space utilized by your audit topics.

You cannot change the values of auditLogging.partitions and auditLogging.replicationFactor after enabling audit logging because these settings impact the creation of the _redpanda.audit_log topic. The Kafka API allows you to add partitions or alter the replication factor after enabling audit logging, but Redpanda prevents you from altering these two configuration values directly.

Audit logging event types

Redpanda’s auditable events fall into one of eight different event types. The APIs associated with each event type are as follows.

Audit event type	Associated APIs
management	AlterPartitionReassignments CreateACLs CreatePartitions CreateTopics DeleteAcls DeleteGroups DeleteRecords DeleteTopics IncrementalAlterconfigs OffsetDelete
produce	AddPartitionsToTxn EndTxn InitProducerId Produce
consume	AddOffsetsToTxn Fetch JoinGroup LeaveGroup ListOffset OffsetCommit SyncGroup TxOffsetCommit
describe	DescribeAcls DescribeConfigs DescribeGroups DescribeLogDirs FindCoordinator ListGroups ListPartitionReassignments Metadata OffsetForLeaderEpoch DescribeProducers DescribeTransations ListTransactions
heartbeat	Heartbeat
authenticate	All authentication events
schema_registry	All Schema Registry API calls
admin	All Admin API calls

Audit event type

Associated APIs

management

AlterPartitionReassignments
CreateACLs
CreatePartitions
CreateTopics
DeleteAcls
DeleteGroups
DeleteRecords
DeleteTopics
IncrementalAlterconfigs
OffsetDelete

produce

AddPartitionsToTxn
EndTxn
InitProducerId
Produce

consume

AddOffsetsToTxn
Fetch
JoinGroup
LeaveGroup
ListOffset
OffsetCommit
SyncGroup
TxOffsetCommit

describe

DescribeAcls
DescribeConfigs
DescribeGroups
DescribeLogDirs
FindCoordinator
ListGroups
ListPartitionReassignments
Metadata
OffsetForLeaderEpoch
DescribeProducers
DescribeTransations
ListTransactions

heartbeat

Heartbeat

authenticate

All authentication events

schema_registry

All Schema Registry API calls

admin

All Admin API calls

Logged events

The following table identifies the data logging level for each audit event entry.

The Included column captures whether the event itself is included (for example, successful and failed access attempts), or whether a piece of data is included in the event itself (for example, Source IP address).

Data Logging Level Audit Event Included? Details

Data Logging Level	Audit Event	Included?	Details
System Level	Date and time stamp for each entry	Yes	`time` field on each event
Successful and failed access attempts	Yes	The `status_id` field shows success/failure for all access attempts for which auditing is enabled
User ID	Yes	`user.name`
User connect and disconnect time	No	Connect and disconnect time may be inferred from the presence or absence of activity.
Password change	Yes	For SCRAM users managed through Redpanda core, the Admin API call associated with the password change is logged. Note that this does not cover users synced from external IdPs, such as through OIDC.
Changes of security settings	Yes	For example, ACL creation is logged (kafka `create_acls`), and cluster configuration changes are logged (Admin API events)
Successful and failed attempts to add/remove users from the system	Yes	See Password change
Failed attempts to access system data	Yes	Generally, access attempts are logged. For example, kafka `produce` and `consume` calls are audited.
Failed attempts to access critical directories and files	No	Not applicable
Date and time of system start-up and shut-down	Yes	An application lifecycle event is logged when the broker starts/stops
Use of external/peripheral devices	No	Not applicable
Application Level	Transaction date and time	Yes	`time` field on each event
User ID	Yes	`user.name`
Source IP address	Yes	`src_endpoint.ip` field
Type of transaction	Yes	`api.operation` shows the relevant Kafka API accessed or HTTP method used (Admin/Schema Registry)
Transaction data	Yes	`resources[]` for Kafka ops; `http_request` (headers, URL) for HTTP endpoints
Transaction result (success/fail) and reason of failure	Partial	Auditing occurs at request time (start of event). Authentication failures are captured with `status_id` and `status_detail` fields. However, downstream operation outcomes (whether the actual Kafka/Admin API/Schema Registry operation succeeded or failed) are not included in audit logs.
Transaction number	No	Redpanda auditing does not assign a unique number to each request
Failed attempts to access system data	Yes	Access attempts are logged through various audit event types
Failed attempts to access critical directories and files	No	Not applicable
Use of external/peripheral devices	Yes	Device access events are captured when applicable
Network Level	Source IP address (IPv4 or IPv6)	Yes	`src_endpoint.ip` field
Destination IP address (IPv4 or IPv6)	Yes	`dst_endpoint.ip` field
Protocol/Ports	Yes	`service.name` shows the service/protocol used (`kafka`/`http`). `http_request.url.scheme` shows `http`/`https` for http requests. `dst_endpoint.port` may be indicative of the protocol.
Timestamp	Yes	`time` field
Host name	Yes	`url.hostname`: HTTP endpoints (Admin API / Schema Registry) include the HTTP hostname. Not included for Kafka (only source/destination IP).
VLAN ID	No	Not applicable
MAC address	No	Not applicable

System Level

Date and time stamp for each entry

Yes

time field on each event

Successful and failed access attempts

Yes

The status_id field shows success/failure for all access attempts for which auditing is enabled

User ID

Yes

user.name

User connect and disconnect time

Connect and disconnect time may be inferred from the presence or absence of activity.

Password change

Yes

For SCRAM users managed through Redpanda core, the Admin API call associated with the password change is logged. Note that this does not cover users synced from external IdPs, such as through OIDC.

Changes of security settings

Yes

For example, ACL creation is logged (kafka create_acls), and cluster configuration changes are logged (Admin API events)

Successful and failed attempts to add/remove users from the system

Yes

See Password change

Failed attempts to access system data

Yes

Generally, access attempts are logged. For example, kafka produce and consume calls are audited.

Failed attempts to access critical directories and files

Not applicable

Date and time of system start-up and shut-down

Yes

An application lifecycle event is logged when the broker starts/stops

Use of external/peripheral devices

Not applicable

Application Level

Transaction date and time

Yes

time field on each event

User ID

Yes

user.name

Source IP address

Yes

src_endpoint.ip field

Type of transaction

Yes

api.operation shows the relevant Kafka API accessed or HTTP method used (Admin/Schema Registry)

Transaction data

Yes

resources[] for Kafka ops; http_request (headers, URL) for HTTP endpoints

Transaction result (success/fail) and reason of failure

Partial

Auditing occurs at request time (start of event). Authentication failures are captured with status_id and status_detail fields. However, downstream operation outcomes (whether the actual Kafka/Admin API/Schema Registry operation succeeded or failed) are not included in audit logs.

Transaction number

Redpanda auditing does not assign a unique number to each request

Failed attempts to access system data

Yes

Access attempts are logged through various audit event types

Failed attempts to access critical directories and files

Not applicable

Use of external/peripheral devices

Yes

Device access events are captured when applicable

Network Level

Source IP address (IPv4 or IPv6)

Yes

src_endpoint.ip field

Destination IP address (IPv4 or IPv6)

Yes

dst_endpoint.ip field

Protocol/Ports

Yes

service.name shows the service/protocol used (kafka/http). http_request.url.scheme shows http/https for http requests. dst_endpoint.port may be indicative of the protocol.

Timestamp

Yes

time field

Host name

Yes

url.hostname: HTTP endpoints (Admin API / Schema Registry) include the HTTP hostname. Not included for Kafka (only source/destination IP).

VLAN ID

Not applicable

MAC address

Not applicable

Enable audit logging

All audit log settings are applied at the cluster level.

You can configure audit log settings in the Redpanda Helm chart, using Helm values or the Redpanda resource with the Redpanda Operator.

Operator
Helm

If you want to manage the audit topic using a Topic resource:

Create the Redpanda resource with SASL enabled:

redpanda-cluster.yaml

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef: {}
  clusterSpec:
    auth:
      sasl:
        enabled: true
        secretRef: "redpanda-users"

Create a Secret to hold the password of either a user or a superuser:
```
kubectl --namespace <namespace> create secret generic audit-topic-user-password --from-literal=password=<password>
```
If you aren’t using a superuser, make sure the user is authorized to manage the _redpanda.audit_log topic.

Create the Topic resource, and make sure to set overwriteTopicName to _redpanda.audit_log:

apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
  name: audit-topic
spec:
  partitions: <partitions>
  overwriteTopicName: _redpanda.audit_log
  kafkaApiSpec:
    brokers:
      - "redpanda-0.redpanda.<namespace>.svc.cluster.local:9093"
      - "redpanda-1.redpanda.<namespace>.svc.cluster.local:9093"
      - "redpanda-2.redpanda.<namespace>.svc.cluster.local:9093"
    tls:
      caCertSecretRef:
        name: "redpanda-default-cert"
        key: "ca.crt"
    sasl:
      username: <username>
      mechanism: <mechanism>
      passwordSecretRef:
        name: audit-topic-user-password
        key: password

This example configuration contains values for the kafkaApiSpec.tls and kafkaApiSpec.brokers settings that work with the default Helm values. If you modified these settings in your deployment, make sure to replace those values.

Update the Redpanda resource to enable audit logging:

redpanda-cluster.yaml

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef: {}
  clusterSpec:
    auth:
      sasl:
        enabled: true
        secretRef: "redpanda-users"
    auditLogging:
      enabled: true

If you don’t want to use the Topic resource, you can enable audit logging and Redpanda creates the audit topic for you:

redpanda-cluster.yaml

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef: {}
  clusterSpec:
    auth:
      sasl:
        enabled: true
        secretRef: "redpanda-users"
    auditLogging:
      enabled: true

kubectl apply -f redpanda-cluster.yaml --namespace <namespace>

--values
--set

audit-logging.yaml

auth:
  sasl:
    enabled: true
    secretRef: "redpanda-users"
auditLogging:
  enabled: true

helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
--values audit-logging.yaml --reuse-values

helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
  --set auth.sasl.enabled=true \
  --set auth.sasl.secretRef="redpanda-users" \
  --set auditLogging.enabled=true

auth.sasl.enabled: To enable audit logging, you must have:
- SASL authentication enabled.
- At least one Kafka listener that uses the SASL authentication mechanism. By default, the internal Kafka listener is used (listeners.kafka). To use a different listener, see Audit log configuration options.
- At least one superuser.
For details, see Configure Authentication for Redpanda in Kubernetes.
auditLogging.enabled: Enable audit logging. When you set this to true, Redpanda checks for an existing topic named _redpanda.audit_log. If the topic is not found, Redpanda automatically creates one for you. Default: false.

Optimize costs for audit logging

When enabled, audit logging can quickly generate a very large amount of data, especially if all event types are selected. Proper configuration of audit logging is critical to avoid filling your disk or using excess Tiered Storage. The configuration options available help ensure your audit logs contain only the volume of data necessary to meet your regulatory or legal requirements.

With audit logging, the pattern of message generation may be very different from your typical sources of data. These messages reflect usage of your system as opposed to the operational data your topics typically process. As a result, your retention, replication, and Tiered Storage requirements may differ from your other topics.

A typical scenario with audit logging is to route the messages to an analytics platform like Splunk. If your retention period is too long, you may find that you are storing excessive amounts of replicated messages in both Redpanda and in your analytics suite. Identifying the right balance of retention and replication settings minimizes this duplication while retaining your data in a system that provides actionable intelligence.

Assess the retention needs for your audit logs. You may not need to keep the logs for the default seven days. This is controlled by setting retention.ms for the _redpanda.audit_log topic or by setting log_retention_ms at the cluster level.