Audit Log Concepts on Confluent Cloud¶

The primary value of audit logs is that they provide data you can use to assess security risks in your Confluent Cloud clusters. They contain all of the information necessary to follow a user’s interaction with your Confluent Cloud clusters, and provide a way to:

Track user and application access
Identify abnormal behavior and anomalies
Proactively monitor and resolve security risks

Confluent Cloud audit logs provide a way to capture, protect, and preserve Kafka authentication actions, authorization actions, and organization operations into topics in Standard, Enterprise, and Dedicated clusters. Specifically, audit logs record the runtime decisions of the permission checks that occur as users and service accounts connect to clusters and attempt to take actions that are protected by ACLs or RBAC. Audit logs also track organization operations to create, delete, and modify Confluent Cloud resources, such as API keys, Kafka clusters, user accounts, service accounts, single sign-on (SSO) connections, and connectors.

Each auditable event record includes information about who tried to do what, when they tried, and whether or not the system gave permission to proceed. Auditable event messages are emitted at the time of event occurrence; however, short delays or gaps during operational maintenance are possible, although rare.

Within Confluent Cloud, all audit log messages from your clusters are retained for seven days on an independent cluster. Users cannot modify, delete, nor produce messages directly to the audit log topic, and to consume the messages, users must have an API key specific to the audit log cluster.

Auditable events¶

Confluent Cloud audit logs capture event records from auditable event methods for the following event categories. For details on the auditable event methods, click the event category name.

Kafka cluster event categories¶

Event category	Event type	Resource type
Kafka cluster authentication	`io.confluent.kafka.server/authentication`	n/a
Kafka cluster authorization	`io.confluent.kafka.server/authorization`	`Topic`, `Cluster`, `Group`
RBAC	`io.confluent.kafka.server/authorization`	`Environment`, `CloudApiKey`, `SecurityMetadata`, `Billing`

Organization authorization event categories¶

Event type: io.confluent.cloud/authorization

Event category	Resource type
IP filter	`ORGANIZATION`

Organization event categories¶

Event type: io.confluent.cloud/request

Event category	Resource type
API key	`API_KEY`
MarketPlace Entitlement	`MARKETPLACE_ENTITLEMENT`
Billing	`ORGANIZATION`
Sign-in attempt	`ORGANIZATION`
Service account	`ORGANIZATION`
User account	`USER`
User Invitation	`USER_INVITATION`
Role-based access control (RBAC)	`CLOUD_CLUSTER`
Single Sign-on (SSO) connection	`SSO_CONNECTION`
Identity provider (OAuth/OIDC)	`IDENTITY_PROVIDER`
Identity pool (OAuth/OIDC)	`IDENTITY_POOL`
Environment	`ENVIRONMENT`
Schema Registry cluster	`SCHEMA_REGISTRY`
Kafka cluster	`KAFKA_CLUSTER`
ksqlDB cluster	`KSQL_CLUSTER`
Connector	`CONNECTOR`
Network	`NETWORK`
Peering connection	`PEERING`
Private link access	`PRIVATE_LINK_ACCESS`
Transit gateway attachment	`ENVIRONMENT`
Pipeline (for Stream Designer)	`PIPELINE`

Note

Users may attempt to authorize a task solely to find out if they can perform the task, and not follow through with it. In these instances, the authorization is still captured in the audit log.

`confluent-audit-log-events` topic¶

All Confluent Cloud audit log messages are captured in the audit log topic, confluent-audit-log-events.

The following example shows an authentication event that was sent when service account 306343 used the API key MAIDSRFG53RXYTKR to connect to the Kafka cluster lkc-6k8r8q:

{
    "id": "29ca0e51-fdcd-44bd-a393-43193432b614",
    "source": "crn://confluent.cloud/kafka=lkc-6k8r8q",
    "specversion": "1.0",
    "type": "io.confluent.kafka.server/authentication",
    "datacontenttype": "application/json",
    "subject": "crn://confluent.cloud/kafka=lkc-6k8r8q",
    "time": "2020-12-28T22:41:43.395Z",
    "data": {
        "serviceName": "crn://confluent.cloud/kafka=lkc-6k8r8q",
        "methodName": "kafka.Authentication",
        "resourceName": "crn://confluent.cloud/kafka=lkc-6k8r8q",
        "authenticationInfo": {
            "principal": "User:306343",
            "metadata": {
                "mechanism": "SASL_SSL/PLAIN",
                "identifier": "MAIDSRFG53RXYTKR"
            },
            "principalResourceId": "u-yw9507",
            "identity": "crn://confluent.cloud/organization=uuid-for-ourcorp/identity-provider=ourcorp-idp/identity=u-yw9507"
        },
        "result": {
            "status": "SUCCESS",
            "message": ""
        }
    }
}