Audit Log Concepts¶

Audit logs provide a way to capture, protect, and preserve authorization activity into topics in Kafka clusters on Confluent Platform using Confluent Server Authorizer. Specifically, audit logs record the runtime decisions of the permission checks that occur as users attempt to take actions that are protected by ACLs and RBAC. Auditable events are recorded in chronological order, although it is possible for a consumer of audit log messages to fetch them out of order.

Each auditable event includes information about who tried to do what, when they tried, and whether or not the system gave permission to proceed.

By default, audit logs are enabled and are managed by the interbroker principal (typically, the user kafka), who has expansive permissions.

The primary value of audit logs is that they provide data you can use to assess security risks in your local and remote Kafka clusters. They contain all of the information necessary to follow a user’s interaction with your local or remote Kafka clusters, and provide a way to:

Track user and application access across the platform
Identify abnormal behavior and anomalies
Proactively monitor and resolve security risks

You can use Splunk, S3, and other sink connectors to move your audit log data to a target platform for analysis.

You can learn more about using audit logs with your Kafka deployment by watching the Audit Logs video in the Apache Kafka Security course.

Prerequisites¶

Before configuring and using audit logs, you must configure the Confluent Server Authorizer:

authorizer.class.name=io.confluent.kafka.security.authorizer.ConfluentServerAuthorizer

Audit log content¶

The following example shows the contents of an audit log event message that is returned from a cluster when a user creates a new topic:

{
    "id": "889bdcd9-a378-4bfe-8860-180ef8efd208",
    "source": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q",
    "specversion": "1.0",
    "type": "io.confluent.kafka.server/authorization",
    "time": "2019-10-24T16:15:48.355Z",
    "datacontenttype": "application/json",
    "subject": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q/topic=app3-topic",
    "confluentRouting": {
        "route": "confluent-audit-log-events"
    },
    "data": {
      "serviceName": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q",
      "methodName": "kafka.CreateTopics",
      "resourceName": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q/topic=app3-topic",
      "authenticationInfo": {
          "principal": "User:admin"
      },
      "authorizationInfo": {
          "granted": true,
          "operation": "Create",
          "resourceType": "Topic",
          "resourceName": "app3-topic",
          "patternType": "LITERAL",
          "superUserAuthorization": true
      },
      "request": {
          "correlation_id": "3",
          "client_id": "adminclient-6"
      },
      "requestMetadata": {
          "client_address": "/127.0.0.1"
      }
    }
}

The first section of the audit log event message displays information about the log message itself. Think of it as the envelope in which the message is delivered; it contains the following audit log implementation details:

Audit log entry	Description
id	A randomly-generated UUID that ensures uniqueness across all sources.
source	Identifies the Kafka cluster that made the decision about whether or not the authorization was allowed. Always includes the CRN, and is always the same as the `serviceName`.
specversion	The version of CloudEvents in use (CloudEvents is a vendor-neutral specification that defines the format of event data).
type	The type of event that occurred (authorization request).
time	The time at which the event occurred.
datacontenttype	Identifies the CloudEvent format that the audit data is presented in (JSON).
subject	Identifies the resource on which access has been allowed or denied.
confluentRouting	Identifies the topic to which the event is sent. Topic routing is configurable. Refer to Configure audit logs for details.

The next section of the audit log displays data (data) you can use to assess security risks in your local and remote Kafka clusters.

Audit log entry	Description
serviceName	Identifies the Kafka cluster that made the decision about whether or not the authorization was allowed. Always includes the CRN, and is always the same as `source`.
methodName	The type of request that was made. For example, the methodName above shows that the user attempted to create a Kafka topic. This entry will include one of the event names listed above, and display either `mds` (if it’s an Authorize event) or `kafka` (if any other event) before it. For example: `mds.Authorize`, `kafka.Metadata`, `kafka.CreateTopics`.
resourceName	The resource identifier (CRN) of the target of the request. For example, the `resourceName` above indicates the user attempted to create a topic in a uniquely-identified Kafka cluster.
authenticationInfo	Identifies the user, service principal account, or principal attempting the request.
authorizationInfo	Includes details about the decision that was made to authorize the request. The example above shows that the authorization request to create a topic in a Kafka cluster was granted. If RBAC is used, the resource and role (if any) are included. If ACLs are used, the ACL information is also included.
requestMetadata	Identifies how the authorization request was sent and shows the requestor’s IP address, if known.

For details about how to view the contents of an audit log, refer to View audit logs on the fly.

Audit log content examples¶

The following example shows what an audit log message looks like when RBAC is enabled:

{
    "id": "889bdcd9-a378-4bfe-8860-180ef8efd208",
    "source": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q",
    "specversion": "1.0",
    "type": "io.confluent.kafka.server/authorization",
    "time": "2019-10-24T16:15:48.355Z",
    "datacontenttype": "application/json",
    "subject": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q/topic=app3-topic",
    "confluentRouting": {
        "route": "confluent-audit-log-events"
    },
    "data": {
        "serviceName": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q",
        "methodName": "kafka.CreateTopics",
        "resourceName": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q/topic=app3-topic",
        "authenticationInfo": {
            "principal": "User:resourceOwner1"
        },
        "authorizationInfo": {
            "granted": true,
            "operation": "Create",
            "resourceType": "Topic",
            "resourceName": "app3-topic",
            "patternType": "LITERAL",
            "rbacAuthorization": {
                "role": "ResourceOwner",
                "scope": {
                    "outerScope": [],
                    "clusters": {
                        "kafka-cluster": "j94C72q3Qpym0MJ9McEufQ"
                    }
                }
            }
        }
    }
}

The following example shows what an audit log message looks like when you are using an ACL:

{
    "id": "889bdcd9-a378-4bfe-8860-180ef8efd208",
    "source": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q",
    "specversion": "1.0",
    "type": "io.confluent.kafka.server/authorization",
    "time": "2019-10-24T16:15:48.355Z",
    "datacontenttype": "application/json",
    "subject": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q/topic=app3-topic",
    "confluentRouting": {
        "route": "confluent-audit-log-events"
    },
    "data": {
        "serviceName": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q",
        "methodName": "kafka.CreateTopics",
        "resourceName": "crn:///kafka=8caBa-0_Tu-2k3rKSxY64Q/topic=app3-topic",
        "authenticationInfo": {
            "principal": "User:alice"
        },
        "authorizationInfo": {
            "granted": true,
            "operation": "DescribeConfigs",
            "resourceType": "Topic",
            "resourceName": "app3-topic",
            "patternType": "LITERAL",
            "aclAuthorization" : {
              "permissionType":"ALLOW",
              "host":"*"
            }
        },
    }
}

Confluent Resource Name (CRN)¶

CRNs provide a uniform way to uniquely identify resources referenced throughout Confluent Platform, and are used in audit logs. Each CRN is a uniform resource identifier (URI) that uniquely identifies the associated Confluent resource cluster and is prefixed with crn://.

You can configure the authority (also known as host name) of a CRN at the cluster level. This setting is empty by default, even if using the Confluent Metadata Service (MDS). Unless you specify an authority name using the confluent.authorizer.authority.name= option in server.properties, it will remain empty. If you have a single cluster or context, this approach could be sufficient.

However, consider the scenario where you have multiple production clusters that all refer to a single MDS for RBAC, and you also have a test group of clusters with its own MDS. In your production environment you are using the same MDS for RBAC, so on every cluster here you specify the authority as confluent.authorizer.authority.name=prod.mds.mycompany.com. In your test environment you specify the authority as confluent.authorizer.authority.name=test.mds.mycompany.com. The production cluster includes a topic named “travel” and your test cluster also has a topic named “travel” (the data in each is different). Because you configured an authority for each cluster, you can quickly and easily identify which cluster each “travel” topic resides in. Any time identical names are used in both your production and test clusters, the distinct authority provides a way to distinguish between the two.

We recommend setting the authority as the DNS name associated with your installation, which is an easy way to make CRNs more meaningful. So all of the brokers in any logical group (for example, your company’s staging environment) should specify the same authority.name (for example, confluent.authorizer.authority.name=mds.mycompany.com) in server.properties. For a logical group, such as clusters managed by a single MDS, you should specify the hostname used to reach the MDS. Do not specify different authority.name values for individual clusters unless you are deliberately keeping them separate (for example, a separate production cluster, separate staging cluster, or separate test cluster).

CRN formats vary by resource:

Kafka cluster: <authority>/kafka=<kafka-cluster-id>
Topic: <authority>/kafka=<kafka-cluster-id>/topic=<topic-name>
Consumer group: <authority>/kafka=<kafka-cluster-id>/group=<consumer-group-name>
ksqlDB cluster: <authority>/kafka=<kafka-cluster-id>/ksql=<ksql-cluster>
Connect: <authority>/kafka=<kafka-cluster-id>/connect=<connect-cluster>/connector=<connector-name>

Here are examples of CRNs used to associate audit logs with specific Confluent resources:

Kafka cluster: crn://mds.mycompany.com/kafka=rKtuRNiDQb2k9NMml6rLfA
Topic: crn://mds.mycompany.com/kafka=rKtuRNiDQb2k9NMml6rLfA/topic=app1-topic
Consumer group: crn://mds.mycompany.com/kafka=rKtuRNiDQb2k9NMml6rLfA/group=app1-consumer-group
KQSL cluster: crn://mds.mycompany.com/kafka=rKtuRNiDQb2k9NMml6rLfA/ksql=default_
Connect: crn://mds.mycompany.com/kafka=rKtuRNiDQb2k9NMml6rLfA/connect=ydfk/connector=sink-connector

Default audit log configuration¶

Audit logs are enabled by default, and are managed by the interbroker principal (typically, the user kafka), who has expansive permissions.

If you do not configure anything, then audit logs are sent to a single topic in the local cluster that captures “allowed” and “denied” authorization events. This topic is named confluent-audit-log-events. Default audit logs capture Management and Authorize categories of authorization events only.

The following example shows the default audit log configuration:

{
    "destinations": {
        "topics": {
            "confluent-audit-log-events": {
                "retention_ms": 7776000000
            }
        }
    },
    "default_topics": {
        "allowed": "confluent-audit-log-events",
        "denied": "confluent-audit-log-events"
    }
}

For details about how to disable audit logging refer to Disabling audit logs.

Audit log configuration considerations¶

Most organizations have strict and specific security governance requirements and policies. In such cases, you can configure audit logs on a more granular level. Specifically, you can configure:

Which event categories you want to capture (including categories like produce, consume, and interbroker, which are disabled by default)
Multiple topics to capture logs of differing importance
Which cluster to deliver audit logs to
Topic destination routes optimized for security and performance
Retention periods that serve the needs of your organization
Excluded principals, which ensures performance is not compromised by excessively high message volumes
The Kafka port over which to communicate with your audit log cluster

When enabling audit logging for produce and consume, be very selective about which events you want logged, and configure logging for only the most sensitive topics.

Also consider the following configuration recommendations for securing your audit logs:

For improved security, send audit logs to a different cluster. Limit permissions on the audit log cluster to ensure the audit log is trustworthy.
Create an audit log principal with limited permissions (for example, a user identified as confluent-audit). The default is to use the broker principal, which has expansive permissions.

Audit log destination cluster¶

Consider delivering audit log messages to a specific cluster set aside for the sole purpose of retaining them. This ensures that no one can access or tamper with your organization’s audit logs, and enables you to selectively conduct more in-depth auditing of sensitive data, while keeping log volumes down for less sensitive data.

If you deliver audit logs to another cluster, you must configure the connection to that cluster.

Audit log destination topics¶

Carefully consider the your topic settings when planning your audit logging configuration. You can flexibly route audit log messages to different topics based on subject, category, and whether authorizations were allowed or denied. You can also specify routes for specific subjects, or use prefixes (for example, _secure-* for sensitive authorization events) and wildcards to cover larger groups of resources.

Audit log replication factor and retention¶

You can specify the replication factor and retention period for your audit logs. The retention period specified in the audit log configuration is only enforced if the audit log feature automatically creates a topic that doesn’t already exist.

If you create a topic manually with a retention period value that conflicts with the retention specified in the router configuration, then the manually-created setting is used. The default retention period for audit log messages is 90 days (7776000000 ms).

In cases where the Kafka broker creates the audit log topic automatically, use the confluent.security.event.logger.exporter.kafka.topic.replicas option to specify the replication factor:

confluent.security.event.logger.exporter.kafka.topic.replicas=2

For details about automatic configuration of replication factors in Kafka brokers, refer to Configure replication factors.

Excluded principals¶

You can control the volume of audit log messages with excluded principals. If produce and consume audit logging is turned on, reads and writes conducted by principals generate additional audit logs, which means each log read and every log write results in yet another audit log message being created. This can quickly grow out of control and adversely impact cluster performance.

To keep audit logs from accumulating into an unmanageable volume you should always include the following excluded principals in your configuration:

A trusted principal responsible for writing to audit log topics. Make audit log writes the only task this principal performs. Do not grant additional permissions to this principal, and do not allow other principals to write to audit log topics.
A trusted principal responsible for reading from audit log topics. Make audit log reads the only task this principal performs. Do not grant additional permissions to this principal, and do not allow other principals to read from audit log topics.

Another approach to avoid unwanted audit log accumulation and the need to configure excluded principals altogether, is to direct audit logs to a secured cluster with limited access, and don’t enable audit logs on this cluster.