Use Cluster Linking to Manage Audit Logs on Confluent Cloud

Cluster Linking syncs Confluent Audit Logs in real time to a Dedicated or Enterprise cluster. Confluent Audit Logs provide a view into the activity in a Confluent Cloud organization. Syncing the audit logs lets you:

  • Consume the audit logs directly from your cluster so that you can reuse your existing consumer configurations and use Stream Governance with the audit logs topic.

  • Manipulate the audit logs and build stream processing applications in ksqlDB.

  • Sync the audit logs into an external data store like Amazon S3, Snowflake, or Splunk with Connect.

Setup

  • Create a new cluster or identify an existing Dedicated or Enterprise cluster to serve as the Destination (target) cluster.

  • The audit logs cluster serves as the Source cluster.

  • Use any cloud provider and networking type. This tutorial uses the public internet, and automatic encryption options.

If you need more guidance on this, see the first steps in the Quick Start for Confluent Cloud about how to create a cluster and choose networking.

Configure permissions to consume audit logs

Create an API key for the audit log service account so the destination cluster can read the audit log topic.

Log on to Confluent CLI and select your provisioning environment

confluent environment list
confluent environment use <env_id>
Now using "<env_id>" as the default (active) environment.

Set the active cluster

confluent kafka cluster use <cluster_id>
Set Kafka cluster "<cluster_id>" as the active cluster for environment "<env_id>"

Get the details of your cluster

confluent kafka cluster describe <cluster_id>
+------------------+---------------------------------------------------------+
| ID               | <cluster_id>                                            |
| Name             | abcdefg-xyz-auditlog-linking                            |
| Type             | DEDICATED                                               |
| Ingress          | 50                                                      |
| Egress           | 150                                                     |
| Storage          | Infinite                                                |
| Cloud            | aws                                                     |
| Availability     | single-zone                                             |
| Region           | eu-west-2                                               |
| Status           | UP                                                      |
| Endpoint         | SASL_SSL://pkc-nnnnn.eu-west-2.aws.confluent.cloud:9092 |
| REST Endpoint    | https://pkc-nnnnn.eu-west-2.aws.confluent.cloud:443     |
| Cluster Size     | 1                                                       |
+------------------+---------------------------------------------------------+

Get the audit log cluster, environment, and service account IDs

confluent audit-log describe

This command returns the IDs you need to access the audit logs.

+-----------------+-----------------------------+
| Cluster         | <cluster_id>                |
| Environment     | <env_id>                    |
| Service Account | <sa_id>                     |
| Topic Name      | confluent-audit-log-events  |
+-----------------+-----------------------------+

Create an API key for the audit log service account

confluent environment use <env_id>
confluent kafka cluster use <cluster_id>
confluent api-key create --service-account <service_account_id> --resource <cluster_id>

(Optional) Look up an existing API key

If you already have an API key, you can use the following command to look it up.

confluent api-key list --resource <cluster_id>

The command lists your keys. If you plan to reuse a key, you must have its secret.

Set the active API key for the audit log cluster

confluent api-key use <api_key> --resource <cluster_id>
Set API Key "<api_key>" as the active API key for "<cluster_id>".

Confirm you can consume the audit log topic

confluent kafka topic consume -b confluent-audit-log-events

Use Ctrl+C to exit the consumer.

Set up Cluster Linking

The destination cluster you created in the Setup section hosts the mirrored confluent-audit-log-events topic.

In these steps, you create a link on that cluster (the destination cluster) that connects to the source cluster (the Audit Log cluster).

  1. Make sure that you are working on the destination cluster (the Dedicated cluster).

    You can use these commands to list environments and clusters, and switch to the appropriate ones.

    confluent environment list
    
    confluent environment use <env_id>
    
    confluent kafka cluster list
    
    confluent kafka cluster use <cluster_id>
    
  2. Get the source cluster values. The source values are the audit logs cluster ID, its bootstrap endpoint, and an API key.

    1. To get the --source-bootstrap-server value, first get the audit logs environment and cluster IDs:

      confluent audit-log describe
      

      The output resembles the following:

      +-----------------+----------------------------+
      | Cluster         | <cluster_id>               |
      | Environment     | env-abc123                 |
      | Service Account | sa-12abc3                  |
      | Topic Name      | confluent-audit-log-events |
      +-----------------+----------------------------+
      
    2. Run the following commands to locate and describe the Audit Logs source cluster:

      confluent environment use <audit_logs_env_id>
      
      confluent kafka cluster list
      

      Your output should resemble:

            Id         |             Name             |   Type   | Cloud    |  Region   | Availability | Status
      -----------------+------------------------------+----------+----------+-----------+--------------+---------
        * <cluster_id> | _confluent_audit_log_cluster | DEDICATED| aws      | us-west-2 | single-zone  | UP
      
    3. Run this command to get the details for the Audit Logs source cluster:

      confluent kafka cluster describe <audit_logs_cluster_id>
      

      Your output should resemble:

      +--------------+---------------------------------------------------------+
      | Id           | <cluster_id>                                            |
      | Name         | _confluent_audit_log_cluster                            |
      | Type         | DEDICATED                                               |
      | Ingress      |                                                     100 |
      | Egress       |                                                     100 |
      | Storage      | Infinite                                                |
      | Cloud        | aws                                                     |
      | Availability | single-zone                                             |
      | Region       | us-west-2                                               |
      | Status       | UP                                                      |
      | Endpoint     | SASL_SSL://<pkc_id>.us-west-2.aws.confluent.cloud:9092  |
      | RestEndpoint | https://<pkc_id>.us-west-2.aws.confluent.cloud:443      |
      +--------------+---------------------------------------------------------+
      
    4. Look under Endpoint for the bootstrap server for your source cluster.

  3. Run the following command to create a link called my-link.

    confluent kafka link create my-link --cluster <destination_id> \
        --source-cluster <source_id> \
        --source-bootstrap-server <source_bootstrap_server> \
        --source-api-key <key> --source-api-secret <secret>
    

    Tip

    --source-cluster-id was replaced with --source-cluster in version 3 of Confluent CLI, as described in the command reference for confluent kafka link create.

Mirror the audit logs topic and schema

A mirror topic is a read-only copy of the source topic on the destination cluster.

Mirror the audit logs topic

Click the link name to view the properties of the link and click Add mirror topic. For the topic name, enter confluent-audit-log-events. The link and mirror topic appear:

Cluster link overview showing the confluent-audit-log-events mirror topic with Active status and the Add mirror topic button

Make sure that the Dedicated cluster (destination) is the active cluster, then run confluent kafka mirror create to create the mirror topic under the link:

confluent kafka mirror create confluent-audit-log-events --link my-link

To confirm that the mirror topic was created, run confluent kafka mirror list:

confluent kafka mirror list --link my-link

Import the audit logs topic schema

Import the audit logs topic schema into the Schema Registry for this environment, using the content from the Audit Log Event Schema on Confluent Cloud. Use this schema to create the subject in the Schema Registry, following the schema import steps.

Tip

Any schema whose name ends with a -value suffix is automatically associated as the value schema for the topic, using the schema name.

Updated: March, 2025

Following is the complete audit log event schema for Confluent Cloud:

{
  "$schema":"http://json-schema.org/draft-07/schema#",
  "type":"object",
  "description":"This is v1.2 of the Confluent Cloud audit log event schema. It is based on CloudEvents. Links to the latest version can be found at https://docs.confluent.io/cloud/current/monitoring/audit-logging. Confluent will make non-breaking changes to the schema without advance notice. Breaking changes will be widely communicated at least 180 days in advance, and we will continue to maintain compatibility during this time. Exceptions to this policy apply in case of critical security vulnerabilities or functional defects. Check the above web page for links to our announcement channels and details about our compatibility policy.",
  "properties":{
    "id":{
      "type":"string",
      "minLength":1,
      "description":"Uniquely identifies the event.",
      "examples":[
        "c72a3d0c-e6f3-4196-9b49-a835614452df"
      ]
    },
    "specversion":{
      "type":"string",
      "minLength":1,
      "description":"The version of the CloudEvents specification which the event uses.",
      "examples":[
        "1.0"
      ]
    },
    "source":{
      "type":"string",
      "format":"uri-reference",
      "minLength":1,
      "description":"Identifies the context in which an event happened.",
      "examples":[
        "crn://confluent.cloud",
        "crn://confluent.cloud/kafka=lkc-a1b2c",
        "crn:///kafka=lkc-a1b2c"
      ]
    },
    "subject":{
      "type":[
        "string",
        "null"
      ],
      "minLength":1,
      "description":"Identifies the resource that would be affected by the event.",
      "examples":[
        "crn:///kafka=lkc-a1b2c",
        "crn://confluent.cloud/kafka=lkc-a1b2c",
        "crn://confluent.cloud/kafka=lkc-xyz01/topic=departures",
        "crn://confluent.cloud/kafka=lkc-xyz01/group=delivery-estimator"
      ]
    },
    "type":{
      "type":"string",
      "minLength":1,
      "description":"Describes the type of event.",
      "examples":[
        "io.confluent.kafka.server/authentication",
        "io.confluent.kafka.server/authorization",
        "io.confluent.cloud/request"
      ]
    },
    "time":{
      "type":[
        "string",
        "null"
      ],
      "minLength":1,
      "description":"Timestamp of when the occurrence happened. Adheres to RFC 3339.",
      "examples":[
        "2020-12-06T13:39:03Z",
        "2020-12-06T13:39:03.123Z"
      ]
    },
    "datacontenttype":{
      "type":[
        "string",
        "null"
      ],
      "description":"Content type of the data value. Adheres to RFC 2046 format.",
      "minLength":1,
      "examples":[
        "application/json"
      ]
    },
    "dataschema":{
      "type":[
        "string",
        "null"
      ],
      "format":"uri",
      "minLength":1,
      "description":"Identifies the schema that data adheres to. Currently unused."
    }
  },
  "required":[
    "id",
    "source",
    "specversion",
    "type"
  ],
  "allOf":[
    {
      "if":{
        "required":[
          "type"
        ],
        "properties":{
          "type":{
            "const":"io.confluent.kafka.server/authentication"
          }
        }
      },
      "then":{
        "properties":{
          "data":{
            "type":"object",
            "additionalProperties": false,
            "description":"Additional details about the authentication check.",
            "properties":{
              "serviceName": {
                "type": "string",
                "description": "The resource identifier of the service (the source) that received the request being logged.",
                "examples": [
                  "crn://confluent.cloud",
                  "crn://confluent.cloud/kafka=lkc-a1b2c",
                  "crn:///kafka=lkc-a1b2c"
                ]
              },
              "resourceName":{
                "type":"string",
                "description":"The resource name of the checked authorization rule.",
                "examples":[
                  "kafka-cluster",
                  "delivery-estimator",
                  "departures"
                ]
              },
              "methodName":{
                "type":"string",
                "description":"The type of request being logged.",
                "examples":[
                  "kafka.Authentication"
                ]
              },
              "authenticationInfo":{
                "type":"object",
                "additionalProperties": false,
                "properties":{
                  "principal":{
                    "type":"string",
                    "description":"Identifies the authenticated principal.",
                    "examples":[
                      "User:12345",
                      "None:UNKNOWN_USER"
                    ]
                  },
                  "metadata":{
                    "type":"object",
                    "properties":{
                      "mechanism":{
                        "type":"string",
                        "description":"Authentication mechanism.",
                        "examples":[
                          "SASL_SSL/PLAIN",
                          "SASL_SSL/OAUTHBEARER"
                        ]
                      },
                      "identifier":{
                        "type":"string",
                        "description":"Identifies the numeric user ID or API key supplied by the requester.",
                        "examples":[
                          "MAIDSRFG53RXYTKR",
                          "12345"
                        ]
                      }
                    }
                  },
                  "principalResourceId":{
                    "type":"string",
                    "description":"Authenticated principal resourceId of the requester.",
                    "examples":[
                      "u-yw9507"
                    ]
                  },
                  "identity":{
                    "type":"string",
                    "description":"Identity of the requester in the format.",
                    "examples":[
                      "crn://confluent.cloud/organization=uuid-for-ourcorp/identity-provider=ourcorp-idp/identity=u-yw9507"
                    ]
                  }
                }
              },
              "request":{
                "type":"object",
                "properties":{
                  "remote_logical_cluster_id":{
                    "type":"string",
                    "description":"The ID of the remote logical cluster that performed the authentication.",
                    "examples":[
                      "lkc-abc"
                    ]
                  }
                }
              },
              "requestMetadata":{
                "type":"object",
                "properties":{
                  "connection_id": {
                    "type":"string",
                    "description":"Uniquely identifies an authenticated connection.Only present for successfully authenticated connections.May be used during analysis to correlate events",
                    "examples":[
                      "111222686238900021"
                    ]
                  },
                  "network_id": {
                    "type":"string",
                    "description":"Uniquely identifies an authenticated network.",
                    "examples":[
                      "n-ab1324"
                    ]
                  }
                }
              },
              "result":{
                "type":"object",
                "properties":{
                  "status":{
                    "type":"string",
                    "examples":[
                      "SUCCESS",
                      "UNAUTHENTICATED"
                    ]
                  },
                  "message":{
                    "type":"string",
                    "description":"Indicates the result status."
                  }
                }
              },
              "clientAddress":{
                "type":"array",
                "description":"Network address of the remote client.",
                "items":{
                  "type":"object",
                  "properties":{
                    "ip":{
                      "type":"string",
                      "description":"IPv4 or IPv6 address.",
                      "anyOf":[
                        {
                          "format":"ipv4"
                        },
                        {
                          "format":"ipv6"
                        }
                      ]
                    },
                    "port":{
                       "type":"number",
                       "description":"Port number, if known."
                    }
                  }
                }
              }
            }
          }
        }
      }
    },
    {
      "if":{
        "required":[
          "type"
        ],
        "properties":{
          "type":{
            "const":"io.confluent.kafka.server/authorization"
          }
        }
      },
      "then":{
        "properties":{
          "data":{
            "type":"object",
            "additionalProperties": false,
            "properties":{
              "serviceName": {
                "type": "string",
                "description": "The resource identifier of the service (the source) that received the request being logged.",
                "examples": [
                  "crn://confluent.cloud",
                  "crn://confluent.cloud/kafka=lkc-a1b2c",
                  "crn:///kafka=lkc-a1b2c"
                ]
              },
              "resourceName":{
                "type":"string",
                "description":"The resource name of the checked authorization rule.",
                "examples":[
                  "kafka-cluster",
                  "delivery-estimator",
                  "departures"
                ]
              },
              "methodName":{
                "type":"string",
                "description":"The type of request being logged.",
                "examples":[
                  "kafka.AlterConfigs",
                  "kafka.CreateAcls",
                  "kafka.CreateTopics",
                  "kafka.DeleteAcls",
                  "kafka.DeleteGroups"
                ]
              },
              "authenticationInfo":{
                "type":"object",
                "additionalProperties": false,
                "properties":{
                  "principal":{
                    "type":"string",
                    "description":"Identifies the authenticated principal that made the request.",
                    "examples":[
                      "User:12345"
                    ]
                  },
                  "principalResourceId":{
                    "type":"string",
                    "description":"Authenticated principal resourceId of the requester.",
                    "examples":[
                      "u-yw9507"
                    ]
                  },
                  "identity":{
                    "type":"string",
                    "description":"Identity of the requester in the format.",
                    "examples":[
                      "crn://confluent.cloud/organization=uuid-for-ourcorp/identity-provider=ourcorp-idp/identity=u-yw9507"
                    ]
                  }
                }
              },
              "authorizationInfo":{
                "type":"object",
                "additionalProperties": false,
                "properties":{
                  "granted":{
                    "type":"boolean",
                    "description":"The result of the authorization check."
                  },
                  "operation":{
                    "$ref":"#/$defs/operation"
                  },
                  "resourceType":{
                    "type":"string",
                    "description":"The type of the resource being checked for authorization.",
                    "examples":[
                      "Cluster",
                      "Group",
                      "Topic"
                    ]
                  },
                  "resourceName":{
                    "type":"string",
                    "description":"The resource name of the checked authorization rule.",
                    "examples":[
                      "kafka-cluster",
                      "delivery-estimator",
                      "departures"
                    ]
                  },
                  "patternType":{
                    "type":"string",
                    "description":"The pattern, LITERAL or PREFIX, used to match the resource against the authorization rule.",
                    "examples":[
                      "LITERAL",
                      "PREFIX"
                    ]
                  },
                  "aclAuthorization":{
                    "type":"object",
                    "description":"Details about an ACL rule.",
                    "properties":{
                      "permissionType":{
                        "$ref":"#/$defs/permissionType"
                      },
                      "host":{
                        "$ref":"#/$defs/host"
                      },
                      "actingPrincipal": {
                        "type": "string",
                        "description": "The decisive principal used for authorization."
                      }
                    }
                  },
                  "rbacAuthorization":{
                    "type":"object",
                    "properties":{
                      "role":{
                        "$ref":"#/$defs/role"
                      },
                      "scope":{
                        "type":"object",
                        "properties":{
                          "outerScope":{
                            "type":"array",
                            "description":"The path elements in the outer scopes with the outermost first",
                            "items":{
                              "type":"string"
                            }
                          },
                          "clusters": {
                             "type": "object",
                             "description": "Map of cluster identifiers.",
                             "additionalProperties": {
                               "type": "string"
                             }
                          }
                        },
                        "description":"The scope the action was authorized in"
                      },
                      "actingPrincipal": {
                        "type": "string",
                        "description": "The decisive principal used for authorization."
                      }
                    }
                  },
                  "superUserAuthorization":{
                    "$ref":"#/$defs/superUserAuthorization"
                  },
                  "assignedPrincipals": {
                    "type": "array",
                    "description": "The list of principals the user has access to, including user itself and identity pools",
                    "items": {
                      "type": "string"
                    }
                  }
                }
              },
              "request":{
                "type":"object",
                "properties":{
                  "clientId":{
                    "type":"string",
                    "description":"This is a user-supplied identifier for the client application. The user can use any identifier they like. This ID acts as a logical grouping across all requests from a particular client.",
                    "examples":[
                      "invoice-processor-admin",
                      "adminclient-42",
                      "the-replicator"
                    ]
                  },
                  "correlationId":{
                    "type":"string",
                    "description":"This is a user-supplied integer. It will be passed back in the response by the server, unmodified. It is useful for matching request and response between the client and server."
                  }
                }
              },
              "clientAddress":{
                "type":"array",
                "description":"Network address of the remote client.",
                "items":{
                  "type":"object",
                  "properties":{
                    "ip":{
                      "type":"string",
                      "description":"IPv4 or IPv6 address.",
                      "anyOf":[
                        {
                          "format":"ipv4"
                        },
                        {
                          "format":"ipv6"
                        }
                      ]
                    },
                    "port":{
                       "type":"number",
                       "description":"Port number, if known."
                    }
                  }
                }
              },
              "requestMetadata":{
                "type":"object",
                "properties":{
                  "connection_id": {
                    "type":"string",
                    "description":"Uniquely identifies an authenticated connection.Only present for successfully authenticated connections.May be used during analysis to correlate events",
                    "examples":[
                      "111222686238900021"
                    ]
                  },
                  "request_id": {
                    "type":"string",
                    "description":"Uniquely identifies a client request..",
                    "examples":[
                      "n-ab1324"
                    ]
                  }
                }
              }
            }
          }
        }
      }
    },
    {
     "if": {
       "required": [
         "type"
       ],
       "not": {
         "properties": {
           "type": {
             "enum": [
               "io.confluent.kafka.server/authentication",
               "io.confluent.kafka.server/authorization"
             ]
           }
         }
       }
    },
    "then":{
        "properties":{
          "data":{
            "type":"object",
            "description":"Additional details about the cloud request",
            "additionalProperties": false,
            "properties":{
              "serviceName": {
                "type": "string",
                "description": "The resource identifier of the service (the source) that received the request being logged.",
                "examples": [
                  "crn://confluent.cloud",
                  "crn://confluent.cloud/kafka=lkc-a1b2c",
                  "crn:///kafka=lkc-a1b2c"
                ]
              },
              "resourceName":{
                "type":"string",
                "description":"The resource name of the checked authorization rule.",
                "examples":[
                  "kafka-cluster",
                  "delivery-estimator",
                  "departures"
                ]
              },
              "methodName":{
                "type":"string",
                "description":"The type of request being logged.",
                "examples":[
                  "CreateKafkaCluster",
                  "GetAPIKeys",
                  "DeleteEnvironment",
                  "CreateConnector",
                  "DeleteAPIKey"
                ]
              },
              "cloudResources":{
                "type":"array",
                "description":"The resource(s) or collection(s) targeted in the operation.",
                "items":{
                  "type":"object",
                  "additionalProperties": false,
                  "properties":{
                    "scope":{
                      "type":"object",
                      "additionalProperties": false,
                      "description":"Defines the scope of the action.",
                      "properties":{
                        "resources":{
                          "type":"array",
                          "description":"The list of cloud resources that define the scope of the action.",
                          "items":{
                            "allOf":[{
                                "$ref":"#/$defs/resource"
                            }]
                          }
                        }
                      }
                    },
                    "resource":{
                      "allOf":[{
                        "$ref":"#/$defs/resource"
                      }]
                    }
                  }
                }
              },
              "authenticationInfo":{
                "type":"object",
                "additionalProperties": false,
                "description":"Information about the principal and the credentials used to prove its identity.",
                "properties":{
                  "principal":{
                    "allOf":[{
                      "$ref":"#/$defs/principal",
                      "description":"Identifies the authenticated principal that is used for any authorization checks. Also identifies the authenticated principal that made the request, unless originalPrincipal is set."
                     }]
                  },
                  "originalPrincipal":{
                    "allOf":[{
                      "$ref":"#/$defs/principal",
                      "description":"If set, indicates the original principal that made the request. Used for situations where one principal can assume the identity of a different principal."
                     }]
                  },
                  "result":{
                    "description":"The result of authentication checks on the provided credentials.",
                    "enum":[
                      "UNSET",
                      "SUCCESS",
                      "FAILURE"
                    ]
                  },
                  "errorMessage":{
                    "type":"string",
                    "description":"A short, human-readable description of the reason authentication failed.",
                    "examples":[
                      "Authentication Failed. Wrong Credentials"
                    ]
                  },
                  "credentials":{
                    "allOf":[{
                      "$ref":"#/$defs/credentials"
                    }]
                  },
                 "identity":{
                   "type":"string",
                   "description":"Identity of the requester in the format.",
                   "examples":[
                     "crn://confluent.cloud/organization=uuid-for-ourcorp/identity-provider=ourcorp-idp/identity=u-yw9507"
                   ]
                 }
                }
              },
               "authorizationInfo":{
                 "type":"object",
                 "additionalProperties": false,
                 "description":"The result of this authorization check.",
                 "properties": {
                   "result": {
                     "enum": [
                       "UNSET",
                       "ALLOW",
                       "DENY"
                     ]
                   },
                   "operation": {
                     "$ref": "#/$defs/operation"
                   },
                   "assignedPrincipals": {
                     "type": "array",
                     "description": "The list of principals the user has access to, including user itself and identity pools",
                     "items": {
                       "allOf": [{
                         "$ref": "#/$defs/principal"
                       }]
                     }
                   },
                   "requestMetadata":{
                     "type":"object",
                     "properties":{
                       "connectionId":{
                         "type":"string",
                         "description":"Uniquely identifies an authenticated connection. Only present for successfully authenticated connections."
                       },
                       "requestId":{
                         "type":"array",
                         "description":"Uniquely identifies a client request.",
                         "items":{
                           "type":"string"
                         }
                       },
                       "clientAddress":{
                         "type":"array",
                         "description":"Network address of the remote client.",
                         "items":{
                           "type":"object",
                           "properties":{
                             "ip":{
                               "type":"string",
                               "description":"IPv4 or IPv6 address.",
                               "anyOf":[
                                 {
                                   "format":"ipv4"
                                 },
                                 {
                                   "format":"ipv6"
                                 }
                               ]
                             },
                             "port":{
                               "type":"number",
                               "description":"Port number, if known."
                             }
                           }
                         }
                       },
                       "clientId":{
                         "type":"string",
                         "description":"A client-provided identifier, logged for correlation, as a courtesy to the client."
                       },
                       "clientTraceId":{
                         "type":"string",
                         "description":"A client-provided identifier, logged for correlation, as a courtesy to the client."
                       }
                     }
                   },
                   "request":{
                     "type":"object",
                     "description":"Describes additional audit-worthy details about the request.",
                     "properties":{
                       "accessType":{
                         "description":"The type of the request",
                         "enum":[
                           "UNKNOWN",
                           "READ_ONLY",
                           "MODIFICATION"
                         ]
                       },
                       "data":{
                         "type":[
                           "object",
                           "null"
                         ],
                         "description":"Audit-worthy details from the request. There are currently no guarantees of backwards-compatibility on the contents of this field. The contents of this field should not be relied on for any programmatic access.",
                         "additionalProperties":{
                           "type":[
                             "array",
                             "boolean",
                             "number",
                             "object",
                             "string",
                             "null"
                           ]
                         }
                       }
                     }
                   },
                   "superUserAuthorization": {
                     "allOf": [{
                       "$ref": "#/$defs/superUserAuthorization"
                     }]
                   },
                   "aclAuthorization": {
                     "type": "object",
                     "additionalProperties": false,
                     "description": "An ACL rule that resulted in the authorization check being denied or allowed.",
                     "properties": {
                       "permissionType": {
                         "allOf": [{
                           "$ref": "#/$defs/permissionType"
                         }]
                       },
                       "host": {
                         "allOf": [{
                           "$ref": "#/$defs/host"
                         }]
                       },
                       "resourceType": {
                         "allOf": [{
                           "$ref": "#/$defs/resourceType"
                         }]
                       },
                       "patternType": {
                         "allOf": [{
                           "$ref": "#/$defs/patternType"
                         }]
                       },
                       "patternName": {
                         "allOf": [{
                           "$ref": "#/$defs/patternName"
                         }]
                       },
                       "actingPrincipal": {
                         "allOf": [{
                           "$ref": "#/$defs/principal",
                           "description": "The decisive principal used for authorization."
                         }]
                       }
                     }
                   },
                   "rbacAuthorization": {
                     "type": "object",
                     "description": "A RBAC rule that resulted in the authorization check being allowed.",
                     "properties": {
                       "role": {
                         "allOf": [{
                           "$ref": "#/$defs/role"
                         }]
                       },
                       "cloudScope": {
                         "type": "object",
                         "additionalProperties": false,
                         "description": "The list of cloud resources involved in the scope of an action.",
                         "properties": {
                           "resources": {
                             "type": "array",
                             "description": "The list of cloud resources that define the scope of the action.",
                             "items": {
                               "allOf": [{
                                 "$ref": "#/$defs/resource"
                               }]
                             }
                           }
                         }
                       },
                       "operation": {
                         "$ref": "#/$defs/operation"
                       },
                       "resourceType": {
                         "allOf": [{
                           "$ref": "#/$defs/resourceType"
                         }]
                       },
                       "patternType": {
                         "allOf": [{
                           "$ref": "#/$defs/patternType"
                         }]
                       },
                       "patternName": {
                         "allOf": [{
                           "$ref": "#/$defs/patternName"
                         }]
                       },
                       "actingPrincipal": {
                         "allOf": [{
                           "$ref": "#/$defs/principal",
                           "description": "The decisive principal used for authorization."
                         }]
                       }
                     }
                   },
                   "ipfilterAuthorization": {
                     "type": "object",
                     "additionalProperties": false,
                     "description": "Details about an IP filter rule.",
                     "properties": {
                       "client_ip": {
                         "type": "string",
                         "description": "The request client IP address."
                       },
                       "resource_group": {
                         "type": "string",
                         "description": "Resource group that the IP filter specifies.",
                         "enum": [
                           "UNSET",
                           "MANAGEMENT"
                         ]
                       }
                     }
                   },
                   "privateLinkPolicyAuthorization": {
                     "type": "object",
                     "additionalProperties": false,
                     "description": "Details about a Private Link policy rule.",
                     "properties": {
                       "resource_network_type": {
                         "type": "string",
                         "description": "If the resource being accessed is deemed public or private.",
                         "enum": [
                           "UNKNOWN",
                           "PUBLIC",
                           "PRIVATE"
                         ]
                       },
                       "private_link_context_id": {
                         "type": "string",
                         "description": "Metadata associated with private link connection, if non-empty."
                       }
                     }
                   },
                   "resourceType": {
                     "type": "string",
                     "description": "The type of the resource being checked for authorization.",
                     "examples": [
                       "Cluster",
                       "Group",
                       "Topic"
                     ]
                   },
                   "resourceName": {
                     "type": "string",
                     "description": "The resource name of the checked authorization rule.",
                     "examples": [
                       "kafka-cluster",
                       "delivery-estimator",
                       "departures"
                     ]
                   }
                 }
               },
              "requestMetadata":{
                "type":"object",
                "properties":{
                  "connectionId":{
                    "type":"string",
                    "description":"Uniquely identifies an authenticated connection. Only present for successfully authenticated connections."
                  },
                  "requestId":{
                    "type":"array",
                    "description":"Uniquely identifies a client request.",
                    "items":{
                      "type":"string"
                    }
                  },
                  "clientAddress":{
                    "type":"array",
                    "description":"Network address of the remote client.",
                    "items":{
                      "type":"object",
                      "properties":{
                        "ip":{
                          "type":"string",
                          "description":"IPv4 or IPv6 address.",
                          "anyOf":[
                            {
                              "format":"ipv4"
                            },
                            {
                              "format":"ipv6"
                            }
                          ]
                        },
                        "port":{
                          "type":"number",
                          "description":"Port number, if known."
                        }
                      }
                    }
                  },
                  "clientId":{
                    "type":"string",
                    "description":"A client-provided identifier, logged for correlation, as a courtesy to the client."
                  },
                  "clientTraceId":{
                    "type":"string",
                    "description":"A client-provided identifier, logged for correlation, as a courtesy to the client."
                  },
                  "correlationId":{
                    "type":"string",
                    "description":"This is a user-supplied integer. It will be passed back in the response by the server, unmodified. It is useful for matching request and response between the client and server."
                  },
                  "networkId":{
                    "type":"string",
                    "description":"Uniquely identifies an authenticated network."
                  }
                }
              },
              "request":{
                "type":"object",
                "description":"Describes additional audit-worthy details about the request.",
                "properties":{
                  "accessType":{
                    "description":"The type of the request",
                    "enum":[
                      "UNKNOWN",
                      "READ_ONLY",
                      "MODIFICATION"
                    ]
                  },
                  "data":{
                    "type":[
                      "object",
                      "null"
                    ],
                    "description":"Audit-worthy details from the request. There are currently no guarantees of backwards-compatibility on the contents of this field. The contents of this field should not be relied on for any programmatic access.",
                    "additionalProperties":{
                      "type":[
                        "array",
                        "boolean",
                        "number",
                        "object",
                        "string",
                        "null"
                      ]
                    }
                  }
                }
              },
              "result":{
                "type":"object",
                "description":"Describes the result of the overall operation (i.e. success, failure, etc) along with any audit-worthy details. For example, this field contains the resource identifier of a newly created resource from a CREATE request.",
                "properties":{
                  "status":{
                    "description":"The status of the result.",
                    "enum":[
                      "UNSET",
                      "SUCCESS",
                      "FAILURE"
                    ]
                  },
                  "data":{
                    "type":[
                      "object",
                      "null"
                    ],
                    "description":"Audit-worthy details from the operation's result. There are currently no guarantees of backwards-compatibility on the contents of this field. The contents of this field should not be relied on for any programmatic access.",
                    "additionalProperties":{
                      "type":[
                        "array",
                        "boolean",
                        "number",
                        "object",
                        "string",
                        "null"
                      ]
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  ],
  "$defs":{
    "user":{
      "type":"object",
      "properties":{
        "resourceId":{
          "type":"string",
          "description":"Resource identifier",
          "examples":[
            "u-8191a"
          ]
        }
      }
    },
    "operation":{
      "type":"string",
      "description":"Identifies the operation being checked for authorization.",
      "examples":[
        "Alter",
        "AlterConfigs",
        "Create",
        "Delete",
        "DescribeConfigs"
      ]
    },
    "resourceType":{
      "type":"string",
      "description":"The type of the resource being checked for authorization.",
      "examples":[
        "Cluster",
        "Group",
        "Topic"
      ]
    },
    "resourceName":{
      "type":"string",
      "description":"The resource name of the checked authorization rule.",
      "examples":[
        "kafka-cluster",
        "delivery-estimator",
        "departures"
      ]
    },
    "patternType":{
      "type":"string",
      "description":"The pattern, LITERAL or PREFIX, used to match the resource against the authorization rule.",
      "examples":[
        "LITERAL",
        "PREFIX"
      ]
    },
    "patternName":{
      "type":"string",
      "description":"The resource name or prefix matched by this ACL rule.",
      "examples":[
        "*"
      ]
    },
    "permissionType":{
      "type":"string",
      "examples":[
        "ALLOW",
        "DENY"
      ]
    },
    "host":{
      "type":"string",
      "description":"Host to which the ACL rule applies, usually wildcard (*).",
      "examples":[
        "*"
      ]
    },
    "role":{
      "type":"string",
      "description":"A role that the principal has, granting authorization to perform the operation on the resource.",
      "examples":[
        "OrganizationAdmin",
        "EnvironmentAdmin",
        "CloudClusterAdmin",
        "MetricsViewer"
      ]
    },
    "resource":{
      "type":"object",
      "additionalProperties": false,
      "description":"The actual resource affected within the identified scope.",
      "properties":{
        "type":{
           "type":"string",
           "description":"Resource Identifier",
           "examples":[
             "ORGANIZATION",
             "KAFKA_CLUSTER",
             "CLUSTER_LINK"
           ]
        },
        "resourceId":{
          "type":"string",
          "description":"Resource Identifier",
          "examples":[
            "sa-8191a",
            "lkc-a1b2c"
          ]
        }
      }
    },
    "principal":{
      "type":"object",
      "additionalProperties": false,
      "properties":{
        "email":{
          "type":"string",
          "description":"Email address of the principal.",
          "examples":[
            "abc@confluent.io"
          ]
        },
        "confluentUser":{
          "allOf":[{
            "$ref": "#/$defs/user"
          }]
        },
        "confluentServiceAccount":{
          "allOf":[{
            "$ref": "#/$defs/user"
          }]
        },
        "identityPool":{
          "allOf":[{
            "$ref": "#/$defs/user"
          }]
        },
        "group":{
          "allOf":[{
            "$ref": "#/$defs/user"
          }]
        },
        "externalAccount":{
          "type":"object",
          "properties":{
            "namespace":{
              "type":"array",
              "items":{
                "type":"object",
                "properties":{
                  "type":{
                    "type":"string",
                    "examples":[
                      "idp",
                      "tenant"
                    ]
                  },
                  "id":{
                    "type":"string",
                    "examples":[
                      "confluent.cloud",
                      "a2efbdb2-af88-4b38-99a4-9a1e9d700963"
                    ]
                  }
                }
              }
            },
            "subject":{
              "type":"string",
              "description":"Identity of the requesting party, known to the IdP.",
              "examples":[
                "bilbo.baggins",
                "harry.potter"
              ]
            }
          }
        }
      },
      "oneOf":[
        {
          "required":[
            "confluentServiceAccount"
          ]
        },
        {
          "required":[
            "confluentUser"
          ]
        },
        {
          "required":[
            "externalAccount"
          ]
        },
        {
          "required":[
            "identityPool"
          ]
        },
        {
          "required":[
            "group"
          ]
        }
      ]
    },
    "credentials":{
      "type":"object",
      "description":"Information sent to prove the identity of the principal.",
      "oneOf":[
        {
        "properties":{
          "idSecretCredentials":{
            "type":"object",
            "properties":{
              "credentialId":{
                "type":"string",
                "description":"Identifies the credential",
                "examples":[
                  "bilbo.baggins",
                  "harry.potter"
                ]
              }
            }
          },
          "mechanism":{
            "allOf":[{
                "$ref": "#/$defs/mechanism"
                }]
            }
          },
          "additionalProperties": false
        },
        {
        "properties":{
          "idTokenCredentials":{
            "type":"object",
            "properties":{
              "type":{
                "type":"string",
                "description":"The type of the Id Token Credential. JWT for the foreseeable future.",
                "examples":[
                  "JWT"
                ]
              },
              "issuer":{
                "type":"string",
                "description":"Who signed the token."
              },
              "subject":{
                "type":"string",
                "description":"Identifies the principal."
              },
              "audience":{
                "type":"array",
                "items":{
                  "type":"string"
                }
              },
              "otherSignedClaims":{
               "type":"object"
              }
            }
          },
          "mechanism":{
            "allOf":[{
                "$ref": "#/$defs/mechanism"
            }]
          }
        },
        "additionalProperties": false
        },
        {
        "properties":{
          "certificateCredentials":{
            "type":"object",
            "properties":{
              "dname":{
                "type":"object",
                "description":"The principal identified by this certificate.",
                "properties":{
                  "cn":{
                    "type":"string",
                    "description":"Common name"
                  },
                  "ou":{
                    "type":"string",
                    "description":"Organizational unit"
                  },
                  "o":{
                    "type":"string",
                    "description":"Organization"
                  },
                  "l":{
                    "type":"string",
                    "description":"Locality"
                  },
                  "st":{
                    "type":"string",
                    "description":"State or province"
                  },
                  "c":{
                    "type":"string",
                    "description":"Country"
                  }
                }
              }
            }
          },
        "mechanism":{
            "allOf":[{
                "$ref": "#/$defs/mechanism"
                }]
            }
        },
        "additionalProperties": false
        },
        {
        "properties":{
          "delegateCredentials":{
            "type":"object",
            "properties":{
               "delegatePrincipal":{
                 "allOf": [{
                   "$ref": "#/$defs/principal",
                   "description": "Identifies the authenticated principal that made the request."
                 }]
               },
              "delegateCredentials":{
                "allOf":[{
                "$ref":"#/$defs/credentials"
              }]
            }
          }
        },
        "mechanism":{
          "allOf":[{
            "$ref": "#/$defs/mechanism"
            }]
          }
        },
        "additionalProperties": false
      }]
    },
    "mechanism":{
      "enum":[
        "UNSET",
        "SASL_PLAIN",
        "SASL_SCRAM",
        "SASL_OAUTHBEARER",
        "SASL_GSSAPI",
        "MTLS",
        "HTTP_BASIC",
        "HTTP_BEARER",
        "PLAINTEXT",
        "SSL"
      ]
    },
    "superUserAuthorization":{
      "type":"boolean",
      "description":"If true, access was authorized because principal is a super-user."
    }
  }
}
  1. Navigate to the Confluent Cloud environment that contains your cluster.

  2. In the navigation menu, click Stream Governance > Schema Registry.

  3. Click the Schemas tab, then click Add a schema.

  4. In the Subject name field, enter confluent-audit-log-events-value.

  5. From the Type drop-down, select JSON Schema.

  6. Paste the audit log event schema into the Schema editor.

  7. Click Validate schema to ensure the schema is correct.

    Create schema page with the subject name confluent-audit-log-events-value, the JSON Schema type, and the schema in the editor
  8. Click Create to save the schema.

    Schema Registry Schemas tab listing the confluent-audit-log-events-value subject with the JSON Schema type and a View topics link

    The schema appears in the Schemas list.

  9. Click View topics to see the topic associated with this schema: confluent-audit-log-events. You can click the topic name to view messages, monitoring, and other information about this topic.

  1. Copy the audit log event schema into a local file named confluent-audit-log-events-value.json.

  2. Make sure that the environment that contains your cluster is the active environment. The command uses the Schema Registry cluster for that environment.

    confluent environment use <env_id>
    
  3. Run confluent schema-registry schema create to register the schema with the subject name confluent-audit-log-events-value:

    confluent schema-registry schema create \
        --subject confluent-audit-log-events-value \
        --schema confluent-audit-log-events-value.json \
        --type json
    
  4. To confirm that the schema was registered, run:

    confluent schema-registry schema describe \
        --subject confluent-audit-log-events-value \
        --version latest
    

View the mirror topic from the Dedicated cluster

Follow these steps to view the Audit Logs mirror topic on the Dedicated cluster (destination).

  1. In Confluent Cloud, select your environment and your Dedicated cluster. The URI resembles confluent.cloud/environments/<env_id>/clusters/<cluster_id>/.

  2. Click the confluent-audit-log-events topic, and then click the Messages tab.

    Messages tab for the confluent-audit-log-events mirror topic, showing active mirroring and a list of audit log messages
  3. Click Explore Stream Lineage to see which consumer groups are consuming the audit logs topic.

    Stream lineage graph showing the confluent-audit-log-events topic connected to five consumer groups
  1. Make sure that the Dedicated cluster (destination) is the active cluster:

    confluent environment use <env_id>
    confluent kafka cluster use <destination_cluster_id>
    
  2. View the mirror topic details, including its mirroring status:

    confluent kafka mirror describe confluent-audit-log-events --link my-link
    
  3. Create an API key for the destination cluster (or reuse an existing one), and set it as the active key:

    confluent api-key create --resource <destination_cluster_id>
    confluent api-key use <api_key> --resource <destination_cluster_id>
    
  4. Consume messages from the mirror topic. Use --from-beginning to read from the start of the topic.

    confluent kafka topic consume confluent-audit-log-events --from-beginning
    

    Use Ctrl+C to exit the consumer.

The Confluent CLI doesn’t show stream lineage. To see which consumer groups are consuming the audit logs topic, click Explore Stream Lineage on the Cloud Console tab.