Confluent Metadata API Reference¶
The Confluent Metadata API has many endpoints, conceptually grouped as follows:
Authentication
Authenticates users against LDAP and returns user bearer tokens that can be used with the other MDS endpoints and components in Confluent Platform (when configured to do so).
Authorization
Authorizes users to perform specific actions. Clients are not expected to use these endpoints, which are used by Confluent Platform components (such as Connect and ksqlDB) to authorize user actions.
Role Based Access Control
- Role binding CRUD
- Role binding summaries (used by Confluent CLI)
- High-level role binding management and rollups (used by Confluent Control Center)
Centralized ACL Control
- With Authorization using centralized ACLs Confluent Platform provides the option to store ACLs in Kafka instead of ZooKeeper
- ACL CRUD for legacy Kafka-managed and centralized MDS-based ACLs
Audit Log Configuration
Configuration governing which events get logged, and where those audit log events are sent. Works in conjunction with the Cluster Registry to push configuration changes to Kafka clusters.
Cluster Registry
Tracking and naming CP components / clusters.
- Manually populated and updated by Admins.
- Leveraged by the Audit Log configuration.
- Leveraged by RBAC APIs to allow for RoleBinding calls to use “nice names” for clusters instead of cluster IDs.
Note
In all the RBAC CRUD APIs where you used to have to specify cluster IDs, if
you have registered clusters with Cluster Registry, you can use the clusterName
instead. However, you must adhere to the following restrictions:
- Use
clusterName
orclusters
, but not both. - When you use
clusters
,kafka-cluster
is always required, because other clusters types do not have unique IDs.
Access Restrictions¶
Some endpoints can be called by any authenticated user, while others can only be called by “admins”. Additionally, many of the endpoints in the API involve two users: the user who is calling the endpoint (the “calling” principal) and the user that the API call is about (the “target” principal).
Example: User “alice”, who has the UserAdmin role, and is identified by her HTTP Basic authentication credentials or a bearer token, calls the CRUD endpoint to modify role bindings about user “bob”.
To document what access restrictions each endpoint has, use the following legend, which lists access in order from least restrictive to most restrictive:
- LDAP
- Any authenticated LDAP user
- Admins+User
- Admins or users requesting information about themselves
- Admins+ResourceOwners
- Admins or users with ResourceOwner role
- Admins+AclUsers
- Admins or the user having the required ACL permissions
- Admins
- Admins only, which can be UserAdmin, SystemAdmin, broker super.user, or SecurityAdmin as “Read”
Overview of Responses¶
Valid Responses
- 200: Successful call with a return body.
- 204: Successful call with no return body.
Errors
- 400: Invalid request. JSON parsing error, or otherwise incorrect request.
- 401: Not Authenticated. You need to pass valid HTTP Basic authentication credentials or a user bearer token.
- 403: Not Authorized. Valid request, but you aren’t authorized to perform the requested action.
- 404: Invalid URL. If you get this error from the authenticate endpoint, it
means bearer token authentication needs to be enabled in the configuration
(
confluent.metadata.server.authentication.method=BEARER
) - 405: Method Not Allowed. Using the wrong HTTP method on a valid endpoint (for example, GET instead of POST).
- 409: Conflict. Adding a new resource or updating an existing resource
that would result in a conflict with existing state.
- This error can be thrown by Audit Logs and Cluster Registry APIs.
- 415: Invalid Content Type. Usually, a consequence of “application/json” as request body header.
- 500: Server Error.
Special Resource Types¶
Cluster and KsqlCluster are special ResourceTypes because they grant
resource-scoped roles like ResourceOwner and DeveloperManage limited
access to cluster-level operations (for example, Describe Configs on
Kafka clusters). These special resource types only accept LITERAL
patterns with the values kafka-cluster
and ksql-cluster
,
respectively.
Exploring the API¶
The Metadata Service ships with an OpenAPI spec that you can explore and interact with using an embedded Swagger UI.
To enable the embedded Swagger UI, specify the following property in the broker’s
configuration when configuring the Metadata Service: confluent.metadata.server.openapi.enable=true
.
This will make the Swagger UI available on the broker as http://<host>:<port>/security/openapi/swagger-ui/index.html
.
Note
Specify the same port that MDS is listening on as the Swagger UI port (the default is 8090
).
Warning
The Swagger UI and the Open API spec file are provided for development and test purposes only:
- Do not enable in production environments.
- The Swagger UI works only with HTTP.
Private RBAC UI Endpoints¶
These endpoints were developed specifically to power the Confluent Control Center UI. As such, they only focus on those use cases and have only been tested in the context of Confluent Control Center. These endpoints have not been tested, nor has their usability been evaluated with respect to manual API calls.
API Endpoints¶
Tokens and Authentication¶
-
GET
/security/1.0/authenticate
¶ Get a bearer token.
Callable by LDAP users.
Example request:
GET /security/1.0/authenticate HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
authenticated
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "auth_token": "string", "token_type": "string", "expires_in": 1.0 }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
- 200 OK –
Authorization¶
-
PUT
/security/1.0/authorize
¶ Authorize operations against resourceType for a given user.
Callable by Admins+User.
Example request:
PUT /security/1.0/authorize HTTP/1.1 Host: example.com Content-Type: application/json { "userPrincipal": "User:bob", "actions": [ { "scope": { "clusters": { "kafka-cluster": "K_GUID" } }, "resourceName": "clicksTopic1", "resourceType": "Topic", "operation": "Read" } ] }
Status Codes: - 200 OK –
Authorization proccessed
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "ALLOWED", "DENIED" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
- 200 OK –
Metadata Service Operations¶
-
GET
/security/1.0/activenodes/{protocol}
¶ Returns all the nodes running the Metadata Service REST API.
Clients are expected to round robin call to these endpoints if they don’t set up a load balancer in front of the Metadata Service nodes.
Callable by LDAP users.
Parameters: - protocol (string) – Should be “http” or “https”.
Example request:
GET /security/1.0/activenodes/{protocol} HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
List of other nodes running the Metadata Service.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "http://10.10.10.50:8090", "http://10.10.10.51:8090" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
GET
/security/1.0/metadataClusterId
¶ Returns the ID of the Kafka cluster that MDS is running on.
Callable by LDAP users.
Example request:
GET /security/1.0/metadataClusterId HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
Lists other nodes running the Metadata Service.
Example response:
HTTP/1.1 200 OK Content-Type: application/json ENZ7FS8uTQOhXElRiE-L7g
- 200 OK –
-
GET
/security/1.0/features
¶ What features the running version of MDS supports.
This endpoint does not require authentication.
Example request:
GET /security/1.0/features HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
What features MDS supports, and a human friendly description of them.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "features": {}, "legend": {} }
- 200 OK –
RBAC - Role Definitions¶
-
GET
/security/1.0/roles
¶ Returns all the roles defined in the system.
For information and developer purposes.
Callable by LDAP users.
Example request:
GET /security/1.0/roles HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
List of roles with resourceType and allowed operations.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "name": "string", "accessPolicy": { "scopeType": "string", "allowedOperations": [ { "resourceType": "string", "operations": [ "string" ] } ] } } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
- 200 OK –
-
GET
/security/1.0/roles/{roleName}
¶ List the resourceType and operations allowed for a given role.
Callable by LDAP users.
Parameters: - roleName (string) – Role name to look up.
Example request:
GET /security/1.0/roles/{roleName} HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
A single role.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "name": "string", "accessPolicy": { "scopeType": "string", "allowedOperations": [ { "resourceType": "string", "operations": [ "string" ] } ] } }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
GET
/security/1.0/roleNames
¶ Returns the names of all the roles defined in the system.
For information and developer purposes.
Callable by LDAP users.
Example request:
GET /security/1.0/roleNames HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
The names of the roles defined in the system.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "string" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
- 200 OK –
RBAC - RoleBinding CRUD¶
-
POST
/security/1.0/principals/{principal}/roles/{roleName}
¶ Binds the principal to a cluster-scoped role for a specific cluster or in the given scope.
Callable by Admins.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- roleName (string) – The name of the cluster-scoped role to bind the user to.
Example request:
POST /security/1.0/principals/{principal}/roles/{roleName} HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 204 No Content – Role Granted
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
DELETE
/security/1.0/principals/{principal}/roles/{roleName}
¶ Remove the role (cluster or resource scoped) from the principal at the given scope/cluster.
No-op if the user doesn’t have the role.
Callable by Admins.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- roleName (string) – The name of the role.
Example request:
DELETE /security/1.0/principals/{principal}/roles/{roleName} HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 204 No Content – Role removal processed.
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/principals/{principal}/roles/{roleName}/resources
¶ Look up the rolebindings for the principal at the given scope/cluster using the given role.
Callable by Admins.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- roleName (string) – The name of the role.
Example request:
POST /security/1.0/principals/{principal}/roles/{roleName}/resources HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
Granted
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "resourceType": "Topic", "name": "clicksTopic1", "patternType": "LITERAL" }, { "resourceType": "Topic", "name": "orders-2019", "patternType": "PREFIXED" } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/principals/{principal}/roles/{roleName}/bindings
¶ Incrementally grant the resources to the principal at the given scope/cluster using the given role.
Callable by Admins+ResourceOwners.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- roleName (string) – The name of the role.
Example request:
POST /security/1.0/principals/{principal}/roles/{roleName}/bindings HTTP/1.1 Host: example.com Content-Type: application/json { "scope": { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "resourcePatterns": [ { "resourceType": "string", "name": "string", "patternType": "string" } ] }
Status Codes: - 204 No Content – Granted
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
DELETE
/security/1.0/principals/{principal}/roles/{roleName}/bindings
¶ Incrementally remove the resources from the principal at the given scope/cluster using the given role.
Callable by Admins+ResourceOwners.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- roleName (string) – The name of the role.
Example request:
DELETE /security/1.0/principals/{principal}/roles/{roleName}/bindings HTTP/1.1 Host: example.com Content-Type: application/json { "scope": { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "resourcePatterns": [ { "resourceType": "string", "name": "string", "patternType": "string" } ] }
Status Codes: - 204 No Content – Resources Removed
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
PUT
/security/1.0/principals/{principal}/roles/{roleName}/bindings
¶ Overwrite existing resource grants.
Callable by Admins+ResourceOwners.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- roleName (string) – The name of the role.
Example request:
PUT /security/1.0/principals/{principal}/roles/{roleName}/bindings HTTP/1.1 Host: example.com Content-Type: application/json { "scope": { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "resourcePatterns": [ { "resourceType": "string", "name": "string", "patternType": "string" } ] }
Status Codes: - 204 No Content – Resources Set
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
RBAC - RoleBinding Summaries¶
-
POST
/security/1.0/lookup/principals/{principal}/roleNames
¶ Returns the effective list of role names for a principal.
For groups, these are the roles that are bound.
For users, this is the combination of roles granted to the specific user and roles granted to the user’s groups.
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Example request:
POST /security/1.0/lookup/principals/{principal}/roleNames HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
List of role names.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "Cluster Admin", "Security Admin" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/lookup/principal/{principal}/resources
¶ Look up the resource bindings for the principal at the given scope/cluster.
Includes bindings from groups that the user belongs to.
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Example request:
POST /security/1.0/lookup/principal/{principal}/resources HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
Nested map of principal-to-role-to-resources.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "User:alice": { "DeveloperRead": [ { "resourceType": "Topic", "name": "billing-invoices", "patternType": "LITERAL" } ] }, "Group:Investors": { "DeveloperRead": [ { "resourceType": "Topic", "name": "investing-", "patternType": "PREFIXED" } ] } }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/lookup/role/{roleName}
¶ Look up the KafkaPrincipals who have the given role for the given scope.
Callable by Admins.
Parameters: - roleName (string) – Role name to look up.
Example request:
POST /security/1.0/lookup/role/{roleName} HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
List of fully-qualified KafkaPrincipals.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "User:alice", "Group:FinanceAdmin" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/lookup/role/{roleName}/resource/{resourceType}/name/{resourceName}
¶ Look up the KafkaPrincipals who have the given role on the specified resource for the given scope.
Callable by Admins.
Parameters: - roleName (string) – Role name to look up.
- resourceType (string) – Type of resource to look up.
- resourceName (string) – Name of resource to look up.
Example request:
POST /security/1.0/lookup/role/{roleName}/resource/{resourceType}/name/{resourceName} HTTP/1.1 Host: example.com Content-Type: application/json { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
List of fully-qualified KafkaPrincipals.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "User:alice", "Group:FinanceAdmin" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
Kafka ACL Management¶
-
POST
/security/1.0/acls
¶ Creates Kafka ACLs for given AclBinding.
Callable by Admins+AclUsers.
Example request:
POST /security/1.0/acls HTTP/1.1 Host: example.com Content-Type: application/json { "scope": { "clusters": { "kafka-cluster": "string" } }, "aclBinding": { "pattern": { "resourceType": "UNKNOWN", "name": "string", "patternType": "UNKNOWN" }, "entry": { "principal": "string", "host": "string", "operation": "UNKNOWN", "permissionType": "UNKNOWN" } } }
Status Codes: - 204 No Content – Added AclBinding.
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
DELETE
/security/1.0/acls
¶ Deletes Kafka ACLs according to the supplied filter.
Callable by Admins+AclUsers.
Example request:
DELETE /security/1.0/acls HTTP/1.1 Host: example.com Content-Type: application/json { "scope": { "clusters": { "kafka-cluster": "string" } }, "aclBindingFilter": { "patternFilter": { "resourceType": "UNKNOWN", "name": "string", "patternType": "UNKNOWN" }, "entryFilter": { "principal": "string", "host": "string", "operation": "UNKNOWN", "permissionType": "UNKNOWN" } } }
Status Codes: - 200 OK –
Deleted AclBindings.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "pattern": { "resourceType": "UNKNOWN", "name": "string", "patternType": "UNKNOWN" }, "entry": { "principal": "string", "host": "string", "operation": "UNKNOWN", "permissionType": "UNKNOWN" } } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
- 200 OK –
-
POST
/security/1.0/acls:search
¶ Lists Kafka ACLs according to the supplied filter.
Callable by Admins+AclUsers.
Example request:
POST /security/1.0/acls:search HTTP/1.1 Host: example.com Content-Type: application/json { "scope": { "clusters": { "kafka-cluster": "string" } }, "aclBindingFilter": { "patternFilter": { "resourceType": "UNKNOWN", "name": "string", "patternType": "UNKNOWN" }, "entryFilter": { "principal": "string", "host": "string", "operation": "UNKNOWN", "permissionType": "UNKNOWN" } } }
Status Codes: - 200 OK –
Returns all AclBindings that match the given AclBindingFilter.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "pattern": { "resourceType": "UNKNOWN", "name": "string", "patternType": "UNKNOWN" }, "entry": { "principal": "string", "host": "string", "operation": "UNKNOWN", "permissionType": "UNKNOWN" } } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
- 200 OK –
Cluster Registry¶
-
GET
/security/1.0/registry/clusters
¶ Returns a list of all clusters in the registry, optionally filtered by cluster type.
If the calling principal doesn’t have permissions to see the full cluster info, some information (“hosts”, “protocol”, etc) is redacted.
Callable by Admins+User.
Query Parameters: - clusterType (string) – Optionally filter down by cluster type.
Example request:
GET /security/1.0/registry/clusters HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
List of Clusters.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "clusterName": "string", "scope": { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "hosts": [ { "host": "string", "port": 1 } ], "protocol": "SASL_PLAINTEXT" } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/registry/clusters
¶ Define/overwrite named clusters.
May result in a 409 Conflict if the name and scope combination of any cluster conflicts with existing clusters in the registry.
Callable by Admins.
Example request:
POST /security/1.0/registry/clusters HTTP/1.1 Host: example.com Content-Type: application/json [ { "clusterName": "string", "scope": { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "hosts": [ { "host": "string", "port": 1 } ], "protocol": "SASL_PLAINTEXT" } ]
Status Codes: - 204 No Content – Clusters added.
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
GET
/security/1.0/registry/clusters/{clusterName}
¶ Returns the information for a single named cluster, assuming the cluster exists and is visible to the calling principal.
Callable by Admins+User.
Parameters: - clusterName (string) – The name of cluster (ASCII printable characters without spaces).
Example request:
GET /security/1.0/registry/clusters/{clusterName} HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
The named cluster, if it exists and the caller has permission to see it.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "clusterName": "string", "scope": { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "hosts": [ { "host": "string", "port": 1 } ], "protocol": "SASL_PLAINTEXT" }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
DELETE
/security/1.0/registry/clusters/{clusterName}
¶ Remove a named cluster from the registry.
Callable by Admins.
Parameters: - clusterName (string) – The name of cluster (ASCII printable characters without spaces).
Status Codes: - 204 No Content – Cluster removed.
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
Audit Log Configuration¶
-
GET
/security/1.0/audit/config
¶ Get the entire audit log configuration, including the live retention time policy values (``retention_ms``) for the destination topics.
Requires the “AuditAdmin” role on the metadata service (MDS) cluster and every Kafka cluster in the cluster registry.
Callable by Admins.
Example request:
GET /security/1.0/audit/config HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
The entire audit log configuration.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "destinations": { "topics": { "confluent-audit-log-events_general_allowed_events": { "retention_ms": 2592000000 }, "confluent-audit-log-events_general_denied_events": { "retention_ms": 7776000000 }, "confluent-audit-log-events_finance_reads": { "retention_ms": 157680000000 } } }, "excluded_principals": [ "User:Alice", "User:service_account_id" ], "default_topics": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" }, "routes": { "crn://mds1.example.com/kafka=*/topic=*": { "management": { "allowed": "", "denied": "" }, "authorize": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" } }, "crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/topic=finance-*": { "consume": { "allowed": "confluent-audit-log-events_finance_reads", "denied": "confluent-audit-log-events_general_denied_events" }, "interbroker": { "allowed": "", "denied": "" } } }, "metadata": { "resource_version": "ASNFZ4mrze8BI0VniavN7w", "updated_at": "2019-09-09T12:34:56Z" } }
- 200 OK –
-
PUT
/security/1.0/audit/config
¶ Update the entire audit log configuration on the MDS cluster and all Kafka clusters known to the cluster registry.
Also creates missing destination topics on the destination cluster and updates the retention time policy of destination topics, if necessary.
Requires the “AuditAdmin” role on the MDS cluster and every Kafka cluster in the cluster registry.
May result in a 409 conflict error status if the
resource_version
in the JSON body of the request does not match the current version.Callable by Admins.
Example request:
PUT /security/1.0/audit/config HTTP/1.1 Host: example.com Content-Type: application/json { "destinations": { "topics": { "confluent-audit-log-events_general_allowed_events": { "retention_ms": 2592000000 }, "confluent-audit-log-events_general_denied_events": { "retention_ms": 7776000000 }, "confluent-audit-log-events_finance_reads": { "retention_ms": 157680000000 } } }, "excluded_principals": [ "User:Alice", "User:service_account_id" ], "default_topics": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" }, "routes": { "crn://mds1.example.com/kafka=*/topic=*": { "management": { "allowed": "", "denied": "" }, "authorize": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" } }, "crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/topic=finance-*": { "consume": { "allowed": "confluent-audit-log-events_finance_reads", "denied": "confluent-audit-log-events_general_denied_events" }, "interbroker": { "allowed": "", "denied": "" } } }, "metadata": { "resource_version": "ASNFZ4mrze8BI0VniavN7w", "updated_at": "2019-09-09T12:34:56Z" } }
Status Codes: - 200 OK –
The current audit log configuration spec
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "destinations": { "bootstrap_servers": [ "string" ], "topics": {} }, "excluded_principals": [ "string" ], "default_topics": { "allowed": "string", "denied": "string" }, "routes": {}, "metadata": { "resource_version": "string", "updated_at": "2024-11-03T22:10:49.737598", "modified_since": "2024-11-03T22:10:49.737598" } }
- 409 Conflict –
Concurrent modification resulted in a conflict. Response will be the current audit log configuration.
Example response:
HTTP/1.1 409 Conflict Content-Type: application/json { "destinations": { "bootstrap_servers": [ "string" ], "topics": {} }, "excluded_principals": [ "string" ], "default_topics": { "allowed": "string", "denied": "string" }, "routes": {}, "metadata": { "resource_version": "string", "updated_at": "2024-11-03T22:10:49.737598", "modified_since": "2024-11-03T22:10:49.737598" } }
- 200 OK –
-
GET
/security/1.0/audit/routes
¶ Lists all currently defined routes that match the queried resource or its sub-resources.
Multiple routes may match a resource, but only the most specific route will be selected for events related to the resource. This endpoint returns all matching routes regardless of whether or not they would actually be selected, or ignored in favor of a more specific route.
Requires the “AuditAdmin” role on the metadata service (MDS) cluster and every Kafka cluster in the cluster registry.
Callable by Admins.
The CRN patterns in the audit log config routes can contain wildcards. So a route with a CRN pattern like
crn://mds.example.com/kafka=*/topic=finance-*
would match events associated with the topic at addresscrn://mds.example.com/kafka=abc123/topic=finance-deposits
, or events associated with the topic atcrn://mds.example.com/kafka=xyz789/topic=finance-chargebacks
, but would not match events associated with the topiccrn://mds.example.com/kafka=abc123/topic=server-deployments
. So a route’s CRN pattern can match events from more than one resource, based on where the pattern’s wildcards are.It is possible to write multiple routes with different CRN patterns that match a given resource’s CRN. For example: the resource at
crn://mds.example.com/kafka=abc123/topic=finance-chargebacks
is matched by any of the following route CRN patterns:crn://mds.example.com/kafka=*/topic=*
crn://mds.example.com/kafka=abc123/topic=*
crn://mds.example.com/kafka=*/topic=finance-*
When there are multiple matching routes for an event, we select the matching route with the most specific CRN pattern. The most specific CRN pattern is the one with the greatest length before its first wildcard. So in the above example,
crn://mds.example.com/kafka=abc123/topic=*
wins.To break a tie, ignore the prefix that the patterns have in common. So, for example
crn://mds.example.com/kafka=*/topic=finance-*
is more specific thancrn://mds.example.com/kafka=*/topic=*
.This endpoint lists all currently defined routes that match the queried resource or its sub-resources, regardless of whether or not they would actually be selected, or ignored in favor of a more specific route.
A query pattern like …
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=qa-test
… would match all of the following routes …
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=qa-test/connector=from-db4
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=qa-test/connector=*
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=*/connector=*
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=qa-*
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=*
crn://mds1.example.com/kafka=*/connect=qa-*
crn://mds1.example.com/kafka=*/connect=qa-*/connector=*
… but would not match any of these routes …
crn://mds1.example.com/kafka=*/ksql=*
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=stg-*
crn://mds1.example.com/kafka=zyxwv-UTSRQPO_98765432/connect=qa-*
crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/topic=qa-*
Query Parameters: - q (string) – A Confluent resource name (CRN) .
Example request:
GET /security/1.0/audit/routes HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
Matching routes and their rules
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "default_topics": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" }, "routes": { "crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=qa-test/connector=from-db4": { "management": { "allowed": "", "denied": "" } }, "crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/connect=*": { "authorize": { "allowed": "confluent-audit-log-events_abc-connector-allowed", "denied": "confluent-audit-log-events_abc-connector-denied" } } } }
-
GET
/security/1.0/audit/lookup
¶ Returns the route describing how messages regarding this CRN would be routed.
Requires the “AuditAdmin” role on the metadata service (MDS) cluster and every Kafka cluster in the cluster registry.
The CRN patterns in the audit log config routes can contain wildcards. So a route with a CRN pattern like
crn://mds.example.com/kafka=*/topic=finance-*
would match events associated with the topic at addresscrn://mds.example.com/kafka=abc123/topic=finance-deposits
, or events associated with the topic atcrn://mds.example.com/kafka=xyz789/topic=finance-chargebacks
, but would not match events associated with the topiccrn://mds.example.com/kafka=abc123/topic=server-deployments
. So a route’s CRN pattern can match events from more than one resource, based on where the pattern’s wildcards are.It is possible to write multiple routes with different CRN patterns that match a given resource’s CRN. For example: the resource at
crn://mds.example.com/kafka=abc123/topic=finance-chargebacks
is matched by any of the following route CRN patterns:crn://mds.example.com/kafka=*/topic=*
crn://mds.example.com/kafka=abc123/topic=*
crn://mds.example.com/kafka=*/topic=finance-*
When there are multiple matching routes for an event, we select the matching route with the most specific CRN pattern. The most specific CRN pattern is the one with the greatest length before its first wildcard. So in the above example,
crn://mds.example.com/kafka=abc123/topic=*
wins.To break a tie, ignore the prefix that the patterns have in common. So, for example
crn://mds.example.com/kafka=*/topic=finance-*
is more specific thancrn://mds.example.com/kafka=*/topic=*
.This endpoint resolves the matching and precedence rules of all configured audit log routes for you, and returns the one selected (most specific) matching route describing how messages regarding the given CRN would be routed. Callable by Admins.
Query Parameters: - crn (string) – A Confluent resource name (CRN).
Example request:
GET /security/1.0/audit/lookup HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
The CRN pattern of the route that will be selected and used by the system (or else the string
"default"
), and the route’s rules for various audit log event categories. Categories includeproduce
,consume
,interbroker
,authorize
,authentication
,management
, heartbeat`, anddescribe
. If needed, the server fills in any missing ornull
values in the rules by coalescing with defaults:authentication
,authorize
, andmanagement
categories will default to the destination topics specified indefault_topics
. All other categories default to an empty string. Rules that resolve to an empty string indicate that events will be discarded.Example response:
HTTP/1.1 200 OK Content-Type: application/json { "route": "crn://mds1.example.com/kafka=abcde_FGHIJKL-01234567/topic=finance-*", "categories": { "authorize": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" }, "consume": { "allowed": "", "denied": "confluent-audit-log-events_finance_denied" }, "describe": { "allowed": "", "denied": "" }, "heartbeat": { "allowed": "", "denied": "" }, "interbroker": { "allowed": "", "denied": "" }, "management": { "allowed": "confluent-audit-log-events_general_allowed_events", "denied": "confluent-audit-log-events_general_denied_events" }, "produce": { "allowed": "confluent-audit-log-events_finance_produce_allowed", "denied": "confluent-audit-log-events_finance_denied" } } }
Private RBAC UI - Cluster Visibility¶
-
POST
/security/1.0/lookup/principals/{principal}/visibility
¶ Endpoint for Confluent Control Center to determine visibilty of Kafka and its sub-clusters for the specifed principal.
The intent is that this endpoint is called with cluster IDs that actually exist.
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Example request:
POST /security/1.0/lookup/principals/{principal}/visibility HTTP/1.1 Host: example.com Content-Type: application/json [ { "kafka-cluster": "string", "connect-clusters": [ "string" ], "schema-registry-clusters": [ "string" ], "ksql-clusters": [ "string" ] } ]
Status Codes: - 200 OK –
Returns cluster IDs and true/false if the user has any role associated with it.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "kafka-cluster": { "id": "string", "visible": true, "clusterName": "string" }, "connect-clusters": [ { "id": "string", "visible": true, "clusterName": "string" } ], "schema-registry-clusters": [ { "id": "string", "visible": true, "clusterName": "string" } ], "ksql-clusters": [ { "id": "string", "visible": true, "clusterName": "string" } ] }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
GET
/security/1.0/lookup/managed/clusters/principal/{principal}
¶ Identifies the scopes for the rolebindings that a user can see.
May include rolebindings from scopes and clusters that never existed or previously existed (in other words, rolebindings that have been decommissioned, but are still defined in the system).
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Query Parameters: - clusterType (string) – Filter down by cluster type.
Example request:
GET /security/1.0/lookup/managed/clusters/principal/{principal} HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
List of Scopes
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
Private RBAC UI - My RoleBindings¶
-
GET
/security/1.0/lookup/rolebindings/principal/{principal}
¶ List all rolebindings for the specifed principal for all scopes and clusters that have any rolebindings.
Be aware that this simply looks at the rolebinding data, and does not mean that the clusters actually exist.
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Query Parameters: - clusterType (string) – Filter down by a cluster type.
Example request:
GET /security/1.0/lookup/rolebindings/principal/{principal} HTTP/1.1 Host: example.com
Status Codes: - 200 OK –
List of RoleBindings for the user per scope.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ { "scope": { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "rolebindings": {} } ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/lookup/rolebindings/principal/{principal}
¶ List all rolebindings for the specifed principal and scope.
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Example request:
POST /security/1.0/lookup/rolebindings/principal/{principal} HTTP/1.1 Host: example.com Content-Type: application/json { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
Item per Scope
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "scope": { "clusterName": "string", "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "rolebindings": {} }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
Private RBAC UI - Manage RoleBindings¶
-
POST
/security/1.0/lookup/managed/clusters/principal/{principal}
¶ Identify the rolebinding abilities (view vs manage) the user has on the specified scope.
Used by the Confluent Control Center UI to control access to rolebinding add/remove buttons.
Callable by Admins+ResourceOwners.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Example request:
POST /security/1.0/lookup/managed/clusters/principal/{principal} HTTP/1.1 Host: example.com Content-Type: application/json { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
The rolebinding abilities the user has for a specified scope.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "cluster": [ "string" ], "resources": {} }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
-
POST
/security/1.0/lookup/managed/rolebindings/principal/{principal}
¶ Identify the rolebindings this user can see and manage.
Callable by Admins+ResourceOwners.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
Query Parameters: - resourceType (string) – Filter down by resource type.
Example request:
POST /security/1.0/lookup/managed/rolebindings/principal/{principal} HTTP/1.1 Host: example.com Content-Type: application/json { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
Rolebindings that the user can manage.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "scope": { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }, "cluster_role_bindings": {}, "resource_role_bindings": {} }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
Private RBAC UI - Creation Guidelines¶
-
POST
/security/1.0/lookup/principal/{principal}/resource/{resourceType}/operation/{operation}
¶ Summarizes what resources and rolebindings this principal is allowed to create.
Callable by Admins+User.
Parameters: - principal (string) – Fully-qualified KafkaPrincipal string for a user or group.
- resourceType (string) – The type of resource to create or the type of resource to specify when creating a new rolebinding.
- operation (string) – “Create” for creating an actual resource, “AlterAccess” for creating a rolebinding for a user.
Example request:
POST /security/1.0/lookup/principal/{principal}/resource/{resourceType}/operation/{operation} HTTP/1.1 Host: example.com Content-Type: application/json { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
A deduped and squashed view of the user’s rolebindings for creating resources or rolebindings.
Example response:
HTTP/1.1 200 OK Content-Type: application/json { "result": "SOME", "resourcePatterns": [ { "resourceType": "Topic", "name": "billing-invoices", "patternType": "LITERAL" }, { "resourceType": "Topic", "name": "investing-", "patternType": "PREFIXED" } ] }
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }
Private RBAC UI - Cached User Store Information¶
-
POST
/security/1.0/rbac/principals
¶ List of MDS cached users and groups.
For use by a rolebinding admin on the provided scope.
Callable by Admins+ResourceOwners, but not broker super.users.
Query Parameters: - type (string) – The type of principals requested.
Example request:
POST /security/1.0/rbac/principals HTTP/1.1 Host: example.com Content-Type: application/json { "clusters": { "kafka-cluster": "string", "connect-cluster": "string", "ksql-cluster": "string", "schema-registry-cluster": "string" } }
Status Codes: - 200 OK –
The list of principals for the requested type, or all principals.
Example response:
HTTP/1.1 200 OK Content-Type: application/json [ "Group:admin", "Group:developers", "Group:users", "User:alice", "User:bob", "User:charlie", "User:david" ]
- default –
Error Response
Example response:
HTTP/1.1 default - Content-Type: application/json { "status_code": 1, "error_code": 1, "type": "string", "message": "string", "errors": [ { "error_type": "string", "message": "string" } ] }