Manage Kafka Admin REST APIs

This topic describes how to configure and manage Apache Kafka® Admin REST APIs endpoints when you manage Confluent Platform using Confluent for Kubernetes (CFK).

When either managing Kafka Topics using the KafkaTopic custom resource (CR), or managing Confluent RBAC Rolebindings using the Confluentrolebinding custom resource (CR), you’ll need to first set up an Admin REST APIs custom resource (CR) to specify which REST endpoint to use.

If you have one Kafka cluster that you are managing, it’s recommended to create a default KafkaRestClass CR in the same namespace, with the name default. This will be then be picked up as the default option by any KafkaTopic CR or Confluentrolebinding CR that you create.

You can configure multiple KafkaRestClass CRs to manage topics and role bindings across different Kafka clusters.

Create Admin REST APIs CR

Create a KafkaRestClass CR to configure Kafka Admin REST APIs:

apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestClass
metadata:
  name:                ----- [1]
  namespace:           ----- [2]
spec:
  kafkaClusterRef:     ----- [3]
    name:
    namespace:
  kafkaRest:           ----- [4]
    endpoint:          ----- [5]
    authentication:
      type:            ----- [6]
      basic:
      bearer:
    tls:               ----- [7]
      enabled: true
  • [1] Name of the KafkaRestClass CR. default is recommended.

  • [2] Namespace of the KafkaRestClass CR.

  • [3] Name of the Kafka cluster.

  • [4] Admin REST APIs configuration.

  • [5] Admin REST APIs endpoint. If not specified, CFK tries to discover the Kafka cluster in the same namespace through the kafka CR type.

  • [6] Authentication type. basic and bearer are supported.

    See Basic authentication and Bearer authentication for details.

  • [7] TLS client configuration. Required when MDS is running in the HTTPS mode.

Use the following command to see the complete KafkaRestClass custom resource definition (CRD). See Use kubectl to examine Confluent Platform CRDs for more information.

kubectl explain KafkaRestClass

Specify the Kafka cluster endpoint

By default, CFK tries to discover the Kafka cluster in the same namespace through the Kafka CR.

There are two mechanisms available for you to explicitly specify the Admin REST APIs configuration:

  • Specify the Kafka cluster CR name using kafkaClusterRef in the same or different namespace.

  • Specify the REST endpoint URL using kafkaRest.

    If you need to specify a Kafka cluster REST endpoint that is not in the same namespace, then you need to specify the kafkaRest configuration.

Discover Kafka using the Kafka endpoint in kafkaRest

You provide the Kafka cluster endpoint in spec.kafkaRest.endpoint.

If authentication is required, add spec.kafkaRest.authentication section. The supported authentication types are basic and bearer.

For example:

apiVersion: platform.confluent.io/v1beta1
kind: KafkaTopic
metadata:
  name: topic-a
  namespace: operator
spec:
  kafkaRest:
    endpoint: https://kafka.operator.svc.cluster.local:8090
    authentication:

Discover Kafka using Kafka CR name in kafkaClusterRef

You specify the Kafka cluster CR name using kafkaClusterRef in the same or different namespace.

If authentication is required, configure the spec.kafkaRest.authentication section in the CR.

For example:

apiVersion: platform.confluent.io/v1beta1
kind: KafkaTopic
metadata:
  name: topic-a
  namespace: operator
spec:
  replicas: 3
  partitionCount: 12
  kafkaClusterRef:
    name: kafka
    namespace: operator-test
  kafkaRest:
    authentication:
      type: basic
      basic:
        secretRef: mds-admin-client

Discover Kafka using Kafka CR type

If you do not provide kafkaRestClassRef or kafkaClusterRef in the topic CR configuration, CFK tries to discover the Kafka cluster in the same namespace through the kafka CR type.

If CFK finds more than one Kafka cluster, it will throw an error in logs and events.