Configure and Manage Confluent REST Proxy

The Confluent REST Proxy provides a RESTful interface to a Kafka cluster. The Standalone REST Proxy is a component of Confluent Platform, and the API supports many interactions with your cluster, including producing and consuming messages and accessing cluster metadata such as the set of topics and mapping of partitions to brokers.

With Confluent for Kubernetes (CFK), you can configure and deploy two RESTful interfaces to Kafka:

Embedded Admin REST API: This is built into each broker and makes available RESTful API for administrative functionality for Kafka and MDS.
Standalone REST Proxy: This is a separate component service that makes available RESTful API for producing to, consuming from, and administering Kafka.

For more information on REST Proxy, see Confluent REST APIs.

Like with other Confluent Platform components, CFK provides a custom resource definition (CRD) for the REST Proxy component. This supports:

Configuration overrides for server (in the kafka-rest.properties file), JVM, and log4j
Specification of Kubernetes scheduling mechanisms
Pod annotations
Node labels
Tolerations

REST Proxy in CFK works with Confluent Platform 6.2.0 and later.

Configure REST Proxy

The following is a REST Proxy custom resource (CR) example:

apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestProxy
metadata:
  name: kafkarestproxy
  namespace: confluent
spec:
  authentication:
    basic:
      secretRef: kafkarestproxy-users
    type: basic
  dependencies:
    schemaRegistry:
      authentication:
        basic:
          secretRef: sr-client-basic
        type: basic
      tls:
        enabled: true
      url: https://schemaregistry.operator.svc.cluster.local:8081
  image:
    application: confluentinc/cp-kafka-rest:7.4.5
    init: confluentinc/confluent-init-container:2.6.5
    pullSecretRef:
      - confluent-registry
  replicas: 3
  tls:
    autoGeneratedCerts: true

Configure dependencies

REST Proxy has a direct dependency on the following Confluent Platform components:

Kafka
Metadata Service (MDS) if RBAC is enabled
Schema Registry
Monitoring interceptors if monitoring production and consumption in Confluent Control Center (Legacy)

Explore the REST Proxy custom resource definition (CRD) with the following command for the dependency properties you need to set. See Use kubectl to examine Confluent Platform CRDs for more information about the command.

kubectl explain kafkarestproxy.spec

Kafka

Run the following command iteratively to see the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.kafka

MDS

If RBAC is enabled, MDS needs to be configured for REST Proxy.

Run the following command iteratively to see the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.mds

Schema Registry

REST Proxy has a dependency on Schema Registry if it’s being used to manage schemas.

Run the following command iteratively to see the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.schemaRegistry

Set the following Schema Registry URL explicitly in the REST Proxy CR to be able to access Schema Registry:

dependencies:
  schemaRegistry:
    url:

Interceptor

REST Proxy supports interceptor configurations as part of Java new producer and consumer settings. This can be configured in the interceptor dependencies spec.

Run the following command iteratively to see the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.interceptor

Configure external access

CFK supports the following external access mechanisms to the REST Proxy endpoint:

Session affinity for consumers

Kafka consumers are stateful, and requests from each consumer instance must be routed to the same rest proxy instance. When you configure external access to REST Proxy for a consumer, enable session affinity in the REST Proxy custom resource (CR).

See Load balancer, Nodeports, and Openshift Routes for configuration details of session affinity.

Configure security

Authentication

Authenticate clients with REST Proxy

To authenticate clients with REST Proxy, you can configure one of the following in the REST Proxy CR:

Authenticate REST Proxy with Kafka

To authenticate REST Proxy with Kafka brokers, you can configure one of the following in the REST Proxy CR in the Kafka dependency section:

No authentication
SASL/PLAIN
mTLS

Authenticate REST Proxy with MDS

For REST Proxy to authenticate with MDS when RBAC is enabled, configure authentication of REST Proxy with MDS as described in Bearer authentication.

Authenticate REST Proxy with Schema Registry

To authenticate REST Proxy with Schema Registry to configure and manage schemas when Schema Registry is secured, configure authentication in the REST Proxy CR in the Schema Registry dependency section.

Authorization

REST Proxy supports the following authorization methods:

None (default)
Confluent Role Based Access Control (RBAC) authorization

See Configure internal role binding for a sample scenario of setting up internal role bindings.

Network encryption

REST Proxy supports the following network encryption methods:

No encryption (default)
TLS encryption

See Define SAN for defining REST Proxy SAN values.

Configure Confluent license

REST Proxy is under a subcription license.

Configure the license in the REST Proxy CR as described in Update Confluent Platform License in Confluent for Kubernetes.

Manage REST Proxy

You can scale your REST Proxy deployment up or down based on your needs with multiple instances of REST Proxy. See REST Proxy Deployment and Load Balancing for more information.

For other post-deployment tasks, such as upgrades or a rolling restart of the cluster and how those actions affect associated producers and consumers, refer to Rest Proxy Post Deployment.