Configure REST Proxy

Configure and deploy REST Proxy for Confluent Platform with Confluent for Kubernetes (CFK). The Confluent REST Proxy provides a RESTful interface to an Apache Kafka® cluster. The Standalone REST Proxy is a component of Confluent Platform. Its API supports many interactions with your cluster, including producing and consuming messages, and accessing cluster metadata such as topics and partition-to-broker mappings.

With CFK, you can configure and deploy two RESTful interfaces to Kafka:

Embedded Admin REST API: This is built into each broker and provides a RESTful API for administrative functionality for Kafka and Metadata Service (MDS).
Standalone REST Proxy: This is a separate component service that provides a RESTful API for producing to, consuming from, and administering Kafka.

For more information on REST Proxy, see Confluent REST APIs.

As with other Confluent Platform components, CFK provides a custom resource definition (CRD) for the REST Proxy component. This supports:

Configuration overrides for server (in the kafka-rest.properties file), Java Virtual Machine (JVM), Log4j, and Log4j 2
Specification of Kubernetes scheduling mechanisms
Pod annotations
Node labels
Tolerations

REST Proxy in CFK works with Confluent Platform 6.2.0 and later.

REST Proxy custom resource example

The following is a REST Proxy custom resource (CR) example:

apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestProxy
metadata:
  name: kafkarestproxy
  namespace: confluent
spec:
  authentication:
    basic:
      secretRef: kafkarestproxy-users
    type: basic
  dependencies:
    schemaRegistry:
      authentication:
        basic:
          secretRef: sr-client-basic
        type: basic
      tls:
        enabled: true
      url: https://schemaregistry.operator.svc.cluster.local:8081
  image:
    application: confluentinc/cp-kafka-rest:8.3.0
    init: confluentinc/confluent-init-container:3.4.0
    pullSecretRef:
      - confluent-registry
  replicas: 3
  tls:
    autoGeneratedCerts: true

Configure dependencies

REST Proxy has a direct dependency on the following Confluent Platform components:

Kafka
Metadata Service (MDS) if role-based access control (RBAC) is enabled
Schema Registry
Monitoring interceptors if monitoring production and consumption in Confluent Control Center (Legacy)

Important

Interceptors and Control Center (Legacy) are deprecated and removed in Confluent Platform 8.0 and later.

Explore the REST Proxy custom resource definition (CRD) with the following command for the dependency properties you need to set. See Use kubectl to examine Confluent Platform CRDs for more information about the command.

kubectl explain kafkarestproxy.spec

Configure the Kafka dependency

REST Proxy requires a connection to your Kafka cluster to produce and consume messages.

Run the following command iteratively to view the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.kafka

To point REST Proxy at a Logical Kafka Cluster (LKC) on a Confluent Private Cloud multi-tenant Physical Kafka Cluster (PKC) instead of the PKC itself, set the following in spec.dependencies.kafka.discovery:

kind: LogicalKafkaCluster. Set it explicitly. If you omit it, CFK looks for a Kafka CR and a LogicalKafkaCluster CR that have the same name, and fails if both exist.
name: The name of the LogicalKafkaCluster CR.
namespace: The namespace of the LogicalKafkaCluster CR.
secretRef: The name of the Secret that holds the credentials that REST Proxy uses to authenticate to the PKC’s in-cluster token listener. CFK can’t resolve the LKC without it.

REST Proxy resolves topic names that are LKC-local, so deploy a dedicated REST Proxy for each LKC rather than sharing one across LKCs. Unlike an LKC-scoped Connect instance, CFK doesn’t register an LKC-scoped REST Proxy instance in the cluster registry, so Control Center can’t discover it through the cluster registry.

For more information, see Deploy Ecosystem Components on a Multi-Tenant PKC.

Configure the MDS dependency

If RBAC is enabled, configure MDS for REST Proxy.

Run the following command iteratively to view the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.mds

Configure the Schema Registry dependency

REST Proxy has a dependency on Schema Registry if it’s being used to manage schemas.

Run the following command iteratively to view the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.schemaRegistry

Set the following Schema Registry URL explicitly in the REST Proxy CR to be able to access Schema Registry:

dependencies:
  schemaRegistry:
    url:

Configure interceptor settings

REST Proxy supports interceptor configurations as part of new Java producer and consumer settings. Configure this in the interceptor dependencies spec.

Important

Interceptors deprecated and removed in Confluent Platform 8.0 and later.

Run the following command iteratively to view the available dependency properties in the REST Proxy CR:

kubectl explain kafkarestproxy.spec.dependencies.interceptor

Configure external access

CFK supports the following external access mechanisms to the REST Proxy endpoint:

Session affinity for consumers

Kafka consumers are stateful, and requests from each consumer instance must be routed to the same REST Proxy instance. When you configure external access to REST Proxy for a consumer, enable session affinity in the REST Proxy custom resource (CR).

For configuration details of session affinity, see Load balancer, NodePorts, and OpenShift Routes.

Configure security

Configure authentication, authorization, and network encryption for REST Proxy as described in the following sections.

Configure authentication for REST Proxy

Authenticate clients with REST Proxy

To authenticate clients with REST Proxy, you can configure one of the following in the REST Proxy CR:

Authenticate REST Proxy with Kafka

To authenticate REST Proxy with Kafka brokers, you can configure one of the following in the REST Proxy CR in the Kafka dependency section:

No authentication
SASL/PLAIN
mTLS
OAuth

Authenticate REST Proxy with MDS

For REST Proxy to authenticate with MDS when RBAC is enabled, configure authentication of REST Proxy with MDS as described in:

Bearer
OAuth

Authenticate REST Proxy with Schema Registry

To authenticate REST Proxy with Schema Registry to configure and manage schemas when Schema Registry is secured, configure authentication in the REST Proxy CR in the Schema Registry dependency section.

See Configure Authentication for Confluent Platform Components for the client-side authentication mechanism you want to use.

Configure authorization for REST Proxy

REST Proxy supports the following authorization methods:

None (default)
Confluent Role-Based Access Control (RBAC) authorization

See Configure internal role binding for a sample scenario of setting up internal role bindings.

Network encryption

REST Proxy supports the following network encryption methods:

No encryption (default)
TLS encryption

For details on defining REST Proxy Subject Alternative Name (SAN) values, see Define SAN.

Configure Confluent license

REST Proxy is under a subscription license.

Configure the license in the REST Proxy CR as described in Manage Confluent Platform License in Confluent for Kubernetes.

Manage REST Proxy

You can scale your REST Proxy deployment up or down based on your needs with multiple instances of REST Proxy. See REST Proxy Deployment and Load Balancing for more information.

For other post-deployment tasks, such as upgrades or a rolling restart of the cluster and how those actions affect associated producers and consumers, refer to REST Proxy Post Deployment.