Configure and Manage Confluent REST Proxy
The Confluent REST Proxy provides a RESTful interface to a Kafka cluster. The Standalone REST Proxy is a component of Confluent Platform, and the API supports many interactions with your cluster, including producing and consuming messages and accessing cluster metadata such as the set of topics and mapping of partitions to brokers.
With Confluent for Kubernetes (CFK), you can configure and deploy two RESTful interfaces to Kafka:
- Embedded Admin REST API
This is built into each broker and makes available RESTful API for administrative functionality for Kafka and MDS.
- Standalone REST Proxy
This is a separate component service that makes available RESTful API for producing to, consuming from, and administering Kafka.
For more information on REST Proxy, see Confluent REST APIs.
Like with other Confluent Platform components, CFK provides a custom resource definition (CRD) for the REST Proxy component. This supports:
Configuration overrides for server (in the
kafka-rest.properties
file), JVM, and log4jSpecification of Kubernetes scheduling mechanisms
Pod annotations
Node labels
Tolerations
REST Proxy in CFK works with Confluent Platform 6.2.0 and later.
Configure REST Proxy
The following is a REST Proxy custom resource (CR) example:
apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestProxy
metadata:
name: kafkarestproxy
namespace: confluent
spec:
authentication:
basic:
secretRef: kafkarestproxy-users
type: basic
dependencies:
schemaRegistry:
authentication:
basic:
secretRef: sr-client-basic
type: basic
tls:
enabled: true
url: https://schemaregistry.operator.svc.cluster.local:8081
image:
application: confluentinc/cp-kafka-rest:|cp_release|
init: confluentinc/confluent-init-container:2.6.5
pullSecretRef:
- confluent-registry
replicas: 3
tls:
autoGeneratedCerts: true
Configure dependencies
REST Proxy has a direct dependency on the following Confluent Platform components:
Kafka
Metadata Service (MDS) if RBAC is enabled
Schema Registry
Monitoring interceptors if monitoring production and consumption in Confluent Control Center (Legacy)
Explore the REST Proxy custom resource definition (CRD) with the following command for the dependency properties you need to set. See Use kubectl to examine Confluent Platform CRDs for more information about the command.
kubectl explain kafkarestproxy.spec
Kafka
Run the following command iteratively to see the available dependency properties in the REST Proxy CR:
kubectl explain kafkarestproxy.spec.dependencies.kafka
MDS
If RBAC is enabled, MDS needs to be configured for REST Proxy.
Run the following command iteratively to see the available dependency properties in the REST Proxy CR:
kubectl explain kafkarestproxy.spec.dependencies.mds
Schema Registry
REST Proxy has a dependency on Schema Registry if it’s being used to manage schemas.
Run the following command iteratively to see the available dependency properties in the REST Proxy CR:
kubectl explain kafkarestproxy.spec.dependencies.schemaRegistry
Set the following Schema Registry URL explicitly in the REST Proxy CR to be able to access Schema Registry:
dependencies:
schemaRegistry:
url:
Interceptor
REST Proxy supports interceptor configurations as part of Java new producer and consumer settings. This can be configured in the interceptor dependencies spec.
Run the following command iteratively to see the available dependency properties in the REST Proxy CR:
kubectl explain kafkarestproxy.spec.dependencies.interceptor
Configure external access
CFK supports the following external access mechanisms to the REST Proxy endpoint:
Session affinity for consumers
Kafka consumers are stateful, and requests from each consumer instance must be routed to the same rest proxy instance. When you configure external access to REST Proxy for a consumer, enable session affinity in the REST Proxy custom resource (CR).
See Load balancer, Nodeports, and Openshift Routes for configuration details of session affinity.
Configure security
Authentication
Authenticate clients with REST Proxy
To authenticate clients with REST Proxy, you can configure one of the following in the REST Proxy CR:
No authentication
Authenticate REST Proxy with Kafka
To authenticate REST Proxy with Kafka brokers, you can configure one of the following in the REST Proxy CR in the Kafka dependency section:
No authentication
Authenticate REST Proxy with MDS
For REST Proxy to authenticate with MDS when RBAC is enabled, configure authentication of REST Proxy with MDS as described in Bearer authentication.
Authenticate REST Proxy with Schema Registry
To authenticate REST Proxy with Schema Registry to configure and manage schemas when Schema Registry is secured, configure authentication in the REST Proxy CR in the Schema Registry dependency section.
Network encryption
REST Proxy supports the following network encryption methods:
No encryption (default)
-
See Define SAN for defining REST Proxy SAN values.
Configure Confluent license
REST Proxy is under a subcription license.
Configure the license in the REST Proxy CR as described in Update Confluent Platform License in Confluent for Kubernetes.
Manage REST Proxy
You can scale your REST Proxy deployment up or down based on your needs with multiple instances of REST Proxy. See REST Proxy Deployment and Load Balancing for more information.
For other post-deployment tasks, such as upgrades or a rolling restart of the cluster and how those actions affect associated producers and consumers, refer to Rest Proxy Post Deployment.