Important
You are viewing documentation for an older version of Confluent Platform. For the latest, click here.
Connecting Kafka REST Proxy to Confluent Cloud¶
You may configure a local Kafka REST proxy to produce to and consume from a Kafka topic in a Kafka cluster in Confluent Cloud.
To connect Kafka REST to Confluent Cloud, you must download the Confluent Platform tarball and then start Kafka REST by using a customized properties file.
- Prerequisites
- Access to Confluent Cloud
- Confluent Cloud CLI installed
Download Confluent Platform and extract the contents.
Create a topic named
rest-proxy-test
by using the Confluent Cloud CLI:$ ccloud topic create --partitions 4 rest-proxy-test
Create a properties file.
Find the client settings for your cluster by clicking Client config from the Confluent Cloud web interface and copy the configuration.
They should look like the following:
ssl.endpoint.identification.algorithm=https sasl.mechanism=PLAIN request.timeout.ms=20000 bootstrap.servers=<BOOTSTRAP_SERVERS> retry.backoff.ms=500 sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<API_KEY>" password="<API_SECRET>"; security.protocol=SASL_SSL
Create a properties file named
ccloud-kafka-rest.properties
.$ cd <path-to-confluent> $ cat ccloud-kafka-rest.properties
Paste the client config contents and add the
client.
prefix to each of security properties. For example:id=kafka-rest-with-ccloud bootstrap.servers=<BOOTSTRAP_SERVERS> client.sasl.mechanism=PLAIN client.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \ username="<API_KEY>" password="<API_SECRET>"; client.security.protocol=SASL_SSL client.ssl.endpoint.identification.algorithm=https # consumer only properties must be prefixed with consumer. consumer.retry.backoff.ms=600 consumer.request.timeout.ms=25000 # producer only properties must be prefixed with producer. producer.acks=1 # admin client only properties must be prefixed with admin. admin.request.timeout.ms=50000 # uncomment and set correct value if using with schema registry #schema.registry.url=http://localhost:8081
Producers, consumers, and admin client will share the
client.
properties. To specify properties only for the producer or consumer, use the following table:
Start the Kafka REST proxy.
$ ./bin/kafka-rest-start ccloud-kafka-rest.properties
Make REST calls using v2 of the REST API (e.g.
application/vnd.kafka.v2+json
) because v2 has no ZooKeeper dependency. Using v1 of the API (e.g.application/vnd.kafka.v1+json
) will not work because v1 has a ZooKeeper dependency that does not work with Confluent Cloud.