Connecting REST Proxy to Confluent Cloud

You may configure a local Confluent REST Proxy to produce to and consume from a Apache Kafka® topic in a Kafka cluster in Confluent Cloud.

To connect REST Proxy to Confluent Cloud, you must download the Confluent Platform tarball and then start REST Proxy by using a customized properties file.

Prerequisites

• Access to Confluent Cloud.
• Confluent Cloud CLI installed and configured.

1. Download Confluent Platform and extract the contents.

2. Create a topic named rest-proxy-test by using the Confluent Cloud CLI:

   ccloud kafka topic create --partitions 4 rest-proxy-test
   

3. Create a properties file.

   1. Find the client settings for your cluster by clicking CLI & client configuration from the Confluent Cloud web interface and copy the configuration.

      

      They should look like the following:

      ssl.endpoint.identification.algorithm=https
      sasl.mechanism=PLAIN
      request.timeout.ms=20000
      bootstrap.servers=<BOOTSTRAP_SERVERS>
      retry.backoff.ms=500
      sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<api-key>" password="<api-secret>";
      security.protocol=SASL_SSL
      

   2. Create a properties file named ccloud-kafka-rest.properties.

      cd <path-to-confluent>
      cat ccloud-kafka-rest.properties
      

   3. Paste the client config contents and add the client. prefix to each of security properties. For example:

      id=kafka-rest-with-ccloud
      bootstrap.servers=<BOOTSTRAP_SERVERS>
      client.sasl.mechanism=PLAIN
      client.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
      username="<api-key>" password="<api-secret>";
      client.security.protocol=SASL_SSL
      client.ssl.endpoint.identification.algorithm=https
      
      # consumer only properties must be prefixed with consumer.
      consumer.retry.backoff.ms=600
      consumer.request.timeout.ms=25000
      
      # producer only properties must be prefixed with producer.
      producer.acks=1
      
      # admin client only properties must be prefixed with admin.
      admin.request.timeout.ms=50000
      
      # uncomment and set correct value if using with schema registry
      #schema.registry.url=http://localhost:8081
      

      Producers, consumers, and admin client will share the client. properties. To specify properties only for the producer or consumer, use the following table:

      Component	Prefix	Example
      Admin Client	admin.	admin.request.timeout.ms
      Consumer	consumer.	consumer.request.timeout.ms
      Producer	producer.	producer.acks

4. (Optional) Add configs for Confluent Cloud Schema Registry per the examples on GitHub at ccloud/examples/template_delta_configs.

5. Start the REST Proxy.

   ./bin/kafka-rest-start ccloud-kafka-rest.properties
   

6. Make REST calls using v2 of the REST API (e.g. application/vnd.kafka.v2+json) because v2 has no ZooKeeper dependency. Using v1 of the API (e.g. application/vnd.kafka.v1+json) will not work because v1 has a ZooKeeper dependency that does not work with Confluent Cloud.