Advanced Configuration Options for Confluent Platform

This topic describes a few of the advanced configuration options Confluent for Kubernetes (CFK) provides for Confluent components.

Configure Kafka Connect & ksqlDB using Confluent Cloud

Confluent for Kubernetes supports deploying and managing Connect, ksqlDB, and Confluent Control Center to connect to a Confluent Cloud Kafka and Schema Registry.

For a an illustrative walkthrough on configuring this, see the tutorial for connecting to Confluent Cloud.

Provide custom service account

Each Confluent component pod deployed by Confluent for Kubernetes has an associated Kubernetes Service Account. These service accounts serve several functions in Confluent for Kubernetes. For example, to deploy Confluent Docker images from a private Docker image registry, you can associate Docker registry credentials to the service account associated with your Confluent components.

If you do not specify a service account, the default service account is assigned to the component pod in the same namespace.

To provide a custom service account, set serviceAccountName in the component custom resource configuration:

spec:
  podTemplate:
    serviceAccountName:

Configuration overrides

You can override default configuration parameters of a Confluent Platform components in the component custom resource (CR) as below:

spec:
  configOverrides:
    server: []
    jvm: []
    log4j: []

Refer to Configuration Reference for configuration parameters used in Confluent Platform components.

The example below for Kafka has the following effects:

  • Enables automatic Topic creation (disabled by default).
  • Enables the Cluster Linking feature (disabled by default).
  • Sets a few JVM flags related to memory management.
  • Changes the log-level to DEBUG from the default of INFO.
spec:
  configOverrides:
    server:
      - auto.create.topics.enable=true
      - confluent.cluster.link.enable=true
    jvm:
      - "-Xmx6g"
      - "-XX:MetaspaceSize=96m"
      - "-XX:+UseG1GC"
    log4j:
      - log4j.rootLogger=DEBUG, stdout

Apply the changes with the kubectl apply command.