Configure Confluent Platform for Deployment with Confluent for Kubernetes

This topic describes how to work with Kubernetes custom resources (CR) to configure and manage Confluent Platform. The rest of the topics in this section go into detail on configuring specific settings for Confluent components.

Kubernetes Custom Resources for Confluent Platform

Confluent Platform components are deployed to a Kubernetes cluster as custom resources (CRs). Confluent for Kubernetes (CFK) provides the custom resource definitions (CRDs) that were created using Kubernetes API. These CRDs describe how to configure Confluent Platform components. The CRDs are provided in the yaml files in the Confluent for Kubernetes distribution in the crds directory.

For the settings that are not supported in Confluent CRDs, CFK automatically calculates and sets the values based on the relevant factors. For example, the default.replication.factor and min.insync.replicas Kafka configuration values are set based on the broker replica count.

If you need to override these default configuration values that CFK sets, or to provide additional configuration information that is not supported in the CRDs, you can use the configuration overrides feature in CFK to pass the configuration property to Kafka.

For the configuration settings for Confluent Platform components, see Confluent Platform Configuration Reference. For example, supported Kafka configuration settings can be found in Kafka Broker Configurations.

Use kubectl to examine Confluent Platform CRDs

The kubectl explain command describes the fields associated with Kubernetes API resources, including the Confluent Platform CRDs.

  1. To get a list of the Confluent Platform CRDs:

    kubectl api-resources
  2. To describe a specific Confluent Platform CRD:

    kubectl explain <CRD-type>.<fieldName>[.<fieldName>]

    You can drill down the CRD specification by appending a <fieldName> from the kubectl explain output.

    For example:

    kubectl explain kafka.spec.podTemplate

    And to further examine the affinity field listed in the output:

    kubectl explain kafka.spec.podTemplate.affinity

Confluent plugin

Confluent plugin is a composite CLI tool integrated with the Kubernetes ecosystem through the kubectl plugin interface. You can use the plugin to debug and diagnose your Confluent Platform deployed through Confluent for Kubernetes (CFK).

See Confluent plugin for installation instructions.

Use the following command to get the comprehensive usage instruction:

kubectl confluent

The following are some of the example commands:

  • To see all of the external and internal endpoints that you can use to access Confluent components:

    kubectl confluent http-endpoints
  • To convert Operator 1.x to CFK 2.0:

    kubectl confluent migration
  • To check if the current user has cluster-level access:

    kubectl confluent operator --namespaced=false pre-check
  • To check if the current user has the namespaced level access:

    kubectl confluent operator pre-check