.. _co-management: ======================== Managing the deployment ======================== The following sections provide information about managing the deployed |cp| cluster and operating environment. .. include:: includes/upgrade-note-helm-cp.rst .. _co-license-key: Adding a license key -------------------- You can use |co-long| and |c3| for a 30-day trial period without a license key. After 30 days, |co| and |c3-short| require license keys. .. note:: You add the license key to your ``.yaml`` file or to a custom override YAML file, if applicable. The YAML file is read when you run the helm upgrade command to apply the license. ---------------- Operator License ---------------- The sample YAML file ``helm/confluent-operator/charts/operator/values.yaml`` contains the license section below. Add this section to the Operator block in the ``.yaml`` file or your custom override YAML file. :: ## License Key for Operator ## licenseKey: "" If you are adding your license key to a running deployment, run ``helm upgrade`` to activate the license. The example below assumes you are using the ``.yaml`` file to update your configuration. :: helm upgrade \ -f ./providers/.yaml \ --set operator.enabled=true \ ./confluent-operator If you are a subscriber, please contact Confluent Support at support@confluent.io for more information. ------------ |c3| License ------------ The sample yaml file ``helm/confluent-operator/charts/controlcenter/values.yaml`` contains the license section below. Add this section to the Control Center block in the ``.yaml`` file or your custom override yaml file. :: ## C3 License information ## license: "" If you are adding your license key to a running deployment, run ``helm upgrade`` to activate the license. The example below assumes you are using the ``.yaml`` file to update your configuration. :: helm upgrade \ -f ./providers/.yaml \ --set controlcenter.enabled=true \ ./confluent-operator If you are a subscriber, please contact Confluent Support at support@confluent.io for more information. .. _co-delete-a-cluster: Deleting components -------------------- Uninstall a component release from the cluster. .. sourcecode:: bash helm uninstall --namespace .. include:: includes/delete-cluster.rst .. _co-configuration-modification: Modifying Component Configurations ----------------------------------- Refer to the following two sections for additional information about |cp| component modifications. .. _co-update-component-config: -------------------------------- Default Component Modifications -------------------------------- The ``.yaml`` contains default configuration parameters. The ``values.yaml`` file contains the defaults along with additional parameters you can add to your configuration. The ``values.yaml`` files also contain detailed comments that describe each configuration parameter and how to use it. The table below lists each of the ``values.yaml`` files and their location. .. csv-table:: :header: "Component", "Chart Name", "values.yaml path" :widths: 15, 15, 50 "**Operator**", "operator", "helm/confluent-operator/charts/operator/values.yaml" "**Manager**", "manager", "helm/confluent-operator/charts/manager/values.yaml" "**Kafka**", "kafka", "helm/confluent-operator/charts/kafka/values.yaml" "**ZooKeeper**", "zookeeper", "helm/confluent-operator/charts/zookeeper/values.yaml" "**Connect**", "connect", "helm/confluent-operator/charts/connect/values.yaml" "**Schema Registry**", "schemaregistry", "helm/confluent-operator/charts/schemaregistry/values.yaml" "**Control Center**", "controlcenter", "helm/confluent-operator/charts/controlcenter/values.yaml" "**Replicator**", "replicator", "helm/confluent-operator/charts/replicator/values.yaml" "**KSQL**", "ksql", "helm/confluent-operator/charts/ksql/values.yaml" .. important:: You should not modify a component ``values.yaml`` file. When you need to use or modify a component configuration parameter, add it to or change it in the ``.yaml`` file. The ``.yaml`` overrides other yaml files at installation and when you upgrade a component configuration. Complete the following steps to make component configuration changes: #. Find the configuration parameter block in the ``values.yaml`` file that you want to use. #. Copy the configuration parameter into the correct location in the ``.yaml`` file and make the changes you require. #. Enter the following upgrade command: :: helm upgrade -f \ ./providers/.yaml \ --set .enabled=true \ \ ./confluent-operator For example, to change a |ak| configuration parameter, you enter the following upgrade command after saving your configuration changes in the ``.yaml`` file. :: helm upgrade -f \ ./providers/.yaml \ --set kafka.enabled=true \ kafka \ ./confluent-operator .. _co-config-overrides: -------------------------------------- |ak| and |zk| Configuration Overrides -------------------------------------- You can override default |ak| and |zk| configuration parameters using the following ``values.yaml`` configuration parameters: :: configOverrides: server: [] jvm: [] You can only change settings for configuration parameters that are key/value pairs. Configuration parameters with only a key can't be modified using the override. Refer to the following Confluent documents for configuration parameters used in |ak| and |zk|: * |ak|: :ref:`cp-config-brokers` * |zk|: :ref:`zk-prod-config` For example, if you want to change the default *false* parameter for ``auto.create.topics.enable``, you add the following configuration block to your ``.yaml`` file. :: configOverrides: server: - auto.create.topics.enable=true jvm: [] Then, run the upgrade command. For example, if you are adding this override for |ak| you enter the following command: :: helm upgrade -f \ ./providers/.yaml \ --set kafka.enabled=true \ kafka \ ./confluent-operator .. _co-schema-validation: ----------------- Schema Validation ----------------- You use a configuration override in |ak| to set up schema validation. The following example shows HTTP endpoint override for an example release name ``schemaregistry`` on namespace ``operator``. :: configOverrides: server: - confluent.schema.registry.url=http://schemaregistry.operator.svc.cluster.local:8081 If |sr| is deployed using a secure HTTPS endpoint, use the following configuration override: :: configOverrides: server: - confluent.schema.registry.url=https://:8081 OR (using the same example names as above): :: configOverrides: server: - confluent.schema.registry.url=http://schemaregistry.operator.svc.cluster.local:9081 .. note:: You can view the ```` name by running ``helm status ``. See :ref:`schema_validation` in the |sr| documentation for additional details. .. _co-metric-reporter: Monitoring |ak| metrics ------------------------ The configuration parameter ``metricReporter`` is provided in the |ak| ``values.yaml`` file. If you add the parameters below to the ``provider.yaml`` file, messages are published to the bootstrap endpoint every 30000 milliseconds (30 seconds). .. sourcecode:: bash metricReporter: enabled: true ## Period (millisecond) the report should use to publish messages to bootstrapEndpoint ## publishMs: 30000 ## ReplicationFactor, if empty, the value is based on the replicas count ## replicationFactor: "" tls: enabled: false ## If true, inter-communication will be encrypted if TLS is enabled. The bootstrapEndpoint will have FQDN name. ## If false, the security setting is configured to use either SASL_PLAINTEXT or PLAINTEXT internal: false authentication: type: "" ## If above tls.internal is true, configure with Kafka bootstrap DNS configuration on port 9092 e.g .:9092 ## If above tls.internal is false, configure with Kafka service name on port 9071 eg. :9071 or FQDN name of Kafka serivce name e.g ..svc.cluster.local:9071 bootstrapEndpoint: "" .. important:: If metricReporter is enabled with a bootstrapEndpoint as a DNS name, the DNS name must be resolved successfully or the |ak| pod will crash (CrashLoopBackOff). If the boostrapEndpoint is using an internal address (like ``kafka:9071``), this is not an issue. For additional security information, see the comments in the ``values.yaml`` file. Data balancing --------------- -------- Scale Up -------- Complete data balancing when a |ak| cluster scale-up is triggered after an update or upgrade, or when a |ak| resource is scaled up. #. On your local machine, use `kubectl exec `_ to start a bash session on one of the pods in the cluster. The example uses the default pod name ``kafka-0`` on a |ak| cluster using the default name ``kafka``. :: kubectl -n operator exec -it kafka-0 bash #. Create a ``config.properties`` properties file using **cat**. You can get ```` and ```` from yaml files by entering the following command: :: kubectl -n operator describe kafka | grep "internalClient"`` The following shows the ``config.properties`` to connect to the broker on internal port `9071` using SASL_PLAINTEXT. :: cat << EOF > config.properties confluent.license= confluent.rebalancer.metrics.sasl.mechanism=PLAIN confluent.rebalancer.metrics.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="" password=""; confluent.rebalancer.metrics.bootstrap.servers=kafka:9071 confluent.rebalancer.metrics.security.protocol=SASL_PLAINTEXT EOF The following shows TLS (server or client authentication): :: cat << EOF > config.properties confluent.license= confluent.rebalancer.metrics.bootstrap.servers=kafka:9071 confluent.rebalancer.metrics.security.protocol=PLAINTEXT EOF #. Enter the following command to rebalance. The endpoint should be in the format ``[zookeeper.name]:2181/[kafka.name]-[namespace]``. The example shows the |zk| cluster name ``zookeeper``, |ak| cluster name ``kafka``, and the namespace ``operator``. The |ak| bootstrap endpoint should be in the format ``[kafka.name]:9071``. :: confluent-rebalancer execute \ --bootstrap-server brokerhost:brokerport \ --metrics-bootstrap-server kafka:9071 \ --throttle 10000000 --verbose \ —-config-file config.properties #. Verify that the rebalance was successful. :: confluent-rebalancer status --zookeeper zookeeper:2181/kafka-operator ---------- Scale Down ---------- Scaling down |ak| clusters is not supported in the current version of |co-long|. .. _co-useful-cli-commands: Common CLI commands -------------------- The following provides common commands that you may find useful when managing the cluster. .. _co-useful-helm-commands: --------------- Helm commands --------------- .. include:: includes/helm-commands.rst .. _co-useful-k8s-commands: ----------------------- kubectl and oc commands ----------------------- .. include:: includes/kubectl-commands.rst Testing the deployment ----------------------- Complete the following steps to test and validate your deployment. ------------------- Internal validation ------------------- .. include:: includes/internal-validation.rst ------------------- External validation ------------------- .. include:: includes/external-validation.rst