Confluent Telemetry Reporter

The Confluent Telemetry Reporter is a plugin that runs inside each Confluent Platform service to push metadata about the service to Confluent. Telemetry Reporter enables product features based on the metadata, like Health+. Data is sent over HTTP using an encrypted connection.

For a list of available metrics, see Telemetry Reporter Metrics.

Important

• If Self-Balancing is enabled on the cluster, Telemetry Reporter is enabled and runs by default as a requirement of Self-Balancing Clusters. The Self-Balancing algorithm consumes from the Telemetry Reporter internal topic _confluent-telemetry-metrics.
• If Self-Balancing is not enabled on the cluster, the Telemetry Reporter can be fully disabled using the confluent.reporters.telemetry.auto.enable=false property setting in the server configuration file at $CONFLUENT_HOME/etc/kafka/server.properties.
• No data is sent to Confluent Cloud if you set confluent.telemetry.enabled to false.

Installation

Confluent Telemetry Reporter is installed as part of the full Confluent Platform installation. If you choose to install packages individually, ensure that Telemetry Reporter is installed by using yum install confluent-telemetry or the equivalent package manager command for your environment.

Enable telemetry reporting to Confluent Cloud

Prerequisites

• Sign up for Confluent Health+
• Internet connectivity either directly or through a proxy
• Confluent Platform 6.0 or higher

Create a Cloud API key and secret

The following steps show how to create an API key and secret to authenticate with Confluent Cloud.

Cloud API keys are distinct from Kafka cluster API keys. For Telemetry Reporter, you must use a Cloud API Key, not a Kafka cluster API key. Using a cluster key causes an authentication error.

Confluent CLIConfluent Cloud Console

Use the confluent api-key create command to create a Cloud API key.

confluent login
confluent api-key create --resource cloud

It may take a couple of minutes for the API key to be ready. Save the API key and secret. The secret is not retrievable later. Your output should resemble.

+---------+------------------------------------------------------------------+
| API Key | UZ7UB0BEMH7ZZL4T                                                 |
| Secret  | PcFU4EUD0bbSFaz3jnx2jhYmcuPVZShdowwhOkrYUhusFONqQtfU0tMtT8gkZSde |
+---------+------------------------------------------------------------------+

Configure Health+

You have a number of options for configuring Health+.

Confluent Control CenterAnsible Playbooks for Confluent PlatformConfluent for KubernetesCustom deployment

1. Open Confluent Control Center in a browser. The default URL is http://localhost:9021/.
2. On the Home page, click your cluster.
3. In the navigation menu, click Brokers, and scroll to find the list of brokers.
4. Click a broker, and in the details page, click Configuration.
5. Click Health+ and Edit settings. Copy the API key and secret that you created previously into the corresponding fields, and enter true for confluent.telemetry.enabled.
6. Click Save.
7. Repeat the previous steps for all of your brokers.

Configuration with a proxy

Confluent Telemetry Reporter supports routing telemetry data through an authenticated HTTP proxy. In these environments, set the following parameters in the server configuration file to enable telemetry data egress through a proxy.

confluent.telemetry.proxy.url=http://<proxy_server_ip>:<proxy_server_port>
confluent.telemetry.proxy.username=<proxy-username>
confluent.telemetry.proxy.password=<proxy-password>

What data is sent and how is it used?

When you have Confluent Telemetry Reporter enabled for a specific Confluent Platform component, Confluent collects metrics about the Confluent Platform component that include:

• Runtime performance metrics
• The Kafka version
• The Confluent Platform version
• A unique identifier that’s valid for the applicable Confluent Platform component
• A unique identifier for the Kafka cluster
• A unique identifier for your organization.

The data is sent using an encrypted connection to Confluent servers once per minute, by default. The resulting data is used by Confluent to offer Health+ to its customers and to maintain and improve Confluent Products and Services.

For this data to be sent, enable the transmission by setting configuration flags in every Confluent Platform component. You can stop sending data at any time by removing these configuration flags.

Dynamic Configuration

Note

Currently, dynamic configuration applies only to Kafka brokers.

To understand how to apply dynamic configs for your cluster, see Kafka Dynamic Configurations.

Enable Telemetry without a restart

If you’re using Confluent Server, you can enable the Telemetry Reporter without a restart by using the following command. Replace <api_key> and <api_secret> with the key and secret you generated previously.

$CONFLUENT_HOME/bin/kafka-configs --bootstrap-server localhost:9092 \
--alter --entity-type brokers --entity-default \
--add-config confluent.telemetry.enabled=true, \
confluent.telemetry.api.key=<api_key>, \
confluent.telemetry.api.secret=<api_secret>

Supported dynamic configuration properties

• confluent.telemetry.enabled
• confluent.telemetry.api.key
• confluent.telemetry.api.secret
• confluent.telemetry.proxy.url
• confluent.telemetry.proxy.username
• confluent.telemetry.proxy.password