Confluent Telemetry Reporter¶

Overview¶

The Confluent Telemetry Reporter is a plugin that runs inside each Confluent Platform service to push metadata about the service to Confluent. Telemetry Reporter enables product features based on the metadata, like Proactive Support. Data is sent over HTTP using an encrypted connection.

Important

If Self-Balancing is enabled on the cluster, the Telemetry Reporter is enabled and running by default as a requirement of Self-Balancing Clusters. The Self-Balancing algorithm consumes from the Telemetry Reporter internal topic _confluent-telemetry-metrics.
if Self-Balancing is not enabled on the cluster, the Telemetry Reporter can be fully disabled using the property setting confluent.reporters.telemetry.auto.enable=false in $CONFLUENT_HOME/etc/kafka/server.properties.
No data is sent to Confluent Cloud if you set confluent.telemetry.enabled to false.

For the list of available metrics, see Telemetry Reporter Metrics.

Installation¶

Confluent Telemetry Reporter is installed as part of the full Confluent Platform installation. If you choose to install packages individually, ensure that Telemetry Reporter is installed by using yum install confluent-telemetry.

Enable telemetry reporting to Confluent Cloud¶

Prerequisites

Access to Confluent Cloud
Internet connectivity either directly or through a proxy
Confluent Platform 6.0 or higher

Create a Cloud API key to authenticate with Confluent Cloud. For example:

ccloud login
ccloud api-key create --resource cloud

Note

You must use a Cloud API Key. Using the Cluster API Key that is used to communicate with Kafka causes an authentication error.

Configuration with Ansible¶

If you use Ansible Playbooks for Confluent Platform to manage your Confluent Platform installation, add the following configuration overrides to all Confluent Platform roles and deploy the changes.

telemetry_enabled: true
telemetry_api_key: <api-key>
telemetry_api_secret: <api-secret>

Configuration with Operator¶

Enable telemetry at either global or component level by setting telemetry.enabled=true and providing telemetry.secretRef=<k8s-secret-name>, where <k8s-secret-name> is a Kubernetes secret named secretRef. It must contain the key telemetry with base64-encoded values.

apiKey: <ccloud-api-key>
apiSecret: <ccloud-api-secret>

If proxy is set to true, then the secret in telemetry.secretRef must additionally contain proxy configuration.

proxyUrl: <proxyUrl>
proxyUsername: <username>
proxyPassword: <password>

Below is an example global configuration.

telemetry:
 enabled: true
 secretRef: <k8s-secret-name>
 proxy: false

Custom Deployments Configuration¶

For all Confluent Platform components other than Confluent Server, enable telemetry reporting by adding the following lines to the configuration file for the service and restarting the service to deploy the new configuration. Replace <API_KEY> and <API_SECRET> with the key and secret generated above.

metric.reporters=io.confluent.telemetry.reporter.TelemetryReporter
confluent.telemetry.enabled=true
confluent.telemetry.api.key=<API_KEY>
confluent.telemetry.api.secret=<API_SECRET>

For Confluent Server, the metric.reporters configuration is not needed. Add the following lines to the Confluent Server configuration file and restart Confluent Server. If restarting Confluent Server is undesirable, you can add these configurations by using dynamic configuration and the kafka-config CLI.

confluent.telemetry.enabled=true
confluent.telemetry.api.key=<API_KEY>
confluent.telemetry.api.secret=<API_SECRET>

Configuration with a proxy¶

Confluent Telemetry Reporter supports routing telemetry data through an authenticated HTTP proxy. In these environments, set the followingparameters to have telemetry data egress through the proxy.

confluent.telemetry.proxy.url=http://<proxy_server_ip>:<proxy_server_port>
confluent.telemetry.proxy.username=<proxy-username>
confluent.telemetry.proxy.password=<proxy-password>

What data is sent and how it is used¶

When a customer has Confluent Telemetry Reporter enabled for a specific Confluent Platform component, Confluent collects metrics about the Confluent Platform component that include runtime performance metrics, the Kafka version, the Confluent Platform version, a unique identifier that’s valid for the applicable Confluent Platform component, a unique identifier for the Kafka cluster, and a unique identifier for the Customer organization. The data is sent using an encrypted connection to Confluent servers once per minute by default. The resulting data is used by Confluent to offer Proactive Support to its customers and to maintain and improve Confluent Products and Services.For this data to be sent, enable the transmission by setting configuration flags in every Confluent Platform component. You can stop sending data at any time by removing these configuration flags.

Dynamic Configuration¶

Note

Currently, dynamic configuration applies only to Kafka brokers.

To understand how to apply dynamic configs for your cluster, see Kafka Dynamic Configurations.

Enable Telemetry without a restart¶

If you’re using Confluent Server, you can enable the Telemetry Reporter without a restart by using the following command. Replace <API_KEY> and <API_SECRET> with the key and secret generated above.

./bin/kafka-configs --bootstrap-server localhost:9092 --alter --entity-type brokers --entity-default --add-config confluent.telemetry.enabled=true,confluent.telemetry.api.key=<API_KEY>,confluent.telemetry.api.secret=<API_SECRET>

Supported dynamic configuration properties¶

confluent.telemetry.enabled
confluent.telemetry.api.key
confluent.telemetry.api.secret
confluent.telemetry.proxy.url
confluent.telemetry.proxy.username
confluent.telemetry.proxy.password