Confluent Telemetry Reporter¶

Overview¶

The Confluent Telemetry Reporter is a plugin that runs inside each Confluent Platform service to push metadata about the service to Confluent. Telemetry Reporter enables product features based on the metadata, like Proactive Support. Data is sent over HTTP using an encrypted connection.

Important

If Self-Balancing is enabled on the cluster, the Telemetry Reporter is enabled and running by default as a requirement of Self-Balancing Clusters. The Self-Balancing algorithm consumes from the Telemetry Reporter internal topic _confluent-telemetry-metrics.
if Self-Balancing is not enabled on the cluster, the Telemetry Reporter can be fully disabled using the property setting confluent.reporters.telemetry.auto.enable=false in $CONFLUENT_HOME/etc/kafka/server.properties.
No data is sent to Confluent Cloud if you set confluent.telemetry.enabled to false.

For the list of available metrics, see Telemetry Reporter Metrics.

Installation¶

Confluent Telemetry Reporter is installed as part of the full Confluent Platform installation. If you choose to install packages individually, ensure that Telemetry Reporter is installed by using yum install confluent-telemetry.

Enable telemetry reporting to Confluent Cloud¶

Prerequisites

Access to Confluent Cloud
Internet connectivity either directly or through a proxy
Confluent Platform 6.0 or higher

Create a Cloud API key to authenticate with Confluent Cloud. For example:

ccloud login
ccloud api-key create --resource cloud

Note

You must use a Cloud API Key. Using the Cluster API Key that is used to communicate with Kafka causes an authentication error.

Configuration with Ansible Playbooks for Confluent Platform¶

If you use Ansible Playbooks for Confluent Platform to manage your Confluent Platform installation, add the following configuration overrides to all Confluent Platform roles and deploy the changes.

telemetry_enabled: true
telemetry_api_key: <api-key>
telemetry_api_secret: <api-secret>

Configuration with Confluent Operator 1.x¶

When you use Confluent Operator to manage Confluent Platform, you can enable telemetry as described in this section.

Enable telemetry at the global or component level by setting the following in the Operator configuration file ($VALUES_FILE):
```
telemetry:
  enabled: true                            --- [1]
  secretRef: <confluent-telemetry-secret>  --- [2]
  proxy: false                             --- [3]
```
- [1] Set enabled: true to enable telemetry.
- [2] Set secretRef to the Kubernetes secret that contains the key telemetry. You create this secret (noted as <confluent-telemetry-secret>) as described in the last step below.
- [3] Set proxy: true to enable routing telemetry data through an authenticated HTTP proxy. If proxy is set to true, then the secret in telemetry.secretRef must additionally contain proxy configuration.
Create the telemetry.txt file, and add the following in the file:
```
apiKey=<ccloud-api-key>
apiSecret=<ccloud-api-secret>
```
If you set telemetry.proxy: true in the first step above, add the following in the telemetry.txt file:
```
proxyUrl=<proxyUrl>
proxyUsername=<username>
proxyPassword=<password>
```
Create a Kubernetes secret with the name you provided in telemetry.secreteRef in the first step ([2]). <confluent-telemetry-secret> is used as an example in this section.
```
kubectl --namespace <namespace> create secret generic \
   <confluent-telemetry-secret> \
   --from-file=telemetry=<path>/telemetry.txt
```

Custom Deployments Configuration¶

For all Confluent Platform components other than Confluent Server, enable telemetry reporting by adding the following lines to the configuration file for the service and restarting the service to deploy the new configuration. Replace <API_KEY> and <API_SECRET> with the key and secret generated above.

metric.reporters=io.confluent.telemetry.reporter.TelemetryReporter
confluent.telemetry.enabled=true
confluent.telemetry.api.key=<API_KEY>
confluent.telemetry.api.secret=<API_SECRET>

For Confluent Server, the metric.reporters configuration is not needed. Add the following lines to the Confluent Server configuration file and restart Confluent Server. If restarting Confluent Server is undesirable, you can add these configurations by using dynamic configuration and the kafka-config CLI.

confluent.telemetry.enabled=true
confluent.telemetry.api.key=<API_KEY>
confluent.telemetry.api.secret=<API_SECRET>

Configuration with a proxy¶

Confluent Telemetry Reporter supports routing telemetry data through an authenticated HTTP proxy. In these environments, set the followingparameters to have telemetry data egress through the proxy.

confluent.telemetry.proxy.url=http://<proxy_server_ip>:<proxy_server_port>
confluent.telemetry.proxy.username=<proxy-username>
confluent.telemetry.proxy.password=<proxy-password>

What data is sent and how it is used¶

When a customer has Confluent Telemetry Reporter enabled for a specific Confluent Platform component, Confluent collects metrics about the Confluent Platform component that include runtime performance metrics, the Kafka version, the Confluent Platform version, a unique identifier that’s valid for the applicable Confluent Platform component, a unique identifier for the Kafka cluster, and a unique identifier for the Customer organization. The data is sent using an encrypted connection to Confluent servers once per minute by default. The resulting data is used by Confluent to offer Proactive Support to its customers and to maintain and improve Confluent Products and Services.For this data to be sent, enable the transmission by setting configuration flags in every Confluent Platform component. You can stop sending data at any time by removing these configuration flags.

Dynamic Configuration¶

Note

Currently, dynamic configuration applies only to Kafka brokers.

To understand how to apply dynamic configs for your cluster, see Kafka Dynamic Configurations.

Enable Telemetry without a restart¶

If you’re using Confluent Server, you can enable the Telemetry Reporter without a restart by using the following command. Replace <API_KEY> and <API_SECRET> with the key and secret generated above.

./bin/kafka-configs --bootstrap-server localhost:9092 --alter --entity-type brokers --entity-default --add-config confluent.telemetry.enabled=true,confluent.telemetry.api.key=<API_KEY>,confluent.telemetry.api.secret=<API_SECRET>

Supported dynamic configuration properties¶

confluent.telemetry.enabled
confluent.telemetry.api.key
confluent.telemetry.api.secret
confluent.telemetry.proxy.url
confluent.telemetry.proxy.username
confluent.telemetry.proxy.password