<a id="ccloud-integrate-with-3rd-party-monitoring"></a>

# Integrate Confluent Cloud Metrics API with Third-Party Monitoring Tools

Integrating directly with a third-party monitoring tool allows you to monitor
Confluent Cloud alongside the rest of your applications.

Confluent Cloud publishes the full list of available metrics at
[Confluent Cloud Metrics Reference](https://api.telemetry.confluent.cloud/docs/descriptors/datasets/cloud).
All metrics are accessible through the Metrics API `/query` endpoint, and any
metrics marked as exportable are also available through the `/export`
endpoint. If specific metrics from this list do not appear in your third-party
monitoring tool, contact your monitoring vendor to request support for those
metrics.

## Datadog

Datadog provides an integration that connects to Confluent Cloud using a
resource-scoped API key. After you select resources to monitor, Datadog
displays metrics in an out-of-the-box dashboard within minutes. If you
use Datadog, create your Confluent Cloud API key and follow
[the instructions](https://docs.datadoghq.com/integrations/confluent_cloud/)
from Datadog to get started. After configuring the integration, search the
Datadog dashboards for “Confluent Cloud Overview,” the default Confluent Cloud dashboard at
Datadog. Clone the default dashboard so that you can edit it to suit your needs.

Datadog as a third-party integration in Confluent Cloud is an early access feature. For
more information, see
[Manage observability integrations](manage-observability-integrations.md#manage-observability-integrations).

## Dynatrace

Dynatrace provides an extension that connects to Confluent Cloud using a
resource-scoped API key. After you select resources to monitor, Dynatrace
displays metrics in a prebuilt dashboard within minutes. If you use
Dynatrace, create your Confluent Cloud API key and follow
[the instructions](https://www.dynatrace.com/hub/detail/confluent-cloud-kafka/)
to get started.

## Grafana Cloud

Grafana Cloud provides an integration that connects to Confluent Cloud using a
resource-scoped API key. After you select resources to monitor, Grafana
Cloud displays metrics in an out-of-the-box dashboard within minutes.

<a id="grafana-integration"></a>

### Set up Grafana Metrics Endpoint for the latest metrics

The default
[Grafana integration for Confluent Cloud](https://grafana.com/docs/grafana-cloud/integrations/integrations/integration-confluent-cloud/)
doesn’t include metrics launched after the integration’s own last update. Use
Metrics Endpoint in Grafana Cloud to access the latest Confluent Cloud metrics.

A scrape job in Grafana is a configuration that tells Grafana Cloud’s Metrics
Endpoint to collect, or scrape, metrics from a specified source, such as a
Prometheus-compatible endpoint. These jobs define what to monitor, how often
to collect data, and what to do with the data afterward.

After you create a scrape job, no further management is required. Grafana Cloud
automatically handles the scraping of metrics from Confluent Cloud into Grafana Cloud.

#### Considerations

- You must use an API key resource-scoped for resource management to
  communicate with the Metrics API. For more information, see
  [Create an API key to authenticate to the Metrics API](metrics-api.md#ccloud-add-metricsviewer-role-to-new-service-account).
- API keys resource-scoped for Apache Kafka® clusters cause an authentication error.
- This topic contains steps for a third-party tool that might change without
  notice.

#### Get resource IDs for scrape jobs

You must use Confluent Cloud resource IDs to create URLs for the scrape job. For more
information, see [Discover resources and metrics with the Metrics API](metrics-api.md#ccloud-metrics-discovery) and the
[Metrics API reference](https://api.telemetry.confluent.cloud/docs#tag/Version-2/paths/~1v2~1metrics~1%7Bdataset%7D~1export/get).

For example, to scrape metrics of compute pools, you would use the resource ID
for compute pools, which looks like this: `lfcp-examp2`

Use the resource ID to create the scrape job URL:

```text
https://api.telemetry.confluent.cloud/v2/metrics/cloud/export?resource.compute_pool.id=lfcp-examp2
```

#### Create a scrape job

A scrape job configures Grafana Cloud’s Metrics Endpoint to scrape metrics
from a Confluent Cloud resource on a schedule. To create one, add a new connection
in Grafana, configure the **Metrics Endpoint** integration, and enter your
Confluent Cloud API key and secret as Basic authentication credentials, as
described in the following steps.

After you create the scrape job, you can modify it to scrape other resources or
completely delete the scrape job by returning to **Metrics Endpoint**.

Grafana integrations for Confluent Cloud use the API key and secret as a form of basic
authentication. Use the Confluent Cloud API key as the username and the API secret as
the password. To avoid transcription errors, copy the API key and secret from
Confluent Cloud and paste them into the scrape job.

If you encounter an error testing your connection, be sure that you have entered
the right API credentials from your Confluent Cloud account.

![image](images/_monitoring/grafana-metrics-endpoint-config.png)

To create scrape jobs:

1. In your Grafana instance, click **Connections** and then click **Add new
   connection**.
2. In **Search connections**, enter `Metrics Endpoint` and select the
   **Metrics Endpoint** integration.
3. In the **Metrics Endpoint** integration, click **Configuration details**. If
   you already have a scrape job, click **Add new scrape job**.
4. In **Scrape job name**, enter a name for your scrape job.
5. In **Scrape job URL**, enter a URL that contains the resource IDs that need
   to be scraped.
6. In **Scrape interval**, select an appropriate interval for scrapes.
7. In **Types of authentication credentials**, click **Basic**.
8. In **Basic username**, enter your Confluent Cloud API key.
9. In **Basic password**, enter your Confluent Cloud API secret.
10. To test the connection, click **Test Connection**.
11. To save your work, click **Save scrape job**.

#### Create a dashboard

A dashboard visualizes Confluent Cloud metrics that Grafana Cloud scrapes through
the **Metrics Endpoint**.

![image](images/_monitoring/grafana-edit-panel-flink-status.png)

To create dashboards:

1. In your Grafana instance, click **Dashboards** then click **New** and select
   **New dashboard**.
2. Click **Add visualization**.
3. To create a visualization, select a data source, metrics, and filters.

## Prometheus

Prometheus servers can scrape the Confluent Cloud Metrics API directly by making use of
the `export` endpoint. This endpoint returns the single most recent data
point for each metric, for each distinct combination of labels. It returns
this data in the Prometheus exposition or Open Metrics format. For more
information, see
[Export metric values](https://api.telemetry.confluent.cloud/docs?#tag/Version-2/paths/~1v2~1metrics~1{dataset}~1export/get).

## New Relic OpenTelemetry

You can collect metrics about your Confluent Cloud-managed Kafka deployment with the
New Relic OpenTelemetry collector. The collector is a component of OpenTelemetry
that collects, processes, and exports telemetry data to New Relic, or any
observability backend. For more information, see
[Monitoring Confluent Cloud Kafka with OpenTelemetry Collector](https://github.com/newrelic/newrelic-opentelemetry-examples/tree/main/other-examples/collector/confluentcloud).

## Related content

- [Discover resources and metrics with the Metrics API](metrics-api.md#ccloud-metrics-discovery)
- [Metrics API reference](https://api.telemetry.confluent.cloud/docs#tag/Version-2/paths/~1v2~1metrics~1%7Bdataset%7D~1export/get)
- [Grafana integration](https://grafana.com/docs/grafana-cloud/monitor-infrastructure/integrations/integration-reference/integration-confluent-cloud/)
