Configure .NET Clients for OAuth/OIDC on Confluent Cloud

Prerequisites

Before you begin, ensure you have the following:

Language and tooling

  • .NET 6+

  • Confluent.Kafka: 2.0.0 or later

  • Confluent Platform: 7.2.1 or later; 7.1.3 or later

Confluent Cloud environment

  • OAuth setup in Confluent Cloud - Configure OAuth authentication

Credentials and identifiers

  • Client ID - Your application’s identifier (like a username)

  • Client secret - Your application’s password for OAuth

  • Token endpoint URL - Where to request access tokens

  • Scopes - Permissions your application needs (for example, kafka:read kafka:write)

  • Cluster ID (lkc-xxxxx) - Your Kafka cluster identifier

  • Identity pool ID (pool-xxxxx) - If using identity pools

Client library

  • Install the latest version with OAuth support:

    dotnet add package Confluent.Kafka

Configure Kafka .NET clients

.NET Kafka clients can authenticate to Confluent Cloud clusters using the OAuth 2.0 protocol. The .NET client uses the Confluent.Kafka library which is based on librdkafka.

Configuration approach

.NET clients use a delegate-based approach for OAuth token retrieval. The client calls your token refresh handler when authentication is needed, passing OAuth configuration parameters for token requests.

Key components

  • OAuth configuration parameters (client ID, secret, endpoint, and so on)

  • .NET delegate for token refresh

  • Confluent Cloud cluster and identity pool IDs

Token refresh handler requirements

  • Must accept a string parameter (OAuth configuration)

  • Must return the access token as a string

  • Should handle errors and implement retry logic

Token refresh handler

The .NET client requires a token refresh handler with the following signature:

public delegate string OAuthTokenRefreshHandler(string oauth_config);

Complete implementation example

Here’s a complete implementation of the OAuth token refresh handler:

using Confluent.Kafka;
using System.Net.Http;
using System.Text.Json;
using System.Threading.Tasks;

// Define a class to hold the logic
public class OauthHandler
{
    private static readonly HttpClient client = new HttpClient();

    // The actual token refresh handler implementation
    public static async Task SetOAuthToken(IClient client, string oauthCfg)
    {
        try
        {
            var config = JsonSerializer.Deserialize<Dictionary<string, string>>(oauthCfg);

            var requestData = new FormUrlEncodedContent(new[]
            {
                new KeyValuePair<string, string>("grant_type", "client_credentials"),
                new KeyValuePair<string, string>("client_id", config["sasl.oauthbearer.client.id"]),
                new KeyValuePair<string, string>("client_secret", config["sasl.oauthbearer.client.secret"]),
                new KeyValuePair<string, string>("scope", config.GetValueOrDefault("sasl.oauthbearer.scope"))
            });

            var response = await client.PostAsync(config["sasl.oauthbearer.token.endpoint.url"], requestData);
            response.EnsureSuccessStatusCode();

            var responseBody = await response.Content.ReadAsStringAsync();
            var tokenData = JsonSerializer.Deserialize<JsonElement>(responseBody);
            var accessToken = tokenData.GetProperty("access_token").GetString();
            var lifetimeMs = tokenData.GetProperty("expires_in").GetInt64() * 1000;

            // Set the token on the client
            client.OAuthBearerSetToken(accessToken, lifetimeMs, config["sasl.oauthbearer.client.id"]);
        }
        catch (Exception ex)
        {
            // Set a token failure error on the client
            client.OAuthBearerSetTokenFailure(ex.ToString());
        }
    }
}

Configuration example

All settings should be placed in a single configuration object:

// All settings should be in a single configuration object.
var producerConfig = new ProducerConfig
{
    BootstrapServers = "your-bootstrap-server:9092",
    SecurityProtocol = SecurityProtocol.SaslSsl,

    // OAuth settings
    SaslMechanism = SaslMechanism.OAuthBearer,
    SaslOauthbearerTokenEndpointUrl = "https://your-oauth-provider.com/oauth2/token",
    SaslOauthbearerClientId = "your-client-id",
    SaslOauthbearerClientSecret = "your-client-secret",
    SaslOauthbearerScope = "kafka:read kafka:write",
    SaslOauthbearerExtensions = "logical_cluster=lkc-xxxxx,identity_pool_id=pool-yyyyy" // Confluent Cloud specific
};

// The .NET client uses an Action for the handler, not a delegate property.
var producer = new ProducerBuilder<string, string>(producerConfig)
    .SetOAuthBearerTokenRefreshHandler(OauthHandler.SetOAuthToken) // Assign the handler here
    .Build();

// For a consumer, the process is the same
var consumerConfig = new ConsumerConfig(producerConfig) // Reuse producer config
{
    GroupId = "your-consumer-group",
    AutoOffsetReset = AutoOffsetReset.Earliest,
    // Note: Consumer-specific properties override producer ones.
};

var consumer = new ConsumerBuilder<string, string>(consumerConfig)
    .SetOAuthBearerTokenRefreshHandler(OauthHandler.SetOAuthToken)
    .Build();

Test your configuration

  1. Enable debug logging to troubleshoot OAuth issues:

    // Add to your configuration
var debugConfig = new ProducerConfig
{
    // ... your OAuth config
    Debug = "security,protocol,broker"
};

  2. Test with a simple producer:

    // Test producer
using var producer = new ProducerBuilder<string, string>(producerConfig).Build();

var message = new Message<string, string>
{
    Key = "test-key",
    Value = "test-message"
};

var deliveryResult = await producer.ProduceAsync("test-topic", message);
Console.WriteLine($"Message delivered to {deliveryResult.Topic} [{deliveryResult.Partition}] - OAuth is working!");

  3. Check for common errors:

    • “SASL authentication failed” - Check your OAuth credentials and endpoint

    • “Invalid token” - Verify your token refresh handler is returning a valid token

    • “Connection timeout” - Check your bootstrap servers and network connectivity

  4. Verify token refresh - The client should automatically refresh tokens when they expire

Troubleshoot .NET OAuth clients

Common issues and solutions for .NET OAuth clients:

Authentication failures

  • Verify client ID and secret are correct

  • Check token endpoint URL is accessible

  • Ensure logical cluster ID is valid

  • Validate identity pool ID if used

Network issues

  • Confirm network connectivity to OAuth provider

  • Check firewall rules allow OAuth traffic

  • Verify SSL certificate validation

Configuration issues

  • Ensure all required parameters are provided

  • Validate token refresh handler signature

  • Check timeout values are reasonable

Debug logging

Enable detailed client logs from the underlying librdkafka library by setting the Debug property in your configuration. This is the most effective way to troubleshoot authentication and network issues.

var producerConfig = new ProducerConfig
{
    // ... your other settings
    Debug = "security,protocol,broker"
};

Common .NET-specific issues

  • Assembly references: Ensure all required NuGet packages are installed

  • Token refresh handler: Verify the handler signature matches requirements

  • Async/await: Use proper async patterns for token refresh operations

  • Memory management: Dispose of HttpClient and other resources properly

  • Thread safety: Ensure OAuth token provider is thread-safe for multi-threaded applications

Related content