Confluent Developer Marketplace Get started
  1. Home
  2. Cloud
  3. Confluent Cloud Clusters
  4. Connect Confluent Platform Components to Confluent Cloud
  5. Confluent Cloud Gateway
  6. Configure and Deploy Confluent Cloud Gateway
  7. Confluent Cloud Gateway on Docker

Troubleshoot Confluent Gateway

Use this section to collect diagnostics and resolve common issues when running Confluent Gateway with Docker.

Capture Confluent Gateway logs manually

Use Docker commands to inspect the Confluent Gateway container logs.

# View all Confluent Gateway logs
docker logs gateway

# Filter for errors
docker logs gateway | grep -i error

# Filter for license-related messages
docker logs gateway | grep -i license

Begin troubleshooting by inspecting the Confluent Gateway container logs. Most initialization failures, including network timeouts and credential errors, are captured here.

Enable debug logging to view stack traces

Set the global Confluent Gateway log level to DEBUG to capture detailed logs, including stack traces. This helps diagnose complex issues when running Confluent Gateway.

Update your docker-compose.yaml file to include the GATEWAY_ROOT_LOG_LEVEL environment variable:

gateway:
  environment:
    GATEWAY_ROOT_LOG_LEVEL: "DEBUG"

The GATEWAY_ROOT_LOG_LEVEL environment variable controls the global logging level for Confluent Gateway, and the DEBUG log level captures detailed system events, including stack traces.

Known and common issues

Confluent Gateway container exits or restarts repeatedly

Symptoms

  • The Confluent Gateway container terminates after startup.

  • Running the docker ps command shows the container is repeatedly restarting.

  • Running the docker compose up command shows that the service failed to start.

  • Logs contain configuration parsing errors, license validation failures, or stack traces.

Likely causes

  • Invalid or incomplete Confluent Gateway configuration file.

  • Misconfigured environment variables in the docker-compose.yaml file.

Resolution

  1. Inspect the logs for configuration and license errors:

    docker logs gateway
docker logs gateway | grep -i license

    Identify messages indicating failed YAML parsing, unknown fields, or unrecognized license keys.

  2. Verify configuration file mapping. Confirm that the volumes section in your docker-compose.yaml file correctly maps the local configuration file to the expected path in the container.

  3. Verify the configuration file structure:

    • Ensure that the streamingDomains section defines at least one streaming domain and a corresponding bootstrap server.

    • Ensure that the routes section defines at least one valid route definition.

  4. Verify enterprise mode configuration:

    If you are using the enterprise license, confirm that the GATEWAY_LICENSES environment variable contains valid and unexpired license keys.

Clients can’t connect to the Confluent Gateway endpoint

Symptoms

  • Client connections to the Confluent Gateway endpoint either time out or stall indefinitely during metadata requests.

  • Clients can directly connect to the Kafka cluster, but connections fail when routed through the Confluent Gateway.

Likely causes

  • The Confluent Gateway container can’t reach the Kafka bootstrap servers due to DNS, firewall, or network issues.

  • Clients are incorrectly configured to connect to broker addresses directly instead of the Confluent Gateway route endpoint.

  • The routes.endpoint value doesn’t match the host and port published by the Docker container.

  • The bootstrapServerId is mismatched across different sections of the configuration.

Resolution

  1. Verify the bootstrap server configuration:

    Ensure the client’s bootstrap.servers property points to the route endpoint such as gateway.example.com:19092, rather than the broker addresses.

  2. Align bootstrap IDs:

    Ensure that bootstrapServerId defined in the streamingDomains section matches the ID referenced in the routes section.

  3. Confirm Docker port mappings:

    If the route endpoint is defined as gateway.example.com:19092, ensure that your docker-compose.yaml file publishes to that port. For example:

    gateway:
  ports:
    - "19092:19092"

  4. Test Confluent Gateway to Kafka connectivity:

    From within the container network or host where Confluent Gateway runs, test connectivity to the Kafka bootstrap endpoints defined under streamingDomains.kafkaCluster.bootstrapServers[].endpoint.

    Inspect DNS resolution, firewall rules, and VPC or VNet routing.

  5. Review logs for authentication issues:

    Inspect logs for repeated connection attempts or terminated during authentication messages. If these errors occur, verify your authentication and SSL/TLS configuration settings.

Protocol or SSL/TLS mismatch

Symptoms

  • Client logs or Confluent Gateway logs show errors such as frame size exceptions or TLS handshake failures.

  • Clients use SASL_SSL or SSL but the route is configured as plaintext, or vice versa.

Likely causes

  • The client’s security.protocol doesn’t match the configured security for the route.

  • TLS-encrypted traffic is directed to a plaintext route endpoint, or plaintext traffic is directed to an SSL-only route.

Resolution

  1. Confirm route configuration:

    • Inspect the gateway.routes[].security.ssl settings in the Confluent Gateway configuration file.

    • If security.ssl is omitted, the route expects plaintext traffic.

    • If security.ssl is defined, configure the client to use an SSL protocol, such as, SASL_SSL or SSL.

  2. Align client configuration:

    • For unencrypted connections, set the client protocol to PLAINTEXT.

    • When using SASL authentication without TLS, set the client protocol to SASL_PLAINTEXT.

    • For TLS routing, set the client protocol to SSL or SASL_SSL and configure the truststore and keystore as required.

  3. After updating the configuration, restart the Confluent Gateway service to apply configuration changes and verify that client connectivity is restored.

Authentication swap not behaving as expected

Symptoms

  • Clients using authentication swap fail to authenticate with the backend cluster.

  • Logs report errors related to secret store lookups, JAAS configuration failures, or callback handler initialization.

  • Backend brokers report invalid credentials despite the client providing valid local certificates.

Likely causes

  • Route security isn’t configured with auth: swap.

  • The secretStore name referenced in the Confluent Gateway configuration is incorrect or missing in the routes section.

  • Secret store provider type is incorrect, or the credentials required to access the store are misconfigured.

Resolution

  1. Verify route swap configuration:

    Ensure the defined routes perform authentication swapping.

    gateway:
  routes:
    - name: <route-name>
      security:
        auth: swap
        swapConfig: <swap-config>

  2. Avoid hardcoded broker credentials for cluster authentication:

    Don’t embed static broker credentials in the Confluent Gateway configuration. For swap scenarios, credentials must be dynamically resolved from a supported provider, such as Amazon Secrets Manager, HashiCorp Vault, or a local file for dynamic resolution.

  3. Verify your secret store configuration:

    • Provider types are case-sensitive. Verify that you are using the exact casing required. For example, use Azure, not AZURE.

    • Confirm the Confluent Gateway has the necessary permissions to read secrets from the provider.

    • Ensure the Confluent Gateway can reach the secret store’s API endpoint over the network.

Passthrough clients fail authentication after a failover

Symptoms

  • Clients that use SASL/OAUTHBEARER identity passthrough authenticate successfully before a failover, but fail to authenticate with the backing Kafka cluster afterward.

  • Logs report authentication failures that reference an invalid or unknown logical cluster (lkc) extension.

Likely causes

Clients that use SASL/OAUTHBEARER identity passthrough send a logical cluster (lkc) extension to identify the target cluster. After a failover, the active cluster changes, so the client-supplied value becomes invalid and authentication fails.

Resolution

  1. Configure passthroughConfig on the route so that Confluent Gateway injects the correct logical cluster extension on behalf of clients. This keeps client configurations independent of the active cluster.

    gateway:
  routes:
    - name: <route_name>
      security:
        auth: passthrough
        passthroughConfig:
          sasl:
            mechanism: OAUTHBEARER
            extensionHeaders:
              logicalCluster: <target-cluster-id>

    For configuration details, see Configure identity passthrough mode.

  2. Restart the Confluent Gateway service to apply the configuration changes, then verify that clients reconnect and authenticate successfully.

OAuth token or JWKS endpoint not allowed

Symptoms

The Confluent Gateway fails to fetch OAuth tokens or validate JWTs against the identity provider’s JWKS endpoint, and logs report a security restriction similar to: ... is not allowed. Update system property 'org.apache.kafka.sasl.oauthbearer.allowed.urls' to allow https://...

For client-side OAuth, an un-allowlisted JWKS endpoint (jwksEndpointUrl) can also prevent the Confluent Gateway from starting, with an error such as The URL cannot be accessed due to restrictions.

Likely causes

For security purposes, the Confluent Gateway requires an allowlist of outbound OAuth URLs. The configured OAuth token endpoint or JWKS endpoint used by the identity provider isn’t included in the default allowlist.

Resolution

  1. Add the OAuth token endpoint, JWKS endpoint, or both to the GATEWAY_OPTS environment variable in the docker-compose.yaml file:

    gateway:
  environment:
    GATEWAY_OPTS:
      -Dorg.apache.kafka.sasl.oauthbearer.allowed.urls=
      https://login.microsoftonline.com/<tenantId>/oauth2/v2.0/token

  2. Restart the Confluent Gateway service to apply the configuration changes. Check the logs to ensure the not allowed error no longer appears during the authentication handshake.

License and route limit issues

Symptoms

  • The Confluent Gateway fails to initialize additional routes.

  • Logs report warnings on license or route-limit exhaustion.

Likely causes

The Confluent Gateway is running in trial mode, which has a maximum limit of four routes. To configure more routes, an enterprise license is required.

Resolution

  1. Understand license modes:

    • Trial mode (Default): Requires no license key. It is strictly limited to four routes.

    • Enterprise mode: Requires a valid Confluent Enterprise license for the Customer-Managed Confluent Platform for Confluent Cloud subscription. For license scope and coverage, see Confluent Platform licenses.

  2. Apply an enterprise license: To enable more than four routes, add your license key to the GATEWAY_LICENSES environment variable in your docker-compose.yaml file.

  3. Inspect the license status: After restarting the service, verify that the Confluent Gateway has successfully recognized the license. Run docker logs gateway | grep -i license to confirm the license status.