Connect Standalone ksqlDB Server Instances to Confluent Control Center on Confluent Platform

You can develop event streaming applications by using the ksqlDB user interface provided by Confluent Control Center. In Control Center, you can create Apache Kafka® topics and develop persistent queries in the ksqlDB query editor. When you install Confluent Platform, ksqlDB Server is integrated with Control Center by default, and you can configure Control Center to interact with other ksqlDB Server instances that run on separate hosts.

Screenshot of the ksqlDB Create Stream interface in Confluent Control Center.

Configuration Settings for ksqlDB and Control Center

Set up the integration between ksqlDB and Control Center by assigning properties in the ksqlDB Server and Control Center configuration files.

  • By default, the ksqlDB Server configuration file is installed at CONFLUENT_HOME/etc/ksqldb/ksql-server.properties`.
  • By default, the Control Center configuration file is installed at CONFLUENT_HOME/etc/confluent-control-center/control-center.properties.

You can set up ksqlDB Server to communicate securely with other components in Confluent Platform. For more information, see Configure Security for ksqlDB.

Note

To run ksqlDB queries from Control Center, both ksqlDB and Control Center must be running on either HTTP or HTTPS.

Network Configuration for ksqlDB and Control Center

Use these configuration settings to set up network connectivity between ksqlDB and Control Center.

  • In the ksqlDB Server configuration file, set the listeners property to the IP address of the REST API endpoint for ksqlDB Server. Typical values are http://0.0.0.0:8088 and http://localhost:8088.
  • In the Control Center configuration file, set the confluent.controlcenter.ksql.<ksql-cluster-name>.url property to the hostnames and listener ports for the ksqlDB cluster specified by <ksql-cluster-name>. This setting specifies how Control Center communicates with ksqlDB Server for regular HTTP requests. For more information, see ksqlDB settings.
  • If ksqlDB Server communicates over an internal DNS that is not externally resolvable or routeable, set the confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url property in the Control Center configuration file. This setting specifies how the browser communicates with ksqlDB Server for websocket requests. For more information, see ksqlDB settings.

When ksqlDB Server and Confluent Control Center run on the same host, you can use the default configuration defined by Confluent Platform setup.

When ksqlDB and Control Center run on different hosts

If ksqlDB Server and Control Center run on different hosts, you must specify a configuration that ensures ksqlDB Server and Control Center can communicate. This is necessary when ksqlDB Server and Control Center:

  • Run in separate containers.
  • Run in separate virtual machines.
  • Communicate over a virtual private network (VPN).
  • Use different URLs, for example when the ksqlDB Server host publishes a public URL that’s different from the private URL for ksqlDB Server.

Note

When ksqlDB and Control Center communicate over a virtual private network (VPN), Control Center proxies your queries, but query results stream directly from ksqlDB Server back to your browser without going through Control Center. Over a VPN, the advertised URL isn’t localhost. Instead, it’s the hostname of the remote server.

Assign the following configuration properties to integrate ksqlDB Server with Control Center when they run on separate hosts.

ksqlDB Server Configuration

In the ksqlDB Server configuration file, set listeners to bind to all interfaces:

listeners=http://0.0.0.0:8088

Configure ksqlDB to accept cross-origin requests from the Confluent Control Center UI by adding the following CORS settings to the ksqlDB Server configuration file.

access.control.allow.origin=*
access.control.allow.methods=GET,POST,HEAD
access.control.allow.headers=X-Requested-With,Content-Type,Accept,Origin,Authorization

Control Center Configuration

In the Control Center configuration file, set confluent.controlcenter.ksql.<ksql-cluster-name>.url to a list of URLs for the ksqlDB Server hosts, which must be reachable from the host that Control Center is installed on. Replace <ksql-cluster-name> with the name that Control Center uses to identify the ksqlDB cluster.

confluent.controlcenter.ksql.<ksql-cluster-name>.url=<internally-resolvable-hostname1>, <internally-resolvable-hostname2>, ...

Also, set confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url to the public IP addresses published by the ksqlDB Server hosts, which must be a list of URLs that the browser can resolve through externally available DNS.

confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url=<externally-resolvable-hostname1>, <externally-resolvable-hostname2>, ...

The Control Center configuration must match the ksqlDB Server listeners values.

Use the curl command to check whether these URLs are reachable. Depending on your deployment, you may need to check from two different hosts:

  • Check from the host where Control Center is running, which is relevant for the confluent.controlcenter.ksql.<ksql-cluster-name>.url setting.
  • Check from the host where the browser is running, which is relevant for the confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url setting.

On both hosts, run the following command to confirm that the ksqlDB Server cluster is reachable. The hostname value is one of the hosts in the listed in the confluent.controlcenter.ksql.<ksql-cluster-name>.url and confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url configuration settings.

curl http://<hostname>:8088/info \
{"KsqlServerInfo":{"version":"7.6.4","kafkaClusterId":"<ksql-cluster-name>","ksqlServiceId":"default_"}}%

Note

You must specify the ports in the ksqlDB URL settings. For example, if the public URL is http://ksql-server-677739697.us-east-1.elb.amazonaws.com:80, be sure to include port 80, or the Control Center connection to ksqlDB Server will fail.

Check Network Connectivity Between ksqlDB and Confluent Control Center-short

Use a web browser to check the configuration of an advertised URL. Make sure that your browser can reach the info endpoint at http://<ksql.advertised.url>/info. If the configuration is wrong, and the browser can’t resolve the URL of the ksqlDB Server host, you’ll receive an error: Websocket error when communicating with <ksql.advertised.url>.

Check ksqlDB Server Network Binding

If Control Center doesn’t connect with your ksqlDB Server instance, check the network binding on the ksqlDB Server host:

sudo netstat -plnt|grep $(ps -ef|grep KsqlServerMain|grep -v grep|awk '')

If your ksqlDB server is bound to localhost only, your output should resemble:

tcp6 0 0 127.0.0.1:8088 :::* LISTEN 64383/java
tcp6 0 0 :::34791 :::* LISTEN 64383/java

If 0.0.0.0 isn’t listed, ksqlDB Server isn’t accepting external connections. In the ksql-server.properties file, set listeners=http://0.0.0.0:8088 and restart ksqlDB Server.