Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Integrate KSQL with Confluent Control Center

You can develop event streaming applications by using the KSQL user interface provided by Confluent Control Center. In Control Center, you can create Apache Kafka® topics and develop persistent queries in the KSQL query editor. When you install Confluent Platform, KSQL Server is integrated with Control Center by default, and you can configure Control Center to interact with other KSQL Server instances that run on separate hosts.

Screenshot of the KSQL Create Stream interface in Confluent Control Center.

Configuration Settings for KSQL and Confluent Control Center

Set up the integration between KSQL and Confluent Control Center by assigning properties in the KSQL Server and Control Center configuration files.

  • By default, the KSQL Server configuration file is installed at <path-to-confluent>/etc/ksql/ksql-server.properties.
  • By default, the Control Center configuration file is installed at <path-to-confluent>/etc/confluent-control-center/control-center.properties.

Secure Communication with KSQL Server

When you use Basic authentication in KSQL, Control Center allows passing credentials as part of the KSQL Server URL in Control Center configuration.

# KSQL cluster URL
confluent.controlcenter.ksql.<name>.url=http://<username>:<password>@localhost:8088

You can set up KSQL Server to communicate securely with other components in Confluent Platform. For more information, see Configuring Security for KSQL.

Network Configuration for KSQL and Control Center

These are the configuration settings that you assign to set up network connectivity between KSQL and Control Center.

  • In the KSQL Server configuration file, set the listeners property to the IP address of the REST API endpoint for KSQL Server. Typical values are http://0.0.0.0:8088 and http://localhost:8088.
  • In the Control Center configuration file, set the confluent.controlcenter.ksql.<name>.url property to the hostnames and listener ports for the KSQL cluster specified by <name>. This setting specifies how Control Center communicates with KSQL Server for regular HTTP requests. For more information, see KSQL Settings.
  • If KSQL Server communicates over an internal DNS that is not externally resolvable or routeable, set the confluent.controlcenter.ksql.<name>.advertised.url property in the Control Center configuration file. This setting specifies how the browser communicates with KSQL Server for websocket requests. For more information, see KSQL Settings.

When KSQL Server and Confluent Control Center run on the same host, you can use the default configuration defined by Confluent Platform setup.

When KSQL and Confluent Control Center run on different hosts

If KSQL Server and Control Center run on different hosts, you must specify a configuration that ensures KSQL Server and Control Center can communicate. This is necessary when KSQL Server and Control Center are deployed in the following situations:

  • KSQL Server and Control Center run in separate containers.
  • They run in separate virtual machines.
  • They communicate over a virtual private network (VPN).
  • The KSQL Server host publishes a public URL that’s different from the private URL for KSQL Server.

Note

When KSQL and Confluent Control Center communicate over a virtual private network (VPN), Control Center proxies your queries, but query results stream directly from KSQL Server back to your browser, without going through Control Center. Over a VPN, the advertised URL isn’t localhost. Instead, it’s the hostname of the remote server.

Assign the following configuration properties to integrate KSQL Server with Control Center when they run on separate hosts.

KSQL Server Configuration

In the KSQL Server configuration file, set listeners to bind to all interfaces:

listeners=http://0.0.0.0:8088

Control Center Configuration

In the Control Center configuration file, set confluent.controlcenter.ksql.<ksql-cluster-name>.url to a list of URLs for the KSQL Server hosts, which must be reachable from the host that Control Center is installed on. Replace <ksql-cluster-name> with the name that Control Center uses to identify the KSQL cluster.

confluent.controlcenter.ksql.<ksql-cluster-name>.url=<internally-resolvable-hostname1>, <internally-resolvable-hostname2>, ...

Also, set confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url to the public IP addresses published by the KSQL Server hosts, which must be a list of URLs that the browser can resolve through externally available DNS.

confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url=<externally-resolvable-hostname1>, <externally-resolvable-hostname2>, ...

The Control Center configuration must match the KSQL Server listeners values.

Use the curl command to check whether these URLs are reachable. Depending on your deployment, you may need to check from two different hosts:

  • Check from the host where Control Center is running, which is relevant for the confluent.controlcenter.ksql.<name>.url setting.
  • Check from the host where the browser is running, which is relevant for the confluent.controlcenter.ksql.<name>.advertised.url setting.

On both hosts, run the following command to confirm that the KSQL Server cluster is reachable. The hostname value is one of the hosts in the listed in the confluent.controlcenter.ksql.<name>.url and confluent.controlcenter.ksql.<name>.advertised.url configuration settings.

curl http://<hostname>:8088/info \
{"KsqlServerInfo":{"version":"5.2.4","kafkaClusterId":"<ksql-cluster-name>","ksqlServiceId":"default_"}}%

Note

You must specify the ports in the KSQL URL settings. For example, if the public URL is http://ksql-server-677739697.us-east-1.elb.amazonaws.com:80, be sure to include port 80, or the Control Center connection to KSQL Server will fail.

Check Network Connectivity Between KSQL and Confluent Control Center

Use a web browser to check the configuration of an advertised URL. Make sure that your browser can reach the info endpoint at http://<ksql.advertised.url>/info. If the configuration is wrong, and the browser can’t resolve the URL of the KSQL Server host, you’ll receive an error: Websocket error when communicating with <ksql.advertised.url>.

Check KSQL Server Network Binding

If Confluent Control Center doesn’t connect with your KSQL Server instance, check the network binding on the KSQL Server host:

sudo netstat -plnt|grep $(ps -ef|grep KsqlServerMain|grep -v grep|awk '')

If your KSQL server is bound to localhost only, your output should resemble:

tcp6 0 0 127.0.0.1:8088 :::* LISTEN 64383/java
tcp6 0 0 :::34791 :::* LISTEN 64383/java

If 0.0.0.0 isn’t listed, KSQL Server isn’t accepting external connections. In the ksql-server.properties file, set listeners=http://0.0.0.0:8088 and restart KSQL Server.