.. _solace-source-connector: Solace Source Connector for |cp| ================================ The |kconnect-long| Solace Source Connector is used to move messages from a `Solace PubSub+ `_ cluster to |ak-tm|. Messages are consumed from the Solace broker using the configured :ref:`message selectors ` and written to a single |ak| topic. A :ref:`connect_transforms_supported` can be used to route messages to multiple |ak| topics. Solace PubSub+ uses the Solace Message Format (SMF) protocol, a proprietary binary message format, for client and message broker communications. To ensure compatibility with the Solace Source Connector, messages should be published to Solace using SMF or they may not have a ``MessageID`` (required for JMS). For more information, see the Solace documentation on `Message Components `_. The connector currently supports consuming JMS `TextMessage `_ and `BytesMessage `_ but not `ObjectMessage `_ or `StreamMessage `_. .. note:: If you are required to use the Java Naming and Directory Interface™ (JNDI) to connect to Solace, there is a general :ref:`JMS Source Connector ` available that uses a JNDI-based mechanism to connect to the JMS broker. .. _solace_source_connector_prereqs: Prerequisites ------------- The following are required to run the |kconnect-long| Solace Source Connector: * |ak| Broker: |cp| 3.3.0 or above, or |ak| 0.11.0 or above * |kconnect|: |cp| 4.1.0 or above, or |ak| 1.1.0 or above (requires header support in |kconnect|) * Solace cluster with JMS 1.1 support * ``com.solacesystems:sol-jms`` Client Library (See :ref:`installing_solace_client_jar`) * Java 1.8 .. _solace_source_connector_install: Install the Solace Source Connector ----------------------------------- .. include:: ../../includes/connector-install.rst .. include:: ../../includes/connector-install-hub.rst .. codewithvars:: bash confluent-hub install confluentinc/kafka-connect-solace-source:latest .. include:: ../../includes/connector-install-version.rst .. codewithvars:: bash confluent-hub install confluentinc/kafka-connect-solace-source:1.0.0-preview ------------------------------ Install the connector manually ------------------------------ `Download and extract the ZIP file `__ for your connector and then follow the manual connector installation :ref:`instructions `. .. _solace_source_connector_license: License ------- .. include:: ../../includes/enterprise-license.rst See :ref:`solace-source-connector-license-config` for license properties and :ref:`solace_source_license-topic-configuration` for information about the license topic. Configuration Properties ------------------------ For a complete list of configuration properties for this connector, see :ref:`solace_source_connector_config`. .. _solace_client_lib: Solace Client Library ---------------------- The |kconnect-long| Solace Source Connector does not come with the Solace JMS client library. If you are running a multi-node |kconnect| cluster, the connector and Solace JMS client JAR must be installed on **every** |kconnect| worker in the cluster. See below for details. ---------------------------------------- Installing the Solace JMS Client Library ---------------------------------------- This connector relies on a provided ``com.solacesystems:sol-jms`` client JAR distributed by Solace. The connector **will fail to create a connection to Solace** if you have not installed the JAR on each |kconnect| worker node. The installation steps are: #. Download the `Solace JMS API `_. Additional versions are available on `Maven `_. #. Unzip the download and copy **only** the ``lib/sol-jms-{version}.jar`` file into the ``share/java/kafka-connect-solace-source`` directory of your |cp| installation **on each worker node**. If downloading the library from Maven, you do not need to unzip anything as the jar file is the only artifact downloaded. #. Restart all of the |kconnect| worker nodes. .. note:: The ``share/java/kafka-connect-solace-source`` directory mentioned above is for |cp|. If you are using a different installation, find the location of the Confluent Solace source connector JAR files and place the ``sol-jms`` JAR file into the same directory. .. include:: ../../includes/jms-message-schemas.rst .. _solace_source_connector_quickstart: Quick Start ----------- This quick start uses the Solace Source Connector to consume records from a Solace PubSub+ Standard broker and send them to |ak|. #. Install the connector through the `Confluent Hub Client `_. .. codewithvars:: bash # run from your Confluent Platform installation directory confluent-hub install confluentinc/kafka-connect-solace-source:latest #. :ref:`Install the Solace JMS Client Library `. #. Start the |cp|. .. include:: ../../../includes/cli-new.rst .. codewithvars:: bash |confluent_start| #. Start a Solace PubSub+ Standard docker container. .. codewithvars:: bash docker run -d --name "solace" \ -p 8080:8080 -p 55555:55555 -p 9000:9000 \ --shm-size=1000000000 \ --tmpfs /dev/shm \ --ulimit nofile=2448:38048 \ -e username_admin_globalaccesslevel=admin \ -e username_admin_password=admin \ solace/solace-pubsub-standard:9.1.0.77 #. Once the Solace docker container has started, navigate to the `Solace UI `_ and configure a ``connector-quickstart`` queue in the ``Default`` Message VPN. #. Publish messages to the Solace queue using the REST endpoint. .. codewithvars:: bash curl -X POST -d "m1" http://localhost:9000/Queue/connector-quickstart -H "Content-Type: text/plain" -H "Solace-Message-ID: 1000" # repeat the above command to send additional messages (change the Solace-Message-ID header on each message) #. Create a ``solace-source.json`` file with the following contents: .. codewithvars:: json { "name": "SolaceSourceConnector", "config": { "connector.class": "io.confluent.connect.solace.SolaceSourceConnector", "tasks.max": "1", "kafka.topic": "from-solace-messages", "solace.host": "smf://localhost:55555", "solace.username": "admin", "solace.password": "admin", "jms.destination.type": "queue", "jms.destination.name": "connector-quickstart", "key.converter": "org.apache.kafka.connect.storage.StringConverter", "value.converter": "io.confluent.connect.avro.AvroConverter", "value.converter.schema.registry.url": "http://localhost:8081", "confluent.topic.bootstrap.servers": "localhost:9092", "confluent.topic.replication.factor": "1" } } #. Load the Solace Source Connector. .. include:: ../../../includes/confluent-local-consume-limit.rst .. codewithvars:: bash |confluent_load| solace|dash| -d solace-source.json #. Confirm that the connector is in a ``RUNNING`` state. .. codewithvars:: bash |confluent_status| SolaceSourceConnector #. Confirm the messages were delivered to the ``from-solace-messages`` topic in |ak|. .. codewithvars:: bash kafka-avro-console-consumer --bootstrap-server localhost:9092 --property schema.registry.url=http://localhost:8081 --topic from-solace-messages --from-beginning Additional Documentation ------------------------ .. toctree:: :maxdepth: 1 connector_config changelog