# Connect on z/OS for Confluent Platform Confluent’s certified version of Kafka Connect for [IBM’s z/OS operating system](https://www.ibm.com/it-infrastructure/z/zos) allows you to run certified premium connectors on the z/OS operating system. For more information about the premium connectors certified on z/OS, see the [Certified Connectors on z/OS](#connect-on-zos-certified-connectors) section in this page. #### NOTE Confluent’s Kafka Connect for z/OS requires an additional subscription for the premium connectors which are certified on IBM z/OS. ## Connect on z/OS and Apache Kafka Compatibility Connect on z/OS shares the same release and compatibility matrix with Kafka as the Confluent Platform. For more information, see the [Confluent Platform and Apache Kafka compatibility](../installation/versions-interoperability.md#cp-ak-compatibility) matrix. ## Connect on z/OS and Kafka Broker Compatibility Kafka Connect workers that are included in Confluent Platform 3.2 and later are compatible with any Kafka broker that is included in Confluent Platform 3.0 or later. ## Certified Connectors on z/OS The following connectors are certified on IBM z/OS and require a premium subscription. | Connector | Supported Version | |----------------------------------------------------------------------------------------------------------------------|---------------------| | [Confluent’s IBM MQ Source connector for z/OS](https://www.confluent.io/hub/confluentinc/kafka-connect-ibmmq-zos) | 12.1.1 | | [Confluent’s IBM MQ Sink connector for z/OS](https://www.confluent.io/hub/confluentinc/kafka-connect-ibmmq-zos-sink) | 2.1.2 | ## Certified z/OS Version Connect on z/OS is certified with z/OS version 2, release 4. For more details, see [IBM z/OS Version 2 Release 4](https://www.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_ca/0/877/ENUSZP19-0410/index.html) ## Get Started With Connect on z/OS This section helps you get started with Connect on z/OS. Before you begin, ensure you meet all the following [requirements](#connect-on-zos-prerequisites). ### Prerequisites - Download the [IBM MQ Source and Sink connectors for z/OS](#connect-on-zos-certified-connectors) from Confluent Marketplace. - Download the Connect on [z/OS tar file](https://packages.confluent.io/archive/8.1/confluent-connect-8.1.0-zOS.tar.gz). - Access to [z Systems Integrated Information Processor (zIIP)](https://www.ibm.com/products/z-integrated-information-processor). Ensure you have SSH and SFTP access enabled on the z/OS environment. You will need this to upload the Connect TAR file for z/OS. - Java compatibility with z/OS. For more details, see the [IBM Support](https://www.ibm.com/support/pages/java-sdk-products-zos) documentation. - [z/OS UNIX (USS)](https://www.ibm.com/docs/en/zos/2.3.0?topic=zos-unix-system-services). - Install [Bash](https://www.rocketsoftware.com/platforms/ibm-z/bash-zos). The user performing the following steps must have access to the Bash command with privileges to run scripts. You can install Bash from [Rocket Software](https://www.rocketsoftware.com/platforms/ibm-z/bash-zos) or [Anaconda](https://anaconda.org/zoss-appdev/bash/files). - Ensure a dedicated user in USS has a bash shell and permissions to read, write, and execute the untarred connectors downloaded from Confluent Marketplace. #### NOTE - Confluent’s IBM MQ Source and Sink connectors will access IBM MQ in bindings mode by default. Bindings mode is more secure than client mode. - The default port for Kafka Connect in distributed mode is 8082. You must ensure this is open so the connector can communicate. To specify a different port, set the `rest.port` configuration property in the connector’s configuration properties file. - The default port for Kafka Connect in standalone mode is 8083. ### Quick start This quick start shows you how to integrate mainframe data with Kafka. Use the following steps to set up Confluent’s IBM MQ connectors on Confluent’s Kafka Connect certified for z/OS. ![Architecture Diagram: Mainframe Offloading](connect/images/connect-zos.png) #### Step 1: Download and extract the TAR file 1. Download and extract the TAR file containing Confluent’s Kafka Connect worker node for z/OS from [https://packages.confluent.io/archive/8.1/confluent-connect-8.1.0-zOS.tar.gz](https://packages.confluent.io/archive/8.1/confluent-connect-8.1.0-zOS.tar.gz). 2. Run the `gunzip` command to remove the compression: ```bash gunzip confluent-8.1.0-zOS.tar.gz ``` This will result in a `confluent-connect-8.1.0-zOS.tar` TAR file. 3. Upload the TAR file to the z/OS machine. 4. Untar the TAR file by running the following command: ```bash tar -xvf confluent-connect-8.1.0-zOS.tar ``` #### Step 2: Convert files to EBCDIC encoding 1. Log into the z/OS machine with a SSH terminal: ```bash ssh -p username@ ``` 2. Navigate to the z/OS folder: ```bash cd confluent-connect-zos ``` 3. Convert the following files to [EBCDIC encoding](https://www.ibm.com/docs/en/zos-basic-skills?topic=mainframe-ebcdic-character-set) by running the [iconv command](https://www.ibm.com/docs/en/zos/2.3.0?topic=scd-iconv-convert-characters-from-one-code-set-another). For example: ```text iconv -f ISO8859-1 -t IBM-1047 ./connect-standalone.properties > connect-standalone2.properties ``` - `bin/connect-standalone.sh` - `bin/kafka-run-class.sh` - `./config/connect-standalone.properties` - `/` - `/` #### Step 3: Configure logging and output Additional configuration is necessary to prevent log display and truncation issues on z/OS. ##### Configure Log4j2 charset To prevent unreadable or garbage characters in logs, add the `charset` parameter to the Console appender `PatternLayout` in the `connect-log4j2.yaml` configuration file: ```yaml Appenders: Console: name: STDOUT PatternLayout: pattern: "${logPattern}" charset: "IBM-1047" ``` ##### Configure JCL for output redirection To prevent log truncation and ensure long-term log availability, configure your Job Control Language (JCL) to redirect `STDOUT` and `STDERR` to z/OS UNIX (USS) files instead of the default JES Spool. ```text //STDOUT DD PATH='/u/kafconn/logs/kafconnect.out', // PATHOPTS=(OWRONLY,OCREAT,OAPPEND), // PATHMODE=(SIRWXU), // FILEDATA=TEXT //STDERR DD PATH='/u/kafconn/logs/kafconnect.err', // PATHOPTS=(OWRONLY,OCREAT,OAPPEND), // PATHMODE=(SIRWXU), // FILEDATA=TEXT ``` #### NOTE Adjust the path `/u/kafconn/logs/` to match your environment’s directory structure. #### Step 4: Set path to dependent libraries Export the following environmental variable: ```bash export LIBPATH=$LIBPATH: ``` This is a path to the bindings shared object–it is important to run the connector in bindings mode so the connector can find all the native libraries. #### NOTE If you are setting up the IBM MQ Source and IBM MQ Sink connectors, ensure you follow the steps listed in the [Client Libraries](https://docs.confluent.io/kafka-connectors/ibmmq-source/current/overview.html#client-libraries) section. #### Step 5: Download the IBM MQ Connectors Download the [IBM MQ Source and Sink connectors for z/OS](#connect-on-zos-certified-connectors) from Confluent Marketplace. #### Step 6: Run the connector worker node Run the Connect standalone worker node by running the following command: ```bash ./bin/connect-standalone.sh ./config/connect-standalone.properties ./ibmmqconnector.properties ``` For more details on the connector configurations, see the [IBM MQ Source](https://docs.confluent.io/kafka-connectors/ibmmq-source/current/overview.html#ibm-mq-source-connector-for-cp) or the [IBM MQ Sink](https://docs.confluent.io/kafka-connectors/ibmmq-sink/current/overview.html) connector documentation.