# Tutorial: Protect Secrets in Confluent Platform ## Overview Confluent Platform secrets allow you to store and manage sensitive information, such as passwords and API tokens. Instead of storing passwords or other sensitive data as cleartext, you can encrypt the data within a configuration file itself. ### Prerequisites * The [Confluent Platform must be installed](../../../installation/overview.md#installation). * The [Confluent CLI](https://docs.confluent.io/confluent-cli/current/installing.html) must be installed. ## Run Tutorial ### Generating the master encryption key based on a passphrase 1. Decide on a master encryption key passphrase. Your passphrase should be longer than the usual password. Choose a phrase you can remember as a string of words. 2. Write the passphrase into a local file (for example, `/path/to/passphrase.txt`) which will be passed into the Confluent CLI. Writing to a file avoids logging history showing the passphrase. 3. Choose the location where the secrets file will reside on your local host. The location shouldn’t be where the Confluent Platform services run. The secrets file will contain encrypted secrets for the master encryption key, data encryption key, and configuration parameters, along with their metadata, such as which cipher was used for encryption. 4. Generate the master encryption key by running the following command: ```text # passphrase: /path/to/passphrase.txt # local-secrets-file: /path/to/secrets.txt confluent secret master-key generate --local-secrets-file /path/to/secrets.txt --passphrase @/path/to/passphrase.txt ``` You should see: ```text Save the master key. It cannot be retrieved later. +------------+----------------------------------------------+ | Master Key | Nf1IL2bmqRdEz2DO//gX2C+4PjF5j8hGXYSu9Na9bao= | +------------+----------------------------------------------+ ``` 5. Save the master key somewhere safe on your computer. 6. Export the key into the environment on the local host and every other host that will have a configuration file with secret protection. For example: ```text export CONFLUENT_SECURITY_MASTER_KEY=Nf1IL2bmqRdEz2DO//gX2C+4PjF5j8hGXYSu9Na9bao= ``` 7. (Optional) To protect the previous environment variable in a production host, you can set the master encryption key at the process level instead of the global machine level. For example, you could set it in the `systemd` overrides for executed processes, restricting the environment directives file to root-only access. ### Encrypting the value of a configuration parameter #### NOTE When you encrypt a configuration parameter value in a file, the file must include the expected `.properties` file extension. Otherwise, the following error message is returned: ```none Error: invalid config file path "client.property" ``` In this section, you encrypt a configuration parameter available in the configuration file example that ships with Confluent Platform. To encrypt the parameter `config.storage.topic` in `$CONFLUENT_HOME/etc/schema-registry/connect-avro-distributed.properties`, complete the following steps: 1. Make a backup of the file `$CONFLUENT_HOME/etc/schema-registry/connect-avro-distributed.properties`, because the CLI currently does in-place modification on the original file. 2. Select the exact path where the secrets file will reside on the remote hosts where the Confluent Platform services run, e.g. `/path/to/secrets-remote.txt`. 3. View the value before encryption. ```shell grep "config\.storage\.topic" connect-avro-distributed.properties ``` Your output should resemble: ```text config.storage.topic=connect-configs ``` 4. Encrypt the field: ```text # remote-secrets-file: /path/to/secrets-remote.txt confluent secret file encrypt --local-secrets-file /path/to/secrets.txt --remote-secrets-file /path/to/secrets-remote.txt --config-file connect-avro-distributed.properties --config config.storage.topic ``` 5. View the value after encryption. ```shell grep "config\.storage\.topic" connect-avro-distributed.properties ``` Your output should resemble: ```text config.storage.topic = ${securepass:/path/to/secrets-remote.txt:connect-avro-distributed.properties/config.storage.topic} ``` 6. Validate that the configuration parameter `config.storage.topic` setting was changed from `connect-configs` to `${securepass:/path/to/secrets-remote.txt:connect-avro-distributed.properties/config.storage.topic}`. This is a tuple that directs the service to use to look up the encrypted value of the file/parameter pair `connect-avro-distributed.properties/config.storage.topic` from the secrets file `/path/to/secrets-remote.txt`. 7. View the contents of the local secrets file `/path/to/secrets.txt`, which should contain the encrypted secret for this file or parameter pair along with the metadata (for example, which cipher was used for encryption): ```shell cat /path/to/secrets.txt ``` You should see: ```text ... connect-avro-distributed.properties/config.storage.topic = ENC[AES/GCM/NoPadding,data:CUpHh5lRDfIfqaL49V3iGw==,iv:vPBmPkctA+yYGVQuOFmQJw==,type:str] ``` ### Decrypting the value of a configuration parameter You can also decrypt the value of a configuration parameter into a file by completing the following steps: 1. Run the following command to decrypt the value into a file: ```shell confluent secret file decrypt --local-secrets-file /path/to/secrets.txt --config-file connect-avro-distributed.properties --output-file decrypted.txt ``` 2. View the file: ```shell cat decrypted.txt ``` You should see the following output: ```text config.storage.topic = connect-configs ``` ### Updating the value of the configuration parameter You might need to update secrets on a regular basis to ensure the secrets don’t get stale. The configuration parameter `config.storage.topic` was originally set to `connect-configs`. If you must change this value in the future, you can update it using the CLI. 1. In the following CLI, pass in a file `/path/to/updated-config-and-value` that has written `config.storage.topic=newTopicName` to avoid logging history showing the new value. ```shell confluent secret file update --local-secrets-file /path/to/secrets.txt --remote-secrets-file /path/to/secrets-remote.txt --config-file connect-avro-distributed.properties --config @/path/to/updated-config-and-value ``` 2. The configuration file `connect-avro-distributed.properties` doesn’t change because it’s a pointer to the secrets file, but the secrets file has a new value for the encrypted value for this file/parameter pair. When running the following command: ```shell cat /path/to/secrets.txt ``` You should see: ```text ... connect-avro-distributed.properties/config.storage.topic = ENC[AES/GCM/NoPadding,data:CblF3k1ieNkFJzlJ51qAAA==,iv:dnZwEAm1rpLyf48pvy/T6w==,type:str] ``` ### Verify on Confluent Platform cluster Run your Confluent Platform cluster and start the modified Connect worker with the encrypted value of `config.storage.topic=newTopicName` by completing the following steps: 1. Start your Kafka cluster (brokers/controllers): ```shell confluent local services kafka start ``` 2. Run the modified Connect worker: ```shell connect-distributed connect-avro-distributed.properties > connect.stdout 2>&1 & ``` 3. List the topics: ```text kafka-topics --bootstrap-server localhost:9092 --list __confluent.support.metrics __consumer_offsets _confluent-metrics connect-offsets connect-statuses newTopicName <<<<<<< ``` ### Going to production So far you’ve learned how to create the master encryption key and encrypt secrets in the configuration files. Confluent recommends you operationalize the workflow by augmenting your orchestration tooling to distribute everything you need for secret protection to work to the destination hosts. These hosts can include Confluent Server brokers, Connect workers, Confluent Schema Registry instances, ksqlDB servers, Confluent Control Center , and more–any service using password encryption. The CLI is flexible to accommodate whatever secret distribution model you prefer. You can either perform the secret generation and configuration modification on each destination host directly, or do it all on a single host and then distribute the encrypted secrets to the destination hosts. Here are four required tasks: 1. Export the master encryption key into the environment on every host that needs a configuration file with secret protection. 2. Distribute the secrets file: copy the secrets file `/path/to/secrets.txt` from the local host on which you have been working to `/path/to/secrets-remote.txt` on the destination hosts. 3. Propagate the necessary configuration file changes: update the configuration file on all hosts so that the configuration parameter now has the tuple for secrets. 4. Restart the services if they were already running. You might also have a requirement to rotate the master encryption key or data encryption key on a regular basis. You can do either of these with the CLI. To rotate the data encryption key only, run the following command: ```shell confluent secret file rotate --data-key --local-secrets-file /path/to/secrets.txt --passphrase @/path/to/passphrase.txt ```