Last active
June 3, 2021 18:10
-
-
Save osowski/a4fb4b8e2724cf8853181abe9cff5af3 to your computer and use it in GitHub Desktop.
Kafka Security Article #1 Snippets
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bootstrap.servers=kafka.{kubernetes-cluster-fully-qualified-domain-name}:443 | |
| sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="{USERNAME}" password="{PASSWORD}"; | |
| security.protocol=SASL_SSL | |
| sasl.mechanism=PLAIN | |
| ssl.truststore.location={/provided/to/you/by/the/kafka/administrator} | |
| ssl.truststore.password={__provided_to_you_by_the_kafka_administrator__} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bootstrap.servers=kafka.{namespace}.svc.cluster.local:9071 | |
| sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="{USERNAME}" password="{PASSWORD}"; | |
| security.protocol=SASL_PLAINTEXT | |
| sasl.mechanism=PLAIN |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bootstrap.servers=broker-0-{cluster-id}.kafka.{service-name}.eventstreams.cloud.ibm.com:9093,…,broker-5-{cluster-id}.kafka.{service-name}.eventstreams.cloud.ibm.com:9093 | |
| sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="{USERNAME}" password="{PASSWORD}"; | |
| security.protocol=SASL_SSL | |
| sasl.mechanism=PLAIN |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| listeners: | |
| - name: plain | |
| port: 9092 | |
| type: internal | |
| tls: false | |
| - name: tls | |
| port: 9093 | |
| type: internal | |
| tls: true | |
| - name: external | |
| port: 9094 | |
| type: route | |
| tls: true | |
| authentication: | |
| type: scram-sha-512 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl https://archive.apache.org/dist/kafka/2.6.1/kafka_2.13-2.6.1.tgz -o kafka.tgz | |
| tar xvf kafka.tgz | |
| cd kafka_2.13–2.6.1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| oc project kafka-security |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| export BOOTSTRAP="$(oc get route my-cluster-kafka-bootstrap -ojsonpath='{.spec.host}'):443" | |
| export CONFIG_FILE=local-config.properties | |
| rm -f ${CONFIG_FILE} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| echo "sasl.jaas.config=$(oc get secret my-user -o json | jq -r '.data["sasl.jaas.config"]' | base64 -d -)" >> ${CONFIG_FILE} | |
| echo "sasl.mechanism=SCRAM-SHA-512" >> ${CONFIG_FILE} | |
| echo "security.protocol=SASL_SSL" >> ${CONFIG_FILE} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| oc extract secret/my-cluster-cluster-ca-cert --confirm --keys=ca.crt | |
| keytool -keystore cluster-ca.jks -import -file ca.crt \ | |
| -storepass my-cluster-password -noprompt | |
| rm -f ca.crt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| echo "ssl.truststore.location=$(pwd)/cluster-ca.jks" >> ${CONFIG_FILE} | |
| echo "ssl.truststore.password=my-cluster-password" >> ${CONFIG_FILE} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bin/kafka-topics.sh --bootstrap-server ${BOOTSTRAP} \ | |
| --command-config ${CONFIG_FILE} --list |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bin/kafka-console-producer.sh --bootstrap-server ${BOOTSTRAP} \ | |
| --producer.config ${CONFIG_FILE} --topic my-topic |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bin/kafka-console-consumer.sh --bootstrap-server ${BOOTSTRAP} \ | |
| --consumer.config ${CONFIG_FILE} --topic my-topic --from-beginning |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl https://archive.apache.org/dist/kafka/2.6.1/kafka_2.13-2.6.1.tgz -o kafka.tgz | |
| tar xvf kafka.tgz | |
| cd kafka_2.13–2.6.1 | |
| oc project kafka-security | |
| export BOOTSTRAP="$(oc get route my-cluster-kafka-bootstrap -ojsonpath='{.spec.host}'):443" | |
| export CONFIG_FILE=local-config.properties | |
| rm -f ${CONFIG_FILE} | |
| echo "sasl.jaas.config=$(oc get secret my-user -o json | jq -r '.data["sasl.jaas.config"]' | base64 -d -)" >> ${CONFIG_FILE} | |
| echo "sasl.mechanism=SCRAM-SHA-512" >> ${CONFIG_FILE} | |
| echo "security.protocol=SASL_SSL" >> ${CONFIG_FILE} | |
| oc extract secret/my-cluster-cluster-ca-cert --confirm --keys=ca.crt | |
| keytool -keystore cluster-ca.jks -import -file ca.crt \ | |
| -storepass my-cluster-password -noprompt | |
| rm -f ca.crt | |
| echo "ssl.truststore.location=$(pwd)/cluster-ca.jks" >> ${CONFIG_FILE} | |
| echo "ssl.truststore.password=my-cluster-password" >> ${CONFIG_FILE} | |
| bin/kafka-topics.sh --bootstrap-server ${BOOTSTRAP} \ | |
| --command-config ${CONFIG_FILE} --list |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| keytool -exportcert -keypass {truststore-password} \ | |
| -keystore {provided-kafka-truststore.jks} \ | |
| -rfc -file {desired-kafka-cert-output.pem} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bootstrap.servers={kafka-cluster-name}-kafka-bootstrap-{namespace}.{kubernetes-cluster-fully-qualified-domain-name}:443 | |
| security.protocol=SASL_SSL | |
| sasl.mechanism=SCRAM-SHA-512 | |
| sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="{USERNAME}" password="{PASSWORD}"; | |
| ssl.truststore.location={/provided/to/you/by/the/kafka/administrator} | |
| ssl.truststore.password={__provided_to_you_by_the_kafka_administrator__} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| bootstrap.servers={kafka-cluster-name}-kafka-bootstrap.{namespace}.svc.cluster.local:9093 | |
| security.protocol = SASL_PLAINTEXT | |
| sasl.mechanism = SCRAM-SHA-512 | |
| sasl.jaas.config = org.apache.kafka.common.security.scram.ScramLoginModule required username="{USERNAME}" password="{PASSWORD}"; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment