Skip to content

Instantly share code, notes, and snippets.

@JeOam

JeOam/Kafka.md

Last active October 9, 2023 17:13
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save JeOam/d5f08938baec3039bd70 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save JeOam/d5f08938baec3039bd70 to your computer and use it in GitHub Desktop.
Download ZIP
Apache Kafka Notes
Raw
Kafka.md

Apache Kafka is publish-subscribe messaging rethought as a distributed commit log.

@JeOam
Copy link
Author

JeOam commented Feb 17, 2016

查看所有的 topic:

~/kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181

查看 topics 的分区情况:

~/kafka/bin/kafka-topics.sh --describe --zookeeper localhost:2181

创建 topic:

~/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --create --partitions 1 --replication-factor 1 --topic topic-name

删除 topic:

~/kafka/bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic [topic_name]

@JeOam
Copy link
Author

JeOam commented Feb 24, 2016 

ProducerConfig values: 
compression.type = none
metric.reporters = []
metadata.max.age.ms = 300000
metadata.fetch.timeout.ms = 60000
reconnect.backoff.ms = 50
sasl.kerberos.ticket.renew.window.factor = 0.8
bootstrap.servers = [192.168.5.130:9092]
retry.backoff.ms = 100
sasl.kerberos.kinit.cmd = /usr/bin/kinit
buffer.memory = 33554432
timeout.ms = 30000
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
ssl.keystore.type = JKS
ssl.trustmanager.algorithm = PKIX
block.on.buffer.full = false
ssl.key.password = null
max.block.ms = 6000
sasl.kerberos.min.time.before.relogin = 60000
connections.max.idle.ms = 540000
ssl.truststore.password = null
max.in.flight.requests.per.connection = 5
metrics.num.samples = 2
client.id = 
ssl.endpoint.identification.algorithm = null
ssl.protocol = TLS
request.timeout.ms = 30000
ssl.provider = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
acks = all
batch.size = 16384 # This is an upper limit of how many messages Kafka Producer will attempt to batch before sending – specified in bytes (Default is 16K bytes)
ssl.keystore.location = null
receive.buffer.bytes = 32768
ssl.cipher.suites = null
ssl.truststore.type = JKS
security.protocol = PLAINTEXT
retries = 0
max.request.size = 1048576
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
ssl.truststore.location = null
ssl.keystore.password = null
ssl.keymanager.algorithm = SunX509
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
send.buffer.bytes = 131072
linger.ms = 3000  # How long will the producer wait before sending in order to allow more messages to get accumulated in the same batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment