Skip to content

Instantly share code, notes, and snippets.

@igagankalra

igagankalra/producer.py

Last active January 21, 2021 08:30
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save igagankalra/507a2ec74651ab9b8900047f9c7125dd to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save igagankalra/507a2ec74651ab9b8900047f9c7125dd to your computer and use it in GitHub Desktop.
Download ZIP
Produce Load on a Kafka Server -- Need to install the kafka-python library before running the script. Can use `pip install kafka-python` command
Raw
producer.py
from time import sleep
from kafka import KafkaProducer
def generate_logs(topic_name, log_count):
producer = KafkaProducer(
bootstrap_servers=['localhost:9092'],
)
data = b'''{
"Beat_name": "kafkabeat_automation",
"protocol": "Logrhythm",
"sub-protocol": "Globallogic",
"Reason": "Soak Testing",
"types": [
{"name": "Greeting", "type": "record", "fields": [
{"name": "message", "type": "string"}]},
{"name": "Curse", "type": "error", "fields": [
{"name": "message", "type": "string"}]}
],
"messages": {
"hello": {
"doc": "Say hello.",
"request": [{"name": "greeting", "type": "Greeting" }],
"response": "Greeting",
"errors": ["Curse"]
}
}
}
'''
for j in range(log_count):
print(f"Sending {j} out of {log_count} message to Kafka {topic_name}")
producer.send(topic_name, value=data)
sleep(0.1)
generate_logs('beat-topic', 10000000)
Raw
requirements.txt
kafka-python==2.0.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment