Skip to content

Instantly share code, notes, and snippets.

@lurenx

lurenx/producer.py

Created March 24, 2016 10:20
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save lurenx/08de7eb49ec825de5bb4 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save lurenx/08de7eb49ec825de5bb4 to your computer and use it in GitHub Desktop.
Download ZIP
kafka python producer
Raw
producer.py
from pykafka import KafkaClient
import avro.schema
import io, random
from avro.io import DatumWriter
schema_path="user.avsc"
schema = avro.schema.parse(open(schema_path).read())
client = KafkaClient(hosts="127.0.0.1:9092")
topic = client.topics['avro']
with topic.get_sync_producer() as producer:
for i in xrange(10):
writer = avro.io.DatumWriter(schema)
bytes_writer = io.BytesIO()
encoder = avro.io.BinaryEncoder(bytes_writer)
writer.write({"name": "123", "favorite_color": "111", "favorite_number": random.randint(0,10)}, encoder)
raw_bytes = bytes_writer.getvalue()
producer.produce(raw_bytes)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment