A Simple Kafka and Python Walkthrough
Getting data into Apache Kafka is easy if you know how, so let's go through an example step-by-step. In this walkthrough we take you through the process of creating a simple Kafka Producer using the Quix Streams library.
📥 Clone, fork or star the source code: github.com/quixio/simple-kafk...
--
0:00 Introduction
0:29 Setting up a Python Environment
1:10 Fetching some JSON
3:30 Writing it into Kafka
7:40 Job Done! Let's extend it...
10:07 kcat as a debugging tool
10:58 Next steps...
Пікірлер: 24
The way you explain is awesome. You are really good at teaching and explaining :)
I like how he formatted his code with a single space then a save at around 8:10. Had to fix the syntax error so the formatted could properly parse the Python. Dude is used to coding like this. Epic
@KyleReevesSci
Күн бұрын
It's true, they're so at ease programming in python, it's really enjoyable to watch. Their clear commentary at the same time is like effortless choreography.
Neat video
Thank you for this. It was incredibly easy to follow.
Love your teaching style
@QuixStreams
Ай бұрын
Thank you!
This is neat
@QuixStreams
21 сағат бұрын
Glad you liked it! 🙂
Thanks. Love this
Thank you🙏 for this video. Much needed. Please keeps coming this type of tutorial videos of kafka python with quixstreams. Please do make video on making dataframe with quixstreams (next level or say continuation of this video)
@QuixStreams
Ай бұрын
Will do! Stay tuned, there's a video like that planned for this month. :-)
Wonderful! I dont do any python nor kafka, but surely its entertaining watching setting up a project from the ground up : )
@QuixStreams
Ай бұрын
Thank you! Glad you enjoyed it.
Really good design patterns, thanks for the tutorial
You just got a new subscriber!!! Simple andl understandable
@QuixStreams
18 сағат бұрын
Awesome, thank you!
Subscribed, liked and notification turned on. Great video ❤
Nice! I didnt know this library. Any advantages/disadvantages over confluent-kafka and kafka-python?
@krisajenkins
8 күн бұрын
The big advantage is stream processing. All three libraries support producers and consumers, quixstreams also adds in Kafka Streams-style processing. If you ever need to write the kind of code that reads from one topic and writes to another, that's a huge addition. (There's a new video coming out later today that showcases the stream-processing part of the library. 🥳) For the producer/consumer side, quixstreams is actually using confluent-kafka under the hood, and exposes all the same options, so it's basically a superset of that library. Kafka-Python's big advantage is that it's native Python. Both quixstreams and confluent-kafka wrap the C client, librdkafka. That may be an important factor to you. Though given how good Python wheel support is these days, it's not as big a deal as it used to be. 🤷♂
Hi, nice tutorial! I know it is only an example app, but it’s ok to keep producer open? Would it be better if we open it only just at write time? Thanks. 🙂👍
@QuixStreams
Күн бұрын
Yes, that's fine. It's designed to be a long-running connection. 🙂
why use this over kafka-python or confluent-kafka-python?
@QuixStreams
17 сағат бұрын
The big advantage is stream processing. All three libraries support producers and consumers, quixstreams also adds in Kafka Streams-style processing. If you ever need to write the kind of code that reads from one topic and writes to another, that's a huge addition. For the producer/consumer side, quixstreams is actually using confluent-kafka under the hood, and exposes all the same options, so it's basically a superset of that library. Kafka-Python's big advantage is that it's native Python. Both quixstreams and confluent-kafka wrap the C client, librdkafka. That may be an important factor to you. Though given how good Python wheel support is these days, it's not as big a deal as it used to be. 🤷♂