Kafka Consumers in Python - A Walkthrough
Ғылым және технология
Getting data out of Apache Kafka is easy if you know how, so let's go through an example step-by-step. In this walkthrough we take you through the process of creating a simple Kafka Consumer using the Quix Streams library.
📥 Clone, fork or star the source code: github.com/quixio/simple-kafk...
📺 Watch the producer-side walkthrough: • A Simple Kafka and Pyt...
--
0:00 Intro & Setup
0:47 Connecting & Subscribing
3:15 Reading the Messages
8:13 A Little Refactoring
8:55 We Could Stop There...
9:10 Offset Management
13:08 Recap and Reflect
13:45 Until Next Time...
Пікірлер: 22
This is briliantly explained. As a kafka admin of multiple cluster, we are now bit scared of multiple teams trying to implement consumers and producers by themself because of poor offset management. I should make some Python/JS/Java examples! Thanks.
I came to learn Kafka, learnt that and how to write great python code!!
OMG. I actually understand what you are talking about! Thank you sir.
@QuixStreams
6 күн бұрын
You're welcome!
Great video. Waiting for this and upcoming videos. Thank you for this tutorial series 🙏.
@krisajenkins
Ай бұрын
Glad you're enjoying it. We have quite a few planned, but if there's anything in particular you're interested in seeing, let me know. :-)
@souravbarua3991
Ай бұрын
@@krisajenkins For now I am learning from you. Eagerly waiting for next step/videos.Thank you again for this tutorial series🙏.
fantastic! will have to watch your other videos - thank again!
Nice vids 🎉
Really a great video. I was waiting for such masterpiece on kafka with python, can't find any before. Can you also make a video on how to add a notifier_consumer (may it through email or something). Also consider configuring Kong with kafka.
Another great video Kris
@krisajenkins
Ай бұрын
Thanks! 🙂
Which code editor are u using in this video ?
@krisajenkins
Ай бұрын
It’s Neovim. 👍
@munishkhatri33
Ай бұрын
@krisajenkins Thanks, I tried to install it, but it doesn't give type hints as I saw in your video. Any plugins required for that?
@krisajenkins
Ай бұрын
@@munishkhatri33 Ah yeah, takes a little configuration. `nvim-lspconfig` is what you need, but if I were you I'd start with the Neovim Kickstart project. It's a default config which will set up lsp for you, plus a lot more.
@krisajenkins
Ай бұрын
(I'd put a link to it, but every time I do, KZread auto-deletes my comment. Google will guide you to it. 👍)
I'm still new to Kafka , if i use socket io , should this socket io server sit in Kafka? to manage it?
@QuixStreams
5 күн бұрын
Apache Kafka is essentially a distributed server running in a cluster. You coudn't host your socket.io server inside of Kafka but it could run in the same infrastructure (one option is in Quix Cloud, which can run any Docker container). From there you could use a socket.io client in Quix Streams
@DePhpBug
5 күн бұрын
@@QuixStreams Thanks :)
i have a question, will it gonna listen for ever if i am in a case were i want to have this code written to be always listenning, and never stops, if i run that code and leave it will it works as i want or i need to do something in addition? @Quix
@QuixStreams
Күн бұрын
Hi, this code will run forever, unless there is an issue like the machine it's running on being turned off, it's also not production code so adding some error handling etc would certainly be a good idea. If you want to run this in production with resiliency and SLA's check out Quix Cloud -> quix.io/signup