Kafka Consumers in Python - A Walkthrough

Ғылым және технология

Getting data out of Apache Kafka is easy if you know how, so let's go through an example step-by-step. In this walkthrough we take you through the process of creating a simple Kafka Consumer using the Quix Streams library.
📥 Clone, fork or star the source code: github.com/quixio/simple-kafk...
📺 Watch the producer-side walkthrough: • A Simple Kafka and Pyt...
--
0:00 Intro & Setup
0:47 Connecting & Subscribing
3:15 Reading the Messages
8:13 A Little Refactoring
8:55 We Could Stop There...
9:10 Offset Management
13:08 Recap and Reflect
13:45 Until Next Time...

Пікірлер: 22

  • @michaelslavev
    @michaelslavev8 күн бұрын

    This is briliantly explained. As a kafka admin of multiple cluster, we are now bit scared of multiple teams trying to implement consumers and producers by themself because of poor offset management. I should make some Python/JS/Java examples! Thanks.

  • @JacobChencha
    @JacobChencha11 күн бұрын

    I came to learn Kafka, learnt that and how to write great python code!!

  • @vincentchong6005
    @vincentchong60058 күн бұрын

    OMG. I actually understand what you are talking about! Thank you sir.

  • @QuixStreams

    @QuixStreams

    6 күн бұрын

    You're welcome!

  • @souravbarua3991
    @souravbarua3991Ай бұрын

    Great video. Waiting for this and upcoming videos. Thank you for this tutorial series 🙏.

  • @krisajenkins

    @krisajenkins

    Ай бұрын

    Glad you're enjoying it. We have quite a few planned, but if there's anything in particular you're interested in seeing, let me know. :-)

  • @souravbarua3991

    @souravbarua3991

    Ай бұрын

    @@krisajenkins For now I am learning from you. Eagerly waiting for next step/videos.Thank you again for this tutorial series🙏.

  • @samable9585
    @samable958529 күн бұрын

    fantastic! will have to watch your other videos - thank again!

  • @radoniainarakotonirina2856
    @radoniainarakotonirina285616 күн бұрын

    Nice vids 🎉

  • @dailypakaffairs2590
    @dailypakaffairs2590Ай бұрын

    Really a great video. I was waiting for such masterpiece on kafka with python, can't find any before. Can you also make a video on how to add a notifier_consumer (may it through email or something). Also consider configuring Kong with kafka.

  • @steverosam1265
    @steverosam1265Ай бұрын

    Another great video Kris

  • @krisajenkins

    @krisajenkins

    Ай бұрын

    Thanks! 🙂

  • @munishkhatri33
    @munishkhatri33Ай бұрын

    Which code editor are u using in this video ?

  • @krisajenkins

    @krisajenkins

    Ай бұрын

    It’s Neovim. 👍

  • @munishkhatri33

    @munishkhatri33

    Ай бұрын

    @krisajenkins Thanks, I tried to install it, but it doesn't give type hints as I saw in your video. Any plugins required for that?

  • @krisajenkins

    @krisajenkins

    Ай бұрын

    @@munishkhatri33 Ah yeah, takes a little configuration. `nvim-lspconfig` is what you need, but if I were you I'd start with the Neovim Kickstart project. It's a default config which will set up lsp for you, plus a lot more.

  • @krisajenkins

    @krisajenkins

    Ай бұрын

    (I'd put a link to it, but every time I do, KZread auto-deletes my comment. Google will guide you to it. 👍)

  • @DePhpBug
    @DePhpBug6 күн бұрын

    I'm still new to Kafka , if i use socket io , should this socket io server sit in Kafka? to manage it?

  • @QuixStreams

    @QuixStreams

    5 күн бұрын

    Apache Kafka is essentially a distributed server running in a cluster. You coudn't host your socket.io server inside of Kafka but it could run in the same infrastructure (one option is in Quix Cloud, which can run any Docker container). From there you could use a socket.io client in Quix Streams

  • @DePhpBug

    @DePhpBug

    5 күн бұрын

    @@QuixStreams Thanks :)

  • @michelchaghoury9629
    @michelchaghoury96293 күн бұрын

    i have a question, will it gonna listen for ever if i am in a case were i want to have this code written to be always listenning, and never stops, if i run that code and leave it will it works as i want or i need to do something in addition? @Quix

  • @QuixStreams

    @QuixStreams

    Күн бұрын

    Hi, this code will run forever, unless there is an issue like the machine it's running on being turned off, it's also not production code so adding some error handling etc would certainly be a good idea. If you want to run this in production with resiliency and SLA's check out Quix Cloud -> quix.io/signup

Келесі