A Simple Kafka and Python Walkthrough

Getting data into Apache Kafka is easy if you know how, so let's go through an example step-by-step. In this walkthrough we take you through the process of creating a simple Kafka Producer using the Quix Streams library.
📥 Clone, fork or star the source code: github.com/quixio/simple-kafk...
--
0:00 Introduction
0:29 Setting up a Python Environment
1:10 Fetching some JSON
3:30 Writing it into Kafka
7:40 Job Done! Let's extend it...
10:07 kcat as a debugging tool
10:58 Next steps...

Пікірлер: 24

  • @EndikaMT
    @EndikaMT2 күн бұрын

    The way you explain is awesome. You are really good at teaching and explaining :)

  • @arisweedler4703
    @arisweedler4703Күн бұрын

    I like how he formatted his code with a single space then a save at around 8:10. Had to fix the syntax error so the formatted could properly parse the Python. Dude is used to coding like this. Epic

  • @KyleReevesSci

    @KyleReevesSci

    Күн бұрын

    It's true, they're so at ease programming in python, it's really enjoyable to watch. Their clear commentary at the same time is like effortless choreography.

  • @dreamyangel1858
    @dreamyangel18584 күн бұрын

    Neat video

  • @zedzpan
    @zedzpan3 күн бұрын

    Thank you for this. It was incredibly easy to follow.

  • @shehanreshin
    @shehanreshinАй бұрын

    Love your teaching style

  • @QuixStreams

    @QuixStreams

    Ай бұрын

    Thank you!

  • @SpringerJen
    @SpringerJen23 сағат бұрын

    This is neat

  • @QuixStreams

    @QuixStreams

    21 сағат бұрын

    Glad you liked it! 🙂

  • @steverosam1265
    @steverosam1265Ай бұрын

    Thanks. Love this

  • @souravbarua3991
    @souravbarua3991Ай бұрын

    Thank you🙏 for this video. Much needed. Please keeps coming this type of tutorial videos of kafka python with quixstreams. Please do make video on making dataframe with quixstreams (next level or say continuation of this video)

  • @QuixStreams

    @QuixStreams

    Ай бұрын

    Will do! Stay tuned, there's a video like that planned for this month. :-)

  • @jesusjar11
    @jesusjar11Ай бұрын

    Wonderful! I dont do any python nor kafka, but surely its entertaining watching setting up a project from the ground up : )

  • @QuixStreams

    @QuixStreams

    Ай бұрын

    Thank you! Glad you enjoyed it.

  • @TheMrSnuSnu
    @TheMrSnuSnu10 күн бұрын

    Really good design patterns, thanks for the tutorial

  • @chinyerenwigwe5166
    @chinyerenwigwe51666 күн бұрын

    You just got a new subscriber!!! Simple andl understandable

  • @QuixStreams

    @QuixStreams

    18 сағат бұрын

    Awesome, thank you!

  • @TheAremoh
    @TheAremoh5 күн бұрын

    Subscribed, liked and notification turned on. Great video ❤

  • @ipelezikis
    @ipelezikis8 күн бұрын

    Nice! I didnt know this library. Any advantages/disadvantages over confluent-kafka and kafka-python?

  • @krisajenkins

    @krisajenkins

    8 күн бұрын

    The big advantage is stream processing. All three libraries support producers and consumers, quixstreams also adds in Kafka Streams-style processing. If you ever need to write the kind of code that reads from one topic and writes to another, that's a huge addition. (There's a new video coming out later today that showcases the stream-processing part of the library. 🥳) For the producer/consumer side, quixstreams is actually using confluent-kafka under the hood, and exposes all the same options, so it's basically a superset of that library. Kafka-Python's big advantage is that it's native Python. Both quixstreams and confluent-kafka wrap the C client, librdkafka. That may be an important factor to you. Though given how good Python wheel support is these days, it's not as big a deal as it used to be. 🤷‍♂

  • @fanda10
    @fanda10Күн бұрын

    Hi, nice tutorial! I know it is only an example app, but it’s ok to keep producer open? Would it be better if we open it only just at write time? Thanks. 🙂👍

  • @QuixStreams

    @QuixStreams

    Күн бұрын

    Yes, that's fine. It's designed to be a long-running connection. 🙂

  • @CrypticConsole
    @CrypticConsole4 күн бұрын

    why use this over kafka-python or confluent-kafka-python?

  • @QuixStreams

    @QuixStreams

    17 сағат бұрын

    The big advantage is stream processing. All three libraries support producers and consumers, quixstreams also adds in Kafka Streams-style processing. If you ever need to write the kind of code that reads from one topic and writes to another, that's a huge addition. For the producer/consumer side, quixstreams is actually using confluent-kafka under the hood, and exposes all the same options, so it's basically a superset of that library. Kafka-Python's big advantage is that it's native Python. Both quixstreams and confluent-kafka wrap the C client, librdkafka. That may be an important factor to you. Though given how good Python wheel support is these days, it's not as big a deal as it used to be. 🤷‍♂