Apache Kafka Explained (Comprehensive Overview)

Kafka The Definitive Guide ► amzn.to/3aw1tr6
Apache Kafka is an open-source, publish/subscribe (pub/sub) messaging system, also very often described as a distributed event log where all the new records are immutable and appended to the end of the log.
Kafka aims to provide a reliable and high-throughput platform for handling real-time data streams and building data pipelines. It also provides a single place for storing and distributing events that can be fed into multiple downstream systems which helps to fight the ever-growing problem of integration complexity. Besides all of that Kafka can also be easily used to build a modern and scalable ETL, CDC or big data ingest systems.
Kafka is used across multiple industries, from companies like Twitter and Netflix to Goldman Sachs and Paypal. It was originally developed by Linkedin and open sourced in 2011.
Post ► finematics.com/apache-kafka-e...
Website ► finematics.com
Follow me on Twitter ► / finematics
Donations:
Bitcoin ► 18ngQa7qzfg9RpstBz3ZCSm2fAzm6tqaa4
Lightning ► tippin.me/@finematics
Referral links:
Brave Browser ► brave.com/fin661 (ads-free browsing)
Ledger Nano S ► shop.ledger.com?r=2af228941155 (hardware wallet)
If you like this video, give it a like, share it and subscribe to my channel.
Thanks for watching!

Пікірлер: 140

  • @jimmortenson9383
    @jimmortenson93834 жыл бұрын

    This video does what 2 hour conference talks wish they could. Thank you!

  • @Finematics

    @Finematics

    4 жыл бұрын

    thanks Jim!

  • @kenchang3456

    @kenchang3456

    3 жыл бұрын

    I agree with @Jim Mortenson, great video. I'm subscribed. Thank you very much.

  • @osallllllll
    @osallllllll3 жыл бұрын

    This video proves that teaching is not for everybody. 20 minutes straight to the point.

  • @Marius5404
    @Marius54043 жыл бұрын

    Honestly one of the best introductions into a technology I have ever seen. Everything is so clear and it has great examples. Thank you!

  • @cheatman05
    @cheatman053 жыл бұрын

    This is the most comprehensible presentation I've seen in a long time!

  • @Finematics

    @Finematics

    3 жыл бұрын

    Thanks!

  • @andrewhting
    @andrewhting Жыл бұрын

    Wow, the first 1:30 and things have become way more clear to me than the last 3 hours of kafka videos combined from multiple other content creators. Thank you for breaking it down from a high level like that. Bravo!

  • @alexkumpula
    @alexkumpula3 жыл бұрын

    This really answered all the nitty-gritty architectural questions I was puzzling over while watching through other videos. Thanks for the video!

  • @StyleTrick
    @StyleTrick3 жыл бұрын

    This is one of the best videos on Kafka I've seen, thorough and straight to the point!

  • @axelhuachacona6227
    @axelhuachacona62273 жыл бұрын

    Thanks for your work man! This is so far the best video talking about Kafka

  • @imyashdeepsharma
    @imyashdeepsharma4 жыл бұрын

    Loved it, Great Explanation to the each component of the architecture.

  • @oswaldumeh
    @oswaldumeh3 жыл бұрын

    This video has totally lifted me from a zero in terms of my understanding of Kafka. The explanations are concise and clear, shows a firm understanding of the topic from the creator. Thanks for this. Well done!

  • @Finematics

    @Finematics

    3 жыл бұрын

    Thanks Oswald! I'm happy it was helpful

  • @ritviksharmacooldude
    @ritviksharmacooldude3 жыл бұрын

    Every time i “replay” this video to understand kafka 😎

  • @narsiktube
    @narsiktube4 жыл бұрын

    One of the best conceptual introductions to Apache Kafka. Thank you and keep up the great work !

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks a lot!

  • @aycanayhan3680
    @aycanayhan36804 жыл бұрын

    Wonderful explanation, thanks a lot!

  • @mahenrathod5285
    @mahenrathod52852 жыл бұрын

    Very informative and easy to follow. One of the best refresher videos I found on the internet. Thanks for this amazing video 👍

  • @victorgomez335
    @victorgomez335 Жыл бұрын

    Great video, everything was explained clearly and concisely. One of the best introductions to a topic that I have ever seen in my life. Continue like that

  • @demidevil666
    @demidevil6663 жыл бұрын

    Thanks a lot for this comprehensive overview. I recently started a new job and we use Kafka in our team, yet I still have a lot of questions about it. Your video helped a lot! :)

  • @ckhare11
    @ckhare114 жыл бұрын

    Great to get a quick overview of Kafka. Nicely explained.

  • @bws462
    @bws4624 жыл бұрын

    thanks for making these videos. Well done, I learned a lot. Looking forward to more

  • @ardsandlived
    @ardsandlived3 жыл бұрын

    This is an excellent presentation on Kafka. As a solution architect with 20+ years experience in messaging middleware, I will recommend it to anyone who is starting to look at using Kafka (as I am).

  • @SharpKnife523
    @SharpKnife5233 жыл бұрын

    great tutorial in such a short time. excellent work!!

  • @mariomoiane3290
    @mariomoiane3290 Жыл бұрын

    Thank you for such short and comprehensive explanation... THANK YOU!

  • @parkavimani3995
    @parkavimani39952 жыл бұрын

    Such a refresher this video is!

  • @jonathanmeyer4842
    @jonathanmeyer48424 жыл бұрын

    Amazing detailed presentation ! Keep it up :)

  • @user-ds3eq5te8l
    @user-ds3eq5te8l8 ай бұрын

    Thank you for this video!!! One of the best on this topic!

  • @irekd.4077
    @irekd.40773 жыл бұрын

    Please keep up the good work. Great explanation of complicated matter. If we had academic teachers like you, we would be in a better place. Thank you!

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @filipeschenkel
    @filipeschenkel3 жыл бұрын

    Amazing video, you summarize a really great amount of information very well... Thank you :)

  • @thewisearchitect
    @thewisearchitect3 жыл бұрын

    Beautifully explained. Well done.

  • @deepsikhakar9166
    @deepsikhakar9166 Жыл бұрын

    one of the best deep dive in kafka great work

  • @mehdimomeni6671
    @mehdimomeni667118 күн бұрын

    Clean and concise! Niiice work...

  • @AllenLVW
    @AllenLVW3 жыл бұрын

    The best video to begin understanding Kafka!

  • @davidfang2725
    @davidfang27254 жыл бұрын

    Quick, meaningful, and useful overview of Kafka. Thank you!

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks David!

  • @funkiekeda
    @funkiekeda4 жыл бұрын

    Superb content, sir. Please do continue.

  • @user-he6fm8cd3x
    @user-he6fm8cd3x4 жыл бұрын

    Great presentation and explanation. Thank you!

  • @havefuntrading6263
    @havefuntrading62633 жыл бұрын

    Great stuff, very well explained

  • @micha6568
    @micha6568 Жыл бұрын

    Great Kafka tutorial. Thanks a much👍

  • @satyanarayanapeteti2690
    @satyanarayanapeteti26903 жыл бұрын

    Nice explanation with in depth analysis of architecture

  • @laxmigo
    @laxmigo3 жыл бұрын

    One of the best Kafka video, i came across. Thanks you !

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @anaschtourou288
    @anaschtourou2882 жыл бұрын

    excellent video, thank you.

  • @alrighttheresham
    @alrighttheresham4 жыл бұрын

    @Finematics Fantastic video, would love to see similar sized overview for streams and connect

  • @dralps
    @dralps Жыл бұрын

    Greatly explained!

  • @surajmali3885
    @surajmali38854 жыл бұрын

    Excellent explanation.. Thank you

  • @prateekgupta4347
    @prateekgupta43472 жыл бұрын

    Very Informative video. Thanks for sharing

  • @VictorRomanC
    @VictorRomanC4 жыл бұрын

    Great video! I would love to see a Finematics on Spark, Micro-services or Functional Programming

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @pailacognizant
    @pailacognizant2 жыл бұрын

    Fantastic Video.. Great demonstration skills in action and easy to understand

  • @nandrajmeshram8755
    @nandrajmeshram87553 жыл бұрын

    great video thanks for share such worthful information.

  • @robertfowler2811
    @robertfowler28113 ай бұрын

    fantastic explanation one the best explanations I have ever read

  • @dozedyolk
    @dozedyolk4 жыл бұрын

    Great explanation of a complex topic.

  • @MartynInEurope
    @MartynInEurope4 жыл бұрын

    Great job! Thanks!

  • @juanpabloaguirre5837
    @juanpabloaguirre58373 жыл бұрын

    Extremely useful. Is like 4 months of traditional University courses

  • @Finematics

    @Finematics

    3 жыл бұрын

    Thanks Juan!

  • @1982jayanth
    @1982jayanth3 жыл бұрын

    I am new to kafka, and this gives a great idea about the topic. Thanks Finematics.

  • @Finematics

    @Finematics

    3 жыл бұрын

    Thanks Jayanth!

  • @9998872108
    @99988721083 жыл бұрын

    Very good explanation.

  • @rajkrishna5439
    @rajkrishna54392 жыл бұрын

    great intro well summarized

  • @sgwong513
    @sgwong5133 жыл бұрын

    This is a video to make me subscribe. short video but a lot of contents. much better than some 2 or 3 hours video. Thanks for the awesome video, i think the creator did spend a lot of time to come out this video.

  • @user-wh9pc3hb3t
    @user-wh9pc3hb3t4 жыл бұрын

    A good overview, thank you!

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks!

  • @rajkkorrapati
    @rajkkorrapati4 жыл бұрын

    Great Explanation ! I'm new to this but after the video now I can go back and talk , trigger my solutions around it.

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks Rajesh!

  • @suckyboy
    @suckyboy3 жыл бұрын

    Tanks for explaining

  • @miguelangelojeda8382
    @miguelangelojeda83822 жыл бұрын

    really nice video!

  • @lakshay510
    @lakshay5103 жыл бұрын

    Wow, What an amazing video. Subscribed!

  • @Finematics

    @Finematics

    3 жыл бұрын

    Awesome, thank you!

  • @Ikilledthebanks
    @Ikilledthebanks2 жыл бұрын

    Thank you yet again, I owe you.

  • @nditahsamweld346
    @nditahsamweld3463 жыл бұрын

    An amazing video. Well done!

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @morvenhuang1499
    @morvenhuang14993 жыл бұрын

    Great and neat video.

  • @carapungo
    @carapungo3 жыл бұрын

    Excellent presentation bro!!

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @marcosferreira220863
    @marcosferreira2208637 ай бұрын

    Very good introduction

  • @CraigStrachanZA
    @CraigStrachanZA3 жыл бұрын

    Brilliant overview. Very well done.

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @niharikapatil902
    @niharikapatil9024 жыл бұрын

    THIS IS A BRILLIANT VIDEOOOO!!!

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks!

  • @AbdulMannan-rv5yx
    @AbdulMannan-rv5yx3 жыл бұрын

    Really nicely explained. I loved it

  • @evisionwithjps602

    @evisionwithjps602

    3 жыл бұрын

    New kafka practical tutorial ..kzread.info/dash/bejne/n6B8kttwdrLbm7g.html

  • @user-oy4kf5wr8l
    @user-oy4kf5wr8l4 жыл бұрын

    super clear video... thank you...amazing....seriously...man

  • @Finematics

    @Finematics

    4 жыл бұрын

    thanks!

  • @Hraffnir
    @Hraffnir4 жыл бұрын

    Very well explained, well done.

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks

  • @meee3107
    @meee31073 жыл бұрын

    Thank you so much for the great video, so informative in few minutes. I have a question that is a little off topic, what software do you use to create your videos ? Thank you.

  • @rosewinjr
    @rosewinjr2 жыл бұрын

    Nice explanation

  • @juliahuanlingtong6757
    @juliahuanlingtong67573 жыл бұрын

    Really compreheasive with a compact of gold!!~ If given some examples, it would make 10/10!

  • @rishisaireddysudireddy5437
    @rishisaireddysudireddy54374 жыл бұрын

    Great video , keep doing more

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks Rishi!

  • @borisdavidov5
    @borisdavidov53 жыл бұрын

    good stuff!

  • @sahillamba3879
    @sahillamba38794 жыл бұрын

    Thanks for sharing knowledge. Video is good

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks!

  • @floidcosta
    @floidcosta3 жыл бұрын

    good stuff.

  • @user-yw2rm2gy5g
    @user-yw2rm2gy5g4 жыл бұрын

    nice video!

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks!

  • @rohitsinha3876
    @rohitsinha38763 жыл бұрын

    Amazing and well explained. Precise and to the point. @Finematics Can you also please explain the part if exactly once delivery and at least once delivery issue in kafka. Btw thanks for this video

  • @ssingh2k8
    @ssingh2k83 жыл бұрын

    Beautiful.

  • @bp8730
    @bp87302 жыл бұрын

    fantastic

  • @roberts1782
    @roberts17823 жыл бұрын

    Thank you

  • @Manolete919
    @Manolete9194 жыл бұрын

    Excellent

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thank you!

  • @russ_fx
    @russ_fx4 жыл бұрын

    Such a clear explanation. Do you have a video like this for each Apache technology ? (Spark, airflow, flume etc)

  • @Finematics

    @Finematics

    4 жыл бұрын

    Not yet, but there will be more videos coming up on different topics including Apache tech. You can subscribe to the channel to stay in the loop.

  • @mariosergiosilva8820
    @mariosergiosilva8820 Жыл бұрын

    Amazing 😁

  • @nashram344
    @nashram3444 жыл бұрын

    DAM !!! excellent video

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks!!

  • @authhwang8774
    @authhwang87744 жыл бұрын

    good video for knowing kafka

  • @Finematics

    @Finematics

    4 жыл бұрын

    thanks!

  • @srik006
    @srik0063 жыл бұрын

    Very nice explanation. Why was connectors and streaming not explained ?

  • @arj123sub
    @arj123sub4 жыл бұрын

    Super ,

  • @DontTakeCrack
    @DontTakeCrack2 жыл бұрын

    Thanks, i find this summary pretty useful. Not sure about that ending though, things like JMS + backpressure seemed pretty tacked on and im pretty sure JMS is not directly related to rabbitmq.

  • @unknown_device687
    @unknown_device6874 жыл бұрын

    Good video but there are some misconceptions. There is no backpressure in kafka only throttling which implemented with so-named quotas on the producer side. And it is also possible to implement exactly once using low-level producer/consumer api.

  • @saravanprathi6956
    @saravanprathi69563 жыл бұрын

    Thank you, very nice video! I hope you use a black screen with white pen for your future videos. It would be easy on our eyes!

  • @qmar1980
    @qmar19809 ай бұрын

    Thank you for the content. I am totally beginner in Kafka - so I have question related to the distributing messages into the Kafka for specific Topic. In the beginning part of the presentation it was said that if the message will be written in the Round Robin fasion (6:48). I understood that the message will be written only in one specific partition (not all of them). Than starting from (9:21) it is said that if the topic has multiple partitions, than messages would be written into multiple partitions evenly. Can someone please explain the concept in order to be sure how it would be really done ?

  • @justinnguyen3845
    @justinnguyen38453 жыл бұрын

    hi, what tools did you use to create these doodle videos?

  • @mnfchen
    @mnfchen2 жыл бұрын

    12:32 Note that consumers no longer have to connect to the leader partition. They can now listen on a replica (for committed offsets).

  • @windowcrystal4012
    @windowcrystal40124 жыл бұрын

    Could you please share the slides? It is very good

  • @lindilebangose7667
    @lindilebangose76674 жыл бұрын

    Can you please guys do zookeeper as well

  • @sergeykotov4719
    @sergeykotov47194 жыл бұрын

    It would be nice to hear when RabbitMQ is preferable to Kafka, in what use cases Kafka is not the best option as a message system.

  • @jamesburke7033
    @jamesburke70334 жыл бұрын

    Great video and gave me a great understanding at a high level. It seems like Kafka can be a great solution in so many way, but I'm a little confused where and how. If I wanted to build Kafka into a web site (i.e. shopping car, a cloud based accounting system) as a broker and controller of a web site currently built on a LAMP stack would that be possible. How and why would I do this? Can somebody please point me into the right direction to find some answers?

  • @Finematics

    @Finematics

    4 жыл бұрын

    Thanks James. In the systems I built so far Kafka was used to deliver messages between internal services. As an example you can imagine a website that allows you to register a new user. A request to create a new user would go to a backend service usually via REST or Websocket. After that the backend endpoint would construct a message with the newly registered user and send it to Kafka so other services could pick it up and do something with it (for example another service could read it from Kafka to add it to the database).

  • @geolee76

    @geolee76

    3 жыл бұрын

    What about the reply response to user. User would be waiting for consumer to add into db and how it response back to user once done

  • @neyudn7896
    @neyudn78963 жыл бұрын

    I'm wondering which tool did you use to make a full map of slides???

  • @Finematics

    @Finematics

    3 жыл бұрын

    I'm using Videoscribe

  • @danielsonski
    @danielsonski3 жыл бұрын

    Nice... Nothings seems to be simplified. I, sometimes, felt a little backpressure myself :-p. (no worries, I can revisit this later on :-) )

  • @mqtt07
    @mqtt072 жыл бұрын

    How seriously should one consider the "deliver at least once" limitation of Kafka in a solutioning design. In other words how often messages are delivered more than once to become a problem in a solution?