Nest.js Kafka Microservice Tutorial
Ғылым және технология
In this video, we create a Nest.js Microservice using Kafka as our transporter. We use both an event-driven and message-driven style of communication.
Github Repo: github.com/mgu...
Kafdrop: github.com/obs...
Get my highly-rated Udemy courses at a discount here: michaelguay.de...
Пікірлер: 73
Get my highly-rated Udemy courses at a discount here: michaelguay.dev/udemy/
In a short time, something complicated was simplified. Great tutorial.
Amazing. Your tutotials are concise, clear and packed with essential info. Thank you!
Wonderful content man! You are one of few creators who make short and clear. I am watching all your videos even if I aware of the topics to see your perception. Eager to wait for more content :-)
@mguay
2 жыл бұрын
Thank you so much! This is greatly appreciated.
Awesome tutorial! Just a heads up - for anyone using NestJs 10+ and @nest/microservices ^10, the data field contains the value itself (data.value will be undefined)
Hi, Can you follow up this video to dead letter queue, error handling, message repeat, load balance etc. Alot of people who use kafka use it at a enterprise-level for log replication, backups, redundancy. Can you make a another simple video like this for handling real world scenarios?
Thanks for taking the time to code out our suggestion from the previous video, appreciated!
@mguay
2 жыл бұрын
Sure, thanks for the great suggestion!
You simplified a lot of things and have wonderful teaching skills, thank you
Awesommeeeeeeee ! Please make more Microservices with nest & react
you save me from my nightmares from my project, bro
Awesome tutorial.... finally found a good tutorial of Kafka with example using NodeJS
thanks this is the awesome tutorial for beginners
Ha. Not what I expected. I'm used to microservices communicating via pub sub using various cloud message buses. This tutorial looks to me awfully like the dreaded "distributed monolith' - the microservices in this video all seem to have to know about each other, rather than just emitting an event and not knowing or caring what other services might consume those events...The 3 services are tightly coupled - this ends up messy, hard to understand when you have many microservices. Kind of misses the whole point of microservices, for me, anyway. Fair comment or did I miss something? I'm surprised. NestJs has always seemed better thought through in it opinions than this... is it really encouraging direct service to service comms as the default ? Or is that just how this example does it?
I found your channel today and subscribed immediately. your channel is full of awesome tutorial. keep up the good work. Besh wishes
Tutorial is really helpful for beginners
Hi Michael, Thanks a lot for the great channel, i will appreciate it if you can go much deeper with kafka clients and consumers options
Noob Question: Shouldn't the API Gateway also implement a message pattern while communicating with the billing service. An Order would go to Gateway -> Billing -> Auth and it would need to return as response back to the user via same Gateway so that the billing was successful?
Hello, future man speaking here. `data.values` is not a thing anymore, the `data` object is now directly the DTO. Nest version: 9 KIafkajs version: 2.2
Great content as always 👏 Keep up the good work 👍
@mguay
2 жыл бұрын
Thank you very much.
Wondering if is it possible to achieve this using mono repo? Because I want to avoid code duplication and manage single node_modules. The way you set it up works but I wonder wouldn't be a bit difficult to maintain the project code base in the long run?
@eunhakoo2124
Жыл бұрын
Also wondering about this
@crossfitiswack1147
7 ай бұрын
Yeah, this seems like not the right approach tbh.. Pretty sure nest is supposed to manage this.
Could anyone comment pros and cons between setting up kafka consumer as a microservice vs as a Nestjs module as in previous tutorial?
I have a question, lets say like we want to share the same Data Types between Billing and Auth for the handleOrderCreated from billing microservices. Like when we send a message right, then subscribe to it and get a callback with a params inside it, how do we want to make it like predictable? Like what is the "inside" of this particular param? How do we do this?
@permanar_
Жыл бұрын
You called the `user.stripeUserId` because you "know" whats inside the object. But then how do we make it like type-safe and predictable?
Hi Michael, thank you for sharing, for some reason .Everthing is working well but I'm getting an undefined when a consume the event, probably my kakfa configuration is different
Hi. In my case, I just got the undefined value just because my payload in eventPattern does not have value property. I don't know why it is different with the video . Please share if you know the difference with me.
@darrenhudson1033
Жыл бұрын
I had the same issue and just used the data object without calling data.value. may be a change in the JSON.stringify maybe?
Hi Michael, thanks for this tutorial. It is worth every minute of my time. I have a very important question. How do we send back a response to the API gateway, which is intended for the frontend application ? Lets say billing was successfully and frontend application needs a feed back in order to proceed or cancel the order. How do we implement this ?
@mguay
2 жыл бұрын
Thanks! Continually polling the backend, Websockets, or a refresh button are some common approaches.
this is gold, thank you
I am getting this error - "The group coordinator is not available". I am using docker-compose with WSL2 on windows home
hello , thx for the video , what is the difference between this approach and the other approach you did it in the other video using only kafka js , is it the same ?
@user-yr1df4ko6u
3 ай бұрын
me too . its not clear for me .
Hi, first of all thank you for this video. I implemented according but only get empty memberAssignment as {}
Hi, Is there any way to get messages based on offset or get offset of last consumed message?
Hi amazing video! I have a issue, I have a macbook M1 and tried to use docker-compose for intall kafdrop but I had a error. Can you have a mac M1? Do know how to install kafdrop in M1 machines? Thanks!
Great content 🤩
pretty nice. Big ups 👍
Amazing content. Thanks
How to keep nodemodules common for all services ?
Great tutorial, merci!
thank you this is a great tutorial.
thanks amazing video. dont u have a course for this on udemi or somewhere? second, the concept of microservice is to have each service based on one language if team wants, and the way nest js works seems its fit for whole communication through js files? third!, Its a bit confusing when u are sending any type of request to the same root route like localhost:3000. whilst having route for each service might makes more sense what do u think? I mean u are sending all type of request to it and it dispatch events! fine! but there is no actual route so front-end can call in different end point.. can u clarify on that, tnx
It's amazing tutorial.
thanks
Nice video, is there a way to do that using Fastify instead od Express?
Can anyone tell, how to create API with implementation of kafka in nodejs?
Nice one
great content 😊👍
Does the event pattern has a return type as message pattern. I asked because of the subscription you did.
@mguay
2 жыл бұрын
It returns an Observable the same as the message pattern, however I'm not sure what data would come back, if any.
I learned about microservices it is the very best when the services are fully independent from each other. That means they are not communicating directly with each other, but only with events. In this video you are useing the both way to communicate with services, it's weird for me. The independency between microservices is crucial, because they can not effect each other processes (when one service terminated unexpectedly the other services have to work fine). If it necessery you have to store data redundant on each service. What do you think about that? Actually this is a very good video :)
@mguay
2 жыл бұрын
I agree this approach can come with some baggage. Check out my other Nest.js + Kafka video for a more decoupled approach.
@kisstamas6675
2 жыл бұрын
@@mguay thank you, i'm following you, so i will check your videos ;)
Hi great content , would you show a server-frontend mvc architecture using nest-next npm package
Cool !!! Thanx a lot
amazing.
great tutorial! really sorry for the stupid question.. but what vscode theme are you using?
@mguay
2 жыл бұрын
No such thing. I’m using Material theme and icons.
Great
Hi, I'm currently on nest@9.x.x and kafka@2.x.x and had to call the toString() prototype method on the ClientKafka emit() method. The NestJs documentation needs update.
How can I set up the VS Code theme as yours, can u give me so reference? Again, I love your video so much, great work, looks cool! keep going bro!
@mguay
2 жыл бұрын
Thanks! I use Material Theme + Icons which are extensions.
New sub
-21deg Dude do u live in siberia ? :p ... great content BTW.
@mguay
2 жыл бұрын
I'm in the USA.
Nice tutorial, but too much code in one place, making it hard to comprehend
Thanks! I literally love your content! But you're too fast, I think in 5 minutes I paused 100 times!
@mguay
2 жыл бұрын
Thanks! And I appreciate the feedback.