Basant thank you. You are a Genius and a very technical and kind person.

Awesome ...Thank you for sharing your precious knowledge with us.

This is really awesome. I just have created similar producer-consumer. Which use to produce data as an object using Kafka which required lots of configuration at both side. With Spring Cloud Stream we have specified almost null configuration.

@Javatechie

Жыл бұрын

Yes exactly

what a relief, struggling to understand these, thank you so much , excellent explanation, :-),

@Javatechie

2 жыл бұрын

Thanks Kamal ☺️

Great tutorial with so well explanation. Thank you for making such kind of tutorial.

At last my doubts Clarified.... Tq a lot

Simply super, nice videos 👍👌

Awesome explanation..thank you 2 question faced in interview for kafka/Rabbit Mq: 1) how to avoid duplicate message to be consumed in consumer? . 2) how to handel if topic is full of messages?

@baptiste6436

2 жыл бұрын

Did u find a solution to 1 ?

Very helpful tutorial, you are doing great work, keep it up...

The content you showcase is amazing and really good to understand thank you sir

Excellent Bro..👍

Awesome . Very clean explanation. Thank you Bro🙏

Excellent .

God Bless You.

Very informative. Could you also make a video on data transfer using avro instead of json. Also some use case where multiple input and output topics are involved would be great. Thanks.

Quality video tutorial. Just curious to know which software you are using for screen recording and which device you are using for voice recording.

Superb, very easy way explained and good to remember me this concept. please make video on kafka vs rabbitmq and with use case.

Excellent explanation as always sir..!

Extremely helpful for beginners, thanks

Awesome

Thank you :)

nicely explained, very helpful

Thank you, I was wondering how spring boot and kafka are related.

Thank you so much 🎉🎉🎉

Clear and concise explanation! Thanks!

@raghavendrabhandari6150

3 жыл бұрын

Yes, its clear and very easy to understand.

@OmkarShetkar

3 жыл бұрын

@@raghavendrabhandari6150 true

Well explained.Thank you for this explanation

Very nicely explained. Thank you

You created topic in Kafka, and I belive Kafka is running on some port. How publisher and subscriber is communicating with Kafka server to produce/consume from topic, we did not provided any configuration from producer/consumer side?

Thanks for this. It's a shame the @EnableBinding is already deprecated and there is no documentation on referring to "@deprecated as of 3.1 in favor of functional programming model". Doesn't seem to be many reliable resources to go from this.

@baptiste6436

2 жыл бұрын

did u find out how to replace it?

awesome broo.. waiting for this tutorial.. clear explanation

@Enablebinding is deprecated now.What is the alternative for this?I want to build a kafka consumer in spring boot.What are the options?

@ambersingh1493

3 жыл бұрын

Same query.

Great video.. Thank you !!!

Thank you. This is quite detailed.

@ifesinachieze9360

3 жыл бұрын

Hi @Java Techie, I have a specific question. Lets assume that for some reason, your consumer application was down. How would you know that the message sent through the message bus (kafka) has not been consumed and how do you mitigate around that?

@Javatechie

3 жыл бұрын

It doesn't matter whether your consumer is up or not once message received to topic , whenever your consumer will listen to it immediately he will receive events

@ifesinachieze9360

3 жыл бұрын

@@Javatechie, thank you. I will check this out.

kindly do a video on spring cloud streams with supplier, function and consumer

Hello @Basant, Thanks for posting such an informative video in great detail in a simple way. Just one small doubt, When the consumer was down initially, your first request from producer should have parked to the topic (18:39), but was not reflected when the consumer was up (18:55). Did we lose the message published initially, consumer should automatically have picked up the message when started which is parked at 18:39? Please guide.

@Javatechie

2 жыл бұрын

Consumer should automatically pick-up when back to online buddy

Hi Basant, I like to watch your all youtube videos. I have a recommendation for you and it will be quick and easy - Can you please dockerize these projects? (current video) producer consumer of apache kafka with spring cloud on DOCKER?? On Windows OS(please)

@Javatechie

4 жыл бұрын

Ok I will do this

Hi Java Techie, your all videos are worth trying. Could you please make a video on recent spring cloud stream (3.1.3) with Kafka because certain annotations which you have used like EnableBinding, StreamListener are deprecated .

@Javatechie

3 жыл бұрын

Hi Asha , definitely I will try with update one

@subhrajeetpadhy

2 жыл бұрын

@@Javatechie Dear, Thanks a lot for amazing kafka videos, it would be too good if you make a video as per above request.

@Javatechie

2 жыл бұрын

@@subhrajeetpadhy sure buddy will do this

Hi @EnableBinding is deprecated , can u please let us know what is the alternative we can use.

Amazing and very helpful, thanks so much... it's just superb. I would like to achieve exactly the samething but using docker in stead of running localy. Do you have a tutorial that explores something of that nature?

@Javatechie

Жыл бұрын

using docker i haven't tried

Thanks for accepting my request

Very useful videos, Where to add kafka properties.

@Javatechie

2 жыл бұрын

In application.yml

Hi, You might have forgot to link Apache kafka link in Description.

Nice video . i have one question here. if I need to use more than one object to consume. how can i do it? .

@Javatechie

3 жыл бұрын

Publish list of object in source and consume list from sink

Very Informative . Can you show how to configure multiple bindings is a same service .

@Javatechie

4 жыл бұрын

Yes will definitely try

@hootanv1074

4 жыл бұрын

@@Javatechie Can you also show a case where a Sink/Consumer can also publish to another topic after it is done processing? Great video!

Sir, pieces other than the binder looks tightly coupled. In a real time scenario, we expect innumerous back and forth interactions between the microservices. Configuring one way messaging channeling seems quite expensive from development standpoint. Could you please suggest an alternative solution offering a balanced approach?

If we don't add source in binding in publisher whether consumer will not work ?? or if message already published by some one else and if I'm writing consumer with sink binding will i able to receive the message ??

@Javatechie

3 жыл бұрын

Your first question how consumer will consume the message if you won't publish it Source -- publisher Sink -- Consumer

Hello java Techie, Can you give explain also if kakfa is on other server and producer and consumer are on other server, I think we need to define configuration somewhere then so that prodcer and consumer can send message through that server

@Javatechie

3 жыл бұрын

Yes both are running in different . We could separate with 2 different application to understand it better way

Hi. Is it possible to show an example with manmual acknowledge functionality along with some other custom configurations just like Kafka consumer. I am not able to find any refrence.

Hi, In my project I am using spring cloud stream binder and azure event hub as broker. Everything works well for means able to publish the event. But when I tried to load test the producer using Jmeter, let's say for 1000 threads,app publishes some random number of events to event hub and after that it is throwing Emissions exception with message like doNext or doError must be signaled serially. Any idea on this.

@Javatechie

Жыл бұрын

Haven't tried with cloud messing queue Budd will check and update

How to configure consumer service for multiple topics and group?

We didn't mentioned here the kafka IP and host details, that also done required with the binder?

@Javatechie

Жыл бұрын

It's not required that's what the beauty of spring cloud stream. Based on your binder spring cloud is smart enough to identify what messaging channel you are using

@manishdadheech8440

Жыл бұрын

I really respect you sir. You always reply on comments thanks. What if Kafka server is on different server and application on different?

@Javatechie

Жыл бұрын

Then you need to configure bootstrap server details buddy but again that is one go solution

Hi @Basant Nice presentation and I am trying to do the same .But consumer is not listening to the message .I haven't added the iml file .Is there any reason not printing statement in those method

@Javatechie

2 жыл бұрын

Sink annotation is there ?

@jincyv7386

2 жыл бұрын

@@Javatechie yes .streanlistener(Sink.input) and enabled bindings for class and annotated with component.not showing errors but not entering to that method

What if we want to publish data on multiple topic from 1 microservice to multiple micro services from multiple controller, can we deploy kafka server on kubernates pods. Could you please explain the same using GCP pub/sub with multiple topics. And also how can we assured that the subscriber had execute for that payload, does we get any acknowledge id ? and can we see the unacknowledges messages in kafka server which are not passed to subscriber(consumer) ? And with the GCP i have faced the problem using PUB/SUB as until acknowledge come its executing the same message and in my code due to some error its never returning the ack due that the same message executed continuously please help me as i am not configure the pub/sub in my company appreciate if we can discuss over the google meets, Thanks

please post the producer error handling methodologies with DLQ

Hope this is same for STS? Instead of yml we can use application.properties as I am not clear why you require both files

@Javatechie

4 жыл бұрын

Yes we can use only one no issues

Getting this error when running producer while passing data through postman : Failed to convert message: 'GenericMessage [payload=com.creator.cloud.stream.api.model.....Can you help as there is a change in spring version I guess?

Thanks..Can you share video how can configure multiple brokers...

Where do we specify the kafka server details (host,port) this needs to connect

@Javatechie

2 жыл бұрын

Not required spring cloud will identify it based on binder you added in your pom

Hi Nice explanation, I am new to Spring cloud stream concept. I have one doubt i.e. can we use same input chaneel with more than one streamLinsters. Please clarify.

@samirshirodkarr56

Жыл бұрын

You can use but one of the running application will read the message

how to configure kafka..if it is an aws kafka(MSK)..do we need to use msk url in bootstrap_server_config ?

@Javatechie

3 жыл бұрын

Hi Abhishek am not sure , I believe we should use AWS provide messing queue like sqs

Is there any specific reason we have to create application.yml while we already have application.properties? Can't we specify the bindings inside application.properties ?

@Javatechie

4 жыл бұрын

It's upto you ☺️ .I feel good to use application.yml

Hi Basant, I have a question. On How do we get the data from a topic in a consumer application?

@Javatechie

Жыл бұрын

Usually we should get in the form of JSON string

@vinodkumarRavula

Жыл бұрын

@@Javatechie ok. how to read the data in the topic and respond/provide it in a REST API. ?

@Javatechie

Жыл бұрын

Please check out my Kafka producer and consumer example

`@StreamListener` is deprecated now.

Thanks for the video, if possible make an example with RabbitMq also, so that we can differentiate easily

@Javatechie

4 жыл бұрын

Will do

@raveendrareddy8900

4 жыл бұрын

@@Javatechie thank you bro

If we have 2 producer in one service and 2 consumer in another service thenhow will it work

@Javatechie

3 жыл бұрын

You need to define those info in application.yml bindings section

@aayushiagarwal8063

3 жыл бұрын

@@Javatechie Thanks for your answer but can you share example.

If you create a seperate class for publisher instead of putting publisher logic in main, do you have to annotate the @EnableBinding(Source.class) on main class or the publisher class?

@Javatechie

4 жыл бұрын

You need to add both Source and sink in @EnableBinding

Can we define multiple topics in the yml file. If we can then How can we choose specific topic with channel.

any reference for remote partitioning using Spring cloud stream ?

Bro you used spring cloud stream but not Kstream. Isn't it necessary to implement kstream while working with cloud stream. Streaming is done using Streams right ?

@Javatechie

3 жыл бұрын

Hi Satya , May I know purpose of using kstream here ?

@ss-ih7zh

3 жыл бұрын

I'm new to kafka .i implemented your video but when i started to read kafka streams everything code is done creating by using kstream.that why i was asking

Hi bro .. m trying to follow your code But here .. @EnableBinding has been deprecated .. so is there any alternative way ..pls suggest

@Javatechie

3 жыл бұрын

I didn't check update version

If we have more than one controller in publishers and more than one consumer method how it will execute it's Respective consumer method

@Javatechie

4 жыл бұрын

In consumer we are using stream listener right so you need to specify channel name

nice one . can we configure multiple topic channel in consumer side ?

@Javatechie

4 жыл бұрын

Yes we can

@anantababa

4 жыл бұрын

@@Javatechie can you please give some sample code for this . i have some requirements..

@anantababa

4 жыл бұрын

@@Javatechie can you please give your contact details.

If possible make video on kafka async produce and consume, it helps us alot.

@Javatechie

4 жыл бұрын

Definitely I will check this out

@raveendrareddy8900

4 жыл бұрын

@@Javatechie thank you Bro

Kinda weird to see using Lombok without @Log or smth when u need logger

@Javatechie

11 ай бұрын

@Sl4j is enough

Can you make a video to integrate kafka with spring boot microservice architecture on kubernetes?

@Javatechie

3 жыл бұрын

I am not much aware about kunernates

@weitanglau162

3 жыл бұрын

@@Javatechie No problem! Please keep making great videos on Java Concepts! :)

@Javatechie

3 жыл бұрын

Yes Lau I will do

Please video on rabitmq with UI, thanks

@Javatechie

4 жыл бұрын

Will do 👍

Where do we add broker details which is running remotely.

@Javatechie

3 жыл бұрын

Same in application.yml

Tried this IDE not working with spring boot but for your it's working like anything,is there any video for this?

@Javatechie

4 жыл бұрын

It will work add spring assistant plugin on it .

@thannasip8001

4 жыл бұрын

@@Javatechie tried this way bro if failed for recently IDE version 2020.2 think,working version please?

@Javatechie

4 жыл бұрын

I was using 2019.3

@thannasip8001

4 жыл бұрын

@@Javatechie thanks bro,installed same version and spring assistant plugin as well am getting spring initializr option but again autocomplete not coming for application.yml

@Javatechie

4 жыл бұрын

Can we connect in google meet javatechie4u@gmail.com . It looks Some configuration problem

whrere you are specifying broker details of kafka?

Where do we mention the kafka server running ip and port?

Is github link for this application accessible? I am not able to access it. Can someone share the working github link for this application

@Javatechie

4 жыл бұрын

Yes it's public repository everyone can access . Login to GitHub then try to access it

Sir can you tell how to retrieve the data from to two different services at a time please.

@Javatechie

4 жыл бұрын

Hi Yogesh , Are you trying this using rest API call ?

@yogischannel9972

4 жыл бұрын

@@Javatechie yes I am trying rest api

How can we use db

Can we bind source and sink class in the same application?

@samssunny16

2 жыл бұрын

I have one more doubt like how to configure multiple topics in producer and consumer

How to configure more than one destination topics

Could you explain what is cloud pubsub?

@Javatechie

4 жыл бұрын

I never tried this, will check and let you know

are you working only with java?

@Javatechie

4 жыл бұрын

Yes working on Java

Please help me on this , I have 2 methods with separate api call and both are consuming the same topic. when I publish data, only first method is calling, 2nd is not calling. what I need to do.- Sample code: @StreamListener("inputchannel") public void consumeMessage(String message){} @StreamListener("inputchannel") public void consumeMessage2(String message){}

Awesome