Want to learn more Big Data Technology courses. You can get lifetime access to our courses on the Udemy platform. Visit the below link for Discounts and Coupon Code. www.learningjournal.guru/courses/

Very good videos. Loved it. The sound quality is good as well. Thanks Learning Journal!

One of the best tutorials Thank you for sharing

Man, you are awesome....please keep sharing the knowledge

Must say Prashant has an amazing knack for teaching and making complex topics more simpler and understandable to all audiences, please keep up the good work Prashant, wish you all the best and keep it coming.

Great stuff man! Thanks!

great video, straight to the result.

nice work sir - great work sharing information :-)

Hi sir, your videos are amazing. Could you also make a tutorial on Apache kafka integration with Apache flink and streaming it.?

great video

very informative video and channel .

Hi sir, can you please make a video on how to install various big data components in Google data proc e.g nifi or flink

Thank you so much sir for the Kafka tutorial. Could you please do a tutorial on integration of Kafka with Spark streaming

@ScholarNest

6 жыл бұрын

Yes, that will be part of my upcoming Spark streaming playlist.

Hi can you please make a video on..how to setup kafka in cloudera quickstart vm. That will be really helpful.

thank you so much, Sir, for this video. please, can you Explain how we will take data from other sources like Sensor data and how we will use that data to get the reliable result through Spark Streaming?

@ScholarNest

6 жыл бұрын

That's a great question but hard to answer as a comment reply. Sensors are programmable devices. You can push simple program to your sensor devices. These programs can send data to an IoT gateway over MQTT or HTTP protocol. All cloud providers like Google and AWS, both of them offer IoT gateway service. Once data reaches to IoT gateway, you can configure the gateway to pass it to Kafka. That's it. Rest is Spark streaming over Kafka. I will create tutorials on this last part. I can show a demo on first part also but I don't think I have funds to invest on that.

can I use AWS instead ?

Sir i am unable to start kafka-server it says permission denies. I am using RHEL server in my local machine

Sir iam a beginner in Kafka. I need to implement it in my project. We can expect some 200k records in the mysql user table. I have to track User events like User Creation, User Address updation, User Email address updation etc.... I have to integrate Kafka producer here. The important requirement is we have to maintain the ordering of events. I have 2 clarifications sir, 1. Is it to good to have one topic called Users with multiple partions based on the Users role. The idea of having more partitions is to make the solution more scalable. 2. How many Kafka brokers we can have in Accepatnce and Development environment to make this an optimal solution.

Hey Great tutorial. i have a question: i have a Node js server configured to the Google cloud Kafka server and in the url i put the External IP and i get an error 'host resolution failure', what could be the reason?

@eliavmaman3203

5 жыл бұрын

also the port is 9092

What is the difference between Kafka streaming and spark streaming ? How to stream customized website data using Kafka ??

@ScholarNest

6 жыл бұрын

Windows phones were good. But we failed them and embraced Android. That's called IT consumerization. Kafka stream is graduating to become a Spark streaming competitor. It's we, developers and architects who are going to choose a winner.

I like your videos, I need Kafka version is 2.8. Please share link for same.

sir am getting timed out exception while creating topics. , no solution in goole also. pls help. Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING

Any guidline or documentation to install kafka on openshift3.11.

I'm getting this error on the fist command: WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092 ) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient).

Where does the messages get stored in the consumer ?

@ScholarNest

5 жыл бұрын

Messages are stored at the broker in the log partition segment files.

Can't we use Apache Kafka on windows 10?

@ScholarNest

5 жыл бұрын

We can. I have a video in Kafka Streams playlist.

1. We can have a consumer facade in the consumer group which talks to Kafka . The only responsibility it has is to distribute workload among other consumers. This way we can autoscale consumers ? 2. Don't we have autoscaling mechanism for Brokers ? Seems like we need to provide number of brokers at the start of server. ???

i am using kafka 2.1.1 on GCP but i am unable to create a producer. i am getting below error. "zookeeper is not a recognized option". i tried with both zookeeper and bootstrap-server. it didn't work

@rajeshdamam8861

4 жыл бұрын

Try this one: kafka-1-vm:~$ kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testCreated topic "test" kafka-1-vm:~$ kafka-topics.sh --list --zookeeper localhost:2181 test Starting producer and consumer both: kafka-1-vm:~$ kafka-console-kafka-console-consumer.sh kafka-console-producer.sh kafka-1-vm:~$ kafka-console-producer.sh --broker-list localhost:9092 --topic test >This is a test message form Raj kafka-1-vm:~$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning This is a test message form Raj

we have to change the command to kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test if you use version older than 2.2 you should use --zookeeper option and pass connection string to zookeeper

@surbhiagrawal3951

4 жыл бұрын

thanks my error is resolevd through your command ..i have created a topic

When i ran the command got below error kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test Exception in thread "main" joptsimple.UnrecognizedOptionException: bootstrap-server is not a recognized option

@surbhiagrawal3951

4 жыл бұрын

i am also getting same error ,,, can you guide how to resolve

This was going great untill i realised that you get error with every command you try:( e.g. i cannot create a topic,i get an error every time.Please help!

@yassinebenhadi4883

4 жыл бұрын

Here is the fix: stackoverflow.com/questions/55494988/why-is-kafka-not-creating-a-topic-bootstrap-server-is-not-a-recognized-option