Java Concurrency Interview: Implement Producer Consumer pattern using wait-notify
Implementing Producer Consumer using BlockingQueue, Locks/Conditions and Wait-Notify.
Important: The last part about using wait-notify is incorrect. My mistake, sorry about that.
The object used to wait/notify should be the same object used by threads to synchronize (to avoid IllegalMonitorStateException). So correct code should be similar to:
synchronize(sharedQ){
sharedQ.wait();
}
synchronize(sharedQ){
sharedQ.notifyAll();
}
That was a basic mistake which I should have caught. I feel bad about misleading the initial viewers. Sorry.
Channel
----------------------------------
Master difficult programming concepts in few minutes. I try to explain difficult concepts like Java concurrency in simple to understand manner. Explore videos on topics like Spring Boot, Cloud Foundry, Java 8 and more. I am happy to clarify your doubts. Ask me anything in the comments. Also happy to take requests for new videos.
New video added every Sunday.
Subscribe or explore the channel - bit.ly/defog_tech
Current Playlists
----------------------------------
Java Executor Service - bit.ly/exec_srvc
Java Concurrency - bit.ly/java_crncy
Spring Boot 2.0 - bit.ly/spr_boot2
Java 8 - bit.ly/java_8-11
Intellij IDEA Shortcuts - bit.ly/i_idea
Popular Videos
----------------------------------
Executor Service - • Java ExecutorService -...
Introduction to CompletableFuture - • Introduction to Comple...
Understand how ForkJoinPool works - • Understanding how Fork...
Java Memory Model in 10 minutes - • Java Memory Model in 1...
Volatile vs Atomic - • Using volatile vs Atom...
What is Spring Webflux - • What is Spring Webflux...
Пікірлер: 215
Important: The last part about using wait-notify is incorrect. My mistake, sorry about that. The object used to wait/notify should be the same object used by threads to synchronize. So correct code should be similar to: synchronize(sharedQ){ sharedQ.wait(); } synchronize(sharedQ){ sharedQ.notifyAll(); } That was a basic mistake which I should have caught. I feel bad about misleading the initial viewers. Sorry.
@VIRAJBHOSLE
5 жыл бұрын
One more issue with the code is that Condition solution should use methods on Condition not Object. i.e. await signal. Not wait signal. Reference "Java Concurrency in Practice". Only one slide was wrong, no problem. Otherwise great video! Thanks Man!
@viktorvostrikov9625
4 жыл бұрын
I like your courses, however I don't understand this comment.. I keep getting exception about illegal monitor state.. what does sharedQ mean? Please explain or provide full code, which works :)
@viktorvostrikov9625
4 жыл бұрын
can you provide source code with corrected wait/notify?
@singhsaurabh920
4 жыл бұрын
can u provide code of wait and notify section
@singhsaurabh920
4 жыл бұрын
can i call wait and notifyAll on this
Even years later, thank you for this video, it 's very useful and helped me finally understand this topic... sincere thanks
@aasadullin
Ай бұрын
fr
I generally don't put a comment for any video until I really like them. I must say that u explain each and everything very clearly. I don't have hands-on experience on multithreading and concurrency but with your videos, I find myself so comfortable with all these ..Keep uploading .after a long time I liked something good on youtube :)
@DefogTech
5 жыл бұрын
Thank you so much for your kind words! Such wonderful feedback keeps me going too
your videos are crisp and clear! that's the way to talk about complex topics.. thank you so much man!
this video has cleared a lot of my doubts and the way you have explained will help me remembering it for a very very long time ! Thanks for your hard and crisp work !
This channel teaches more about the approach to solving the problem.They don't go to the solution straight away,first they come to common implemented pattern ,than they optimize the solution. Brilliant
GENIUS !! No other words. Such a smooth and clear explanation to the problem
Hi @Defog Tech, This is the my first comment on youtube in last 5 years, Your videos are very helpful, I have really understood the Java concurrency on your channel. Please keep uploading such awesome content. Thanks
Thank you for making this video, I watched it some few years back and I still came back to re-watch it in 2024. Thanks a lot for such a great teaching.
Easy explanation of any topic. I really like your way of explaining any topic.
first time learning about producer-consumer. your explanation is very clear and easy to understand
Hi , I have been following all the videos from you which are really great in content with clarity . Would love to see more from you .
Brilliantly explained! It couldn't be better in any aspect.
your hard work really appreciated expecting more Complex multithreading interview Q&A
this is the best video I have seen ever for producer consumer problem ..thanxx man
this is the only channel that make me realize multithreading can be learnt thanks a lot man keep making more of such gems 😀
No new tutorials for a long time on threads, May God bless you. Will try to share your channel with my mates, but please do not stop illuminating us.
Best clear explanation of a very complicated subject
This channel always have unique topics and content.
Greetings from Russia , having watched hours of content on concurrency/multithreading , this is the by far the best one i have seen , top notch.
A perfect explanation I had ever seen. Thanks a lot!
Best videos of all time and even youtube understood that and suggesting on top of everyone
I saw your correction. The whole thing is now very clear. Thank you.
really liked the way you explained. thumsup. God bless you
You made it so easy to understand. Thank you very much!
BEAUTIFUL! Spoken way better than my professor
Hi Deepak, Please start posting new videos. Your way of explaining the things in simple language helps to understand the concepts better. Great job! Looking forward to learn from your new videos
Just started with going through Defog videos on Threads,Concurrency, etc.. Really a very clear explanation on these complex concepts. Also, could you please paste the final code you have implemented in the slides
Thanks for this video. Very beautifully explained.
Bro you are best guru out there . Put more contents your channel will bloom
Wonderful explaination!!! Thanks a ton.
you are the best.. please keep up doing the good work..
Great explanation on producer and consumer using different ways. It makes the idea really clear. I was wondering if there is any way to handle the fairness for consumers meaning if any consumer goes to wait-state first he will be notified first.
Great videos. it makes it easy to understand complex concepts. Though I have confused about using the while loop instead of the If condition. It could have been better to explain issues with if condition here and then introduce a while loop. anyway amazing explanation with exaples
Neat and clear explanation...
Very well explained thank you !
Nicely explained. Thank you
Great Video, keep up the good work!
8:50 instead of checking if queue.size==0 and then waiting on the condition, we must check that in a while loop because after resuming from the condition, another thread could have resumed before us, and the queue.size that we checked before would be outdated (the other thread that resumed before us took what was in the queue already)
@murali1790able
Жыл бұрын
why should we allow two threads to be in waiting in the first place? could you explain this? basically why java releases the lock after lock condition is false?
@DuyVu-br1do
7 ай бұрын
actually when it reaching the wait() at if statement, the the consumer is in blocked state and added to the waiting set. after being notified by the producer, it accquire the lock again and continue. If using while -> then continue checking the queue size since the next execution is under the wait() statement. If using if -> then jump out of if, then execute the remaining code. As mentioned, the next execution will be under the wait() statement.
I love your tutorials! I did this example myself and found a little but meaningful bug: -notEmpty and notFull should be called ".signal()" instead of ".signalAll()" because in case of releasing one spot in the queue by take() method, if there are multiple "putting" Threads, they are all notified at once about free space and they add concurently its own item exceeding the maxSize (and later condition `queue.size() == max` doesn't catch situation when this size is exceeded)
@6365bharath
Жыл бұрын
Great spot. Even I thought of the same.
@akashbiswas2833
6 ай бұрын
If we call signal() then the longest waiting thread will be notified and the rest of them will still be in a wait state. When they are notified they access the lock again and do a one on one insertion or removal of item.
thank you for your videos they are helpful!! quick question, it might be better to use await instead of wait in the try block to avoid the monitorexception thing.
Could you elaborate on why we used while loop by removing the if condition? Thanks for the videos, they are very concise and up to the point. God Bless and Good Luck :)
I really want to thank you for this video, It taught me a lot of things Blocking queue, await-signal, wait-notify. But at the same time, the last mistake wasted a lot of time of mine. I should have seen this comment earlier. If possible, can you please mention this on the video itself. Maybe as some floating suggestion.
More videos from you will definitely help understand the technical concepts with clarity and ease. I know it will be little challenge for you to post more and more videos on core java,etc, while doing your regular work. But, please upload more videos which will be of great help. Keep doing the good work.
@DefogTech
4 жыл бұрын
Thank you for the encouragement! I sure am planning to restart uploading first week of Jan 2020!
@VijayKumar-vv6yw
4 жыл бұрын
@@DefogTech Any update on this?
@DefogTech
4 жыл бұрын
@@VijayKumar-vv6yw sorry sir, was stuck.. can you please let me know the topics you are interested in... sorry about the delay!
@VijayKumar-vv6yw
4 жыл бұрын
@@DefogTech Dude , I understand. Need not say sorry. We are eagerly waiting fo your content. I can consume any content from your channel. The explanations are so lucid that it gets stuck. I am more interested to see videos on distributed transactions and problems of joins in microservices and how to overcome them.
Thanks for the video.. Nice explanation.. it would have been even more helpful if u had come up with examples and execute them..
Best java channel ever
very very logical explaining..thnkx brother for ur wonderful efforts..
Great video! Thanks for the explanation. I checked your description regarding wait() and notifyAll() and understood. However, I have one question, whenever we call sharedQ.notifyAll(), it will wake up all producer and consumer threads waiting on sharedQ. Is it possible using wait/notify method, that producer thread wakes up only consumer threads and vice versa, consumer thread wakes up only producer threads?
@Defog Tech what is the differce using the "while" or "if"?Because both threads are wait state after the condition. Is "while" re-check the statement?
omg such a simple explanation for a concept we always are reluctant to read... i dint know it was too simple...one query though when replaceing the await/signal with wait/notify , in the code snippet i dint see the usage of while. is it not required to put the "while" instead of "if"
Clear explanation!!
Thanx u bro very nice explanation.
Super, This is the way to describe a tough subject like concurrency. Before watching this video i dont have any interest in concurrency. Now i think that its not so difficult. Thanks a lot
synchronize(sharedQ){ sharedQ.wait(); } synchronize(sharedQ){ sharedQ.notifyAll(); } I am not getting these words In wait and notify section Could u please explain the code i am having trouble in this part ? java.lang.IllegalMonitorStateException at java.lang.Object.notifyAll(Native Method)
@9.04, how two thread could come at that point since we called lock above. I think only one thread will be in await and when the notEmpty.signalall is called that thread will remove the item and call the unlock so the other thread can come inside and do the if check and can go to the await state. What is the need to while there?
Hi, You used while loop so as to restrict more than one consumer thread get into race condition when an item is put into queue. You are also using lock.lock() above... Do you think we really need while loop? and Do we really need lock.lock() when we are using conditions... Thanks.
Bro, you rock. tks a lot!
Another Gem of a video from you on java Concurrency. I've a question though, How does changing if condition to while would solve the problem ? Can you please elaborate ?
Hi, I have one doubt. At 8:52 How can two threads simulatenaosly access the code put after acquiring the lock? Am I missing anything?
@studisthinics
3 жыл бұрын
I had the same question in mind. Putting that while condition seemed unnecessary and impossible scenario.
@stivstivsti
3 жыл бұрын
It looks like await is taking the lock off. But how then it is going to protect integrity of the queue?
@sajjan09028
3 жыл бұрын
Correct ... it's not possible.
@harish-wi3ts
3 жыл бұрын
samething i got.
@talesara74
3 жыл бұрын
Same question..if lock if there an if would have been sifficient.
Just AMAZING.
@Defog Tech i guess it should be while in put method also instead of if.Use case : when queue is full , two producers(A,B) try to put elements then they go into waiting, now one consumer wakes and take one element out. Then producer A will put one element and as soon as it leaves synchronized block producer B will try to insert and that will be successful which will exceeds queue size. Please confirm.Thanks for the explanation.
How can we add a new item if the lock is held by take method and we can't enter the put method?
The solution of locks I think is only suitable if we have multiple producers or multiple consumers. In case if there is single producer and single consumer we dont need to make put and remove method synchronized right? @Defog Tech
At 10.06 even an if would have checked size of q for the new thread..why there is need of while?
Option 3 with semaphore would have been great. Thank you.
Which is the better way of implementation of blocking queue? "Lock and Condition" or "Wait-Notify" and why?
Awesome Video
earned a sub !! Thanks a lot.
Hi @Defog Why there is no while condition on put method It will have the same problem in adding the item in max index position Pls correct me if iam wrong
I have similar problem where two threads need to read element by element.. i tried same concept but will not work. Two threads started but not moving further
Very Good and clear explanation! Can you make a video on Advanced Collections and how to correctly incorporate them in your System design?
@DefogTech
5 жыл бұрын
Sounds like a good idea, any specific collections you are thinking about?
@pavitrakannan2080
5 жыл бұрын
@@DefogTechThanks so much for replying back. Some thing along the lines of Internal workings of a hashmap. When to use a list over a set. Performance in a multi threaded environment of these interfaces.
Great video! But I think there's a mistake at 9:37. As per official docs when the waiting threads are woken up they must re-acquire the lock to come out of await(). So only one thread(consumer) can poll from queue. So no NullPointerException will be thrown! No need to check queue size after await :)
@ravipasumarthy8732
Жыл бұрын
Even I was thinking the same, otherwise it defeats the purpose of actual lock itself. "Signalall" means to just notify all waiting threads. The jvm will ensure only one thread is given the lock.
@hamidchemlali9168
11 ай бұрын
you're wrong
Awesome man !!!
why cant we use a semaphore instead of lock? we can restrict access to resources that is in this case our queue.
I still don't understand how can two consumer threads wait on the condition because we have been using a lock over that method in the first place.
very clear , thanks
If we are taking same lock in both put and take method, then wouldn't it be that only one thread will do either put or take. If a thread is in put, no thread can go in take. And vice versa. Is that possible?
I like to add my humble opinion here. The while loop in the "take()" may allow N number of threads to enter the next line and which is removing the element. The first thread will remove the element successfully but the remaining threads will be ends up in Null. I would suggest using a slightly modified "Observer design pattern".
@vetri_vel
3 жыл бұрын
Thanks for this interesting Java Concurrency series.
But why the while loop is the solution for return null from collection. Previously it was if statement which is checking for size and both threads successfully pass this check. Why when we use while for check the size behavior cannot be the same and how it passed the if statement if only one thread can join the lock?
@sugarpatro
3 жыл бұрын
The await condition suspends the lock.
Why does java releases the lock when lock state is false. does this allow multiple threads to come in and wait? Because of this we need to solve the problem where multiple consumers trying to read the same item.
Just a question sir , at 09:00, how come 2 threads be at await conditions .. since its inside lock . I presume that only one could go inside, take lock and await and others be waiting for lock only
how can 2 threads be in a lock.lock() block? time 9:05 on the video.
When Using ReentrantLock why not same object we are using for lock and unlock.
Hi Deepak, The complete code for implementing with Locks & Conditions @ 10:10 doesn't reflect the while loop which was discussed @ 9:50. Apologies if I missed something.
@DefogTech
5 жыл бұрын
You're right. I missed it in the final code. It should have while loop. Thanks for pointing it out!
Brilliant
I am getting illegalMonitorstateException. Any idea when it occurs? I followed the same approach.
Hi, According to one of your explanations (Regarding the thread waiting) when one thread gets into waiting pool, other thread can access the code within the lock part ? Is this behavior only available for Locks or also available for synchronized ?
@DefogTech
5 жыл бұрын
Locking works same as synchronized, so yes will be applicable for both.. though I made a mistake in wait notify solution.. please check description or first comment
Good sir 👌👌
Hi Defog_Tech. In case of multiple consumer threads at 9:00, how does thread-1 even get past the lock.lock() statement? Only thread-2 will go past that... I did not get how both threads were trying to execute near the try-block. Doesn't acquiring lock synchronize the method??
@murali1790able
Жыл бұрын
he said, java will release the lock once lock condition is false, this is both threads end being on the same statement. I also ddin't understand why java releases the lock.
There is a problem. When we have our queue full and it's in await state then the consumer consumes one and signals all. in that case, if we have multiple producers then our queue size goes beyond the max capacity. Try this with 3 producers and 1 consumer.
Great explanation! Good job man.
This is a great explanation, but at 9:52 even though u add while loop there, thread 2 go to it,then the size of the queue is not empty ,then it go to remove the item and return,thread 2 will not go back to while loop again!so i hope u can explain it more,thx mate
@murali1790able
Жыл бұрын
agree, he did poor job at 9:00
Awesome.
Your explanations are wonderful! Thanks a lot! I have just one comment.. You are calling "wait()" method on both "notFull" and "notEmpty" conditions. I guess you've meant to call "await()" instead, right?
@DefogTech
4 жыл бұрын
yes thats right..
@chenyangwang7232
4 жыл бұрын
I was also wondering why await() suddenly becomes wait()
One more comment.. :) Why do we need to notify each time we are consuming or producing? Isn't it better to query if the queue has a single element in case of consumer and if the queue has a single index available before notifying? Thanks again..
@DefogTech
4 жыл бұрын
But since threads can work independently and time taken for producing an item and for consuming (processing) an item can vary.. it's bwtter threads notify each other instead of them constantly polling for available items
@user-em6qj4qx8h
4 жыл бұрын
@@DefogTech Let me explain myself.. I think that the following consumer notification: notFull.signalAll(); should be wrapped with the following checking: if (queue.size() == maxSize - 1) { notFull.signalAll(); } or. on the other hand, the notification of the producer: notEmpty.signalAll(); should also be wrapped: if (queue.size() == 1) { notEmpty.signalAll(); } otherwise we are notifying the other threads many times where most of the notifications are not necessary (say for example if consumer took an item and now there are more than one index available, why does he need to notify the producer which wait just for one available index?) thanks
What are the advantages of option 1 over option 2 and vice versa
@DefogTech
5 жыл бұрын
both are the same.. I like to use Locks and Conditions because they are newer (and locks have advantages of synchronized)
Thanks man.....
At 9:16, I am confused. When thread 1 acquires the lock after thread 2 has released it, the thread 1 will then check the statement "if queue.size() == 0" and in this case it is empty so thread 1 will wait at notEmpty.await() step. Am I missing something?
@DefogTech
5 жыл бұрын
In case of 2 readers, 1 thread will acquire the lock, find that queue is empty, will wait and release the lock, after that, thread 2 will acquire the lock, find the queue empty, will wait and release the lock.. Thus we can have 2 threads both will continue after the if condition once the condition is satisfied. They dont restart from above the if statement, they only continue from where they left off.
@hughgrant8749
5 жыл бұрын
@@DefogTech thanks much, I get it now.
@hughgrant8749
5 жыл бұрын
Wait, just look back the code. It is ReentranceLock, so we can never have 2 threads enterring that block or reaching that if statement at the same time
@DefogTech
5 жыл бұрын
@@hughgrant8749 They are not going in at the same time. But both of them continue (one after the other) after the if statement. So first thread will get the item, but second thread will get null. Both will acquire the lock properly. First thread takes lock, gets element releases it. Then Second thread takes lock, gets null..
@_oo.monkeypox.oo_3344
5 жыл бұрын
@@DefogTech while the first thread is in the finally block releasing the lock, the second thread is waiting at lock.lock() line. after first thread releases it, the second one hits the if condition and waits because size is already zeroed by first. why do we need a while here?
Do you have all this code in Git ?
At 9th min, where you replaced "if" with a "while", I personally feel its unnecessary, because the second thread will go in await as the queue.size() will be 0. It will not get any null value or I am missing something.