Model Calibration | Machine Learning
Ғылым және технология
Machine Learning models are great at many tasks. However, one of the biggest challenges is that these models are not calibrated. Watch the video to find out what we mean by calibration for machine learning models and why everyone care about it.
Пікірлер: 42
Very clearly explained. Short and to the point!!
@TwinEdProductions
3 жыл бұрын
Much appreciated! Thank you
Thanks! That's the first explanation I really understand.
@TwinEdProductions
2 жыл бұрын
Glad to hear it :)
You explained it the best way possible ...thanks... 👍👍👍
@TwinEdProductions
2 жыл бұрын
Thank you for your support!
very well-explained - cats and dogs examples are the best!
@TwinEdProductions
2 жыл бұрын
Thank you! Cute animal pictures always helps :)
Great video. Very concise and intuitive :)
@TwinEdProductions
2 жыл бұрын
Thank you!
Well done, thank you!
@TwinEdProductions
2 жыл бұрын
Thanks!
What if they thought me this 5-10 years ago!!!! Really valuable presentation!!!! STAY BLESSED!!!!
@TwinEdProductions
2 жыл бұрын
Thank you!
This tutorial calls for an immediate subscription
@TwinEdProductions
2 жыл бұрын
Thank you very much!
Great explanation, thanks!
@TwinEdProductions
Жыл бұрын
Glad it was useful!
excellent !
@TwinEdProductions
Жыл бұрын
Thanks :)
Awesome 😉
@TwinEdProductions
2 жыл бұрын
Thanks :)
So good.
@TwinEdProductions
Жыл бұрын
Thank you!
Very well done ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
@TwinEdProductions
3 жыл бұрын
Thanks!
so cool,but how should we draw a similar coordinate system for a multiclassification problem?
@TwinEdProductions
Жыл бұрын
Potentially you can do a 1-vs-all approach for each class in turn
Question - if the model curve is below the well calibrated curve, why should we say it is not well calibrated? The well calibrated blue curve seems to me having a worse prediction performance as it will even give 10% at 0.1, 20% at 0.2, 50% at 0.5, while the orange curve gives 0 at 0.1, 0 at 0.2, and 0.25 and 0.5.
@TwinEdProductions
4 ай бұрын
Hi. So a calibrated system and an uncalibrated system have the same performance (typically)! For example, imagine a binary classification system which is very overconfident such that it predicts everything as either 0.01 or 0.99. By calibrating such a model, we are not changing the model performance, but just smoothing out the distribution so that the output values are continuously distributed throughout the whole range such that the output probability value is actually meaningful.
@hangchen
4 ай бұрын
@@TwinEdProductions Thank you for your response! Just in time! I was asked about model calibration during an interview yesterday! I will study more about it! Thanks again and have a great day!
@TwinEdProductions
4 ай бұрын
@@hangchen Good luck :)
@hangchen
4 ай бұрын
Thank you so mcuh! 👍@@TwinEdProductions
@souptikmukhopadhyay6531
2 ай бұрын
You didn't answer his question, the orange line is better and more accurate ,... calibrated. Model is changing accuracy by force fitting to blue line
very nice, however i want to make suggestion, in x axis the probability values of each bin are averaged
@TwinEdProductions
2 жыл бұрын
Thanks for your comment - yes we completely agree with you :)
@panosp5711
2 жыл бұрын
@@TwinEdProductions you should point out this in your video
Awesome. But how to do calibration if we dont gave data .my dataset has only 977 records.so i did train, test split. All data is used up. But calibration model has to be fit using new data. Where and how can i do calibration in this case?
@TwinEdProductions
2 жыл бұрын
Hi Akshay, thanks for your comment. If I were you, I would split the data into train, validation, test. 100 records or so should be more than enough for a validation set and hence you can calibrate using this validation set. If you are really worried about not having much data, then you could consider an n-fold cross-validation approach instead. Hope that helps!
Whoops, left the previous title card by accident :D
@TwinEdProductions
3 жыл бұрын
Thanks for the pop filter tip - hopefully you can tell the difference!
@softerseltzer
3 жыл бұрын
@@TwinEdProductions Yes, the audio is definitely crisper and better!