Model Calibration | Machine Learning

Ғылым және технология

Machine Learning models are great at many tasks. However, one of the biggest challenges is that these models are not calibrated. Watch the video to find out what we mean by calibration for machine learning models and why everyone care about it.

Пікірлер: 42

  • @ajitsekhar2716
    @ajitsekhar27163 жыл бұрын

    Very clearly explained. Short and to the point!!

  • @TwinEdProductions

    @TwinEdProductions

    3 жыл бұрын

    Much appreciated! Thank you

  • @iosifguzeev4362
    @iosifguzeev43622 жыл бұрын

    Thanks! That's the first explanation I really understand.

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Glad to hear it :)

  • @spyder2374
    @spyder23742 жыл бұрын

    You explained it the best way possible ...thanks... 👍👍👍

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thank you for your support!

  • @elvenkim
    @elvenkim2 жыл бұрын

    very well-explained - cats and dogs examples are the best!

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thank you! Cute animal pictures always helps :)

  • @chaerinkong5303
    @chaerinkong53032 жыл бұрын

    Great video. Very concise and intuitive :)

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thank you!

  • @ocamlmail
    @ocamlmail2 жыл бұрын

    Well done, thank you!

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thanks!

  • @user-bz7fj1fk2m
    @user-bz7fj1fk2m2 жыл бұрын

    What if they thought me this 5-10 years ago!!!! Really valuable presentation!!!! STAY BLESSED!!!!

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thank you!

  • @logicboard7746
    @logicboard77462 жыл бұрын

    This tutorial calls for an immediate subscription

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thank you very much!

  • @OscarBedford
    @OscarBedford Жыл бұрын

    Great explanation, thanks!

  • @TwinEdProductions

    @TwinEdProductions

    Жыл бұрын

    Glad it was useful!

  • @user-kv4he4uf8e
    @user-kv4he4uf8e Жыл бұрын

    excellent !

  • @TwinEdProductions

    @TwinEdProductions

    Жыл бұрын

    Thanks :)

  • @happygupta1580
    @happygupta15802 жыл бұрын

    Awesome 😉

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thanks :)

  • @SM-mj5np
    @SM-mj5np Жыл бұрын

    So good.

  • @TwinEdProductions

    @TwinEdProductions

    Жыл бұрын

    Thank you!

  • @qasimarthuna9254
    @qasimarthuna92543 жыл бұрын

    Very well done ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅

  • @TwinEdProductions

    @TwinEdProductions

    3 жыл бұрын

    Thanks!

  • @user-cc8ll8sn4e
    @user-cc8ll8sn4e Жыл бұрын

    so cool,but how should we draw a similar coordinate system for a multiclassification problem?

  • @TwinEdProductions

    @TwinEdProductions

    Жыл бұрын

    Potentially you can do a 1-vs-all approach for each class in turn

  • @hangchen
    @hangchen4 ай бұрын

    Question - if the model curve is below the well calibrated curve, why should we say it is not well calibrated? The well calibrated blue curve seems to me having a worse prediction performance as it will even give 10% at 0.1, 20% at 0.2, 50% at 0.5, while the orange curve gives 0 at 0.1, 0 at 0.2, and 0.25 and 0.5.

  • @TwinEdProductions

    @TwinEdProductions

    4 ай бұрын

    Hi. So a calibrated system and an uncalibrated system have the same performance (typically)! For example, imagine a binary classification system which is very overconfident such that it predicts everything as either 0.01 or 0.99. By calibrating such a model, we are not changing the model performance, but just smoothing out the distribution so that the output values are continuously distributed throughout the whole range such that the output probability value is actually meaningful.

  • @hangchen

    @hangchen

    4 ай бұрын

    @@TwinEdProductions Thank you for your response! Just in time! I was asked about model calibration during an interview yesterday! I will study more about it! Thanks again and have a great day!

  • @TwinEdProductions

    @TwinEdProductions

    4 ай бұрын

    @@hangchen Good luck :)

  • @hangchen

    @hangchen

    4 ай бұрын

    Thank you so mcuh! 👍@@TwinEdProductions

  • @souptikmukhopadhyay6531

    @souptikmukhopadhyay6531

    2 ай бұрын

    You didn't answer his question, the orange line is better and more accurate ,... calibrated. Model is changing accuracy by force fitting to blue line

  • @panosp5711
    @panosp57112 жыл бұрын

    very nice, however i want to make suggestion, in x axis the probability values of each bin are averaged

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Thanks for your comment - yes we completely agree with you :)

  • @panosp5711

    @panosp5711

    2 жыл бұрын

    @@TwinEdProductions you should point out this in your video

  • @AkshayKumar-xo2sk
    @AkshayKumar-xo2sk2 жыл бұрын

    Awesome. But how to do calibration if we dont gave data .my dataset has only 977 records.so i did train, test split. All data is used up. But calibration model has to be fit using new data. Where and how can i do calibration in this case?

  • @TwinEdProductions

    @TwinEdProductions

    2 жыл бұрын

    Hi Akshay, thanks for your comment. If I were you, I would split the data into train, validation, test. 100 records or so should be more than enough for a validation set and hence you can calibrate using this validation set. If you are really worried about not having much data, then you could consider an n-fold cross-validation approach instead. Hope that helps!

  • @softerseltzer
    @softerseltzer3 жыл бұрын

    Whoops, left the previous title card by accident :D

  • @TwinEdProductions

    @TwinEdProductions

    3 жыл бұрын

    Thanks for the pop filter tip - hopefully you can tell the difference!

  • @softerseltzer

    @softerseltzer

    3 жыл бұрын

    @@TwinEdProductions Yes, the audio is definitely crisper and better!

Келесі