Excellent, Sir! Thanks a lot for bringing it so straightforward and consistently. For the first time, I could understand and reproduce the whole thing in r, regarding ROC Curve and AUC.

@bkrai

2 жыл бұрын

You are most welcome!

Unbelievably helpful video - I've been searching all over internet for this. Thank you.

@bkrai

6 жыл бұрын

That's good to know!

thank you Sir, very kind of you to send the R code and the data, appreciate it. Your youtube here explains those concepts of ROC and AUC clearly, with a simple example. Well done.

@bkrai

7 жыл бұрын

Thanks!

Short, simple and covers everything! Thank you!

@bkrai

6 жыл бұрын

Thanks for comments!

Sir... your explanation of ROC and AUC was very simple and easy to understand .Its cleared my all doubts..Thanks a lot...

@bkrai

4 жыл бұрын

You are most welcome!

This was an amazingly clear approach. Thank you.

@bkrai

7 жыл бұрын

Great to hear your feedback!

it is very clear you know what you are doing.Thank you for your contribution !

@bkrai

6 жыл бұрын

Thanks for your feedback!

Thank you so much for explaining in much much simpler way!!!!!

@bkrai

6 жыл бұрын

Thanks for your comments!

Very clear and impressive lecture! Thanks so much!

@bkrai

3 жыл бұрын

You're very welcome!

Thank you so much for your explanation, I could run my code and understand better the process.

@bkrai

6 жыл бұрын

Thanks for the feedback!

superb sir...phenomenal.....u make tough things look simple....proud of you boss

@bkrai

6 жыл бұрын

Thanks!

Thanks a lot sir... for such precise explanation of AU ROC curve. Truly appreciated.!

@bkrai

5 жыл бұрын

Thanks for comments!

Great video and very well explained!

@bkrai

6 жыл бұрын

Thanks for comments!

Yo that beat in the beginning was fire

@bkrai

4 жыл бұрын

Thanks :)

A ROC Curve Tutorial for more than two classes with the 1 vs ALL approach would be a very helpful video :).

@bkrai

5 жыл бұрын

Thanks foe the suggestion, its on my list now.

You nailed it teacher..!!

@bkrai

3 жыл бұрын

Thanks!

Great explanation!

@bkrai

6 жыл бұрын

Thanks!

Great explanation. Thank you Sir! )

@bkrai

3 жыл бұрын

Welcome!

very very helpfull !!! im sending to some brazilians friends

@bkrai

5 жыл бұрын

Thanks and hope you find other videos helpful too!

Thanks for the video! Amazing!

@bkrai

5 жыл бұрын

Thanks for comments!

absolutely excellent explanation. thank you very much.

@bkrai

5 жыл бұрын

Thanks for comments!

I appreciate your help, excellent video 👏🙏

@bkrai

Ай бұрын

You are welcome!

Nicely explained. Thank you.

@bkrai

4 жыл бұрын

Thanks for comments!

Really helpful!

@bkrai

2 жыл бұрын

Glad it was helpful!

Great Video....

Amazing, very usefull, Thanks

@bkrai

3 жыл бұрын

You are very welcome!

Thank you very much, sir.

@bkrai

2 жыл бұрын

You are welcome!

Thanks alot man. you helped

@bkrai

2 жыл бұрын

You are welcome!

i think m the only one who doesn't able to learn any computer language except R and this all happen just bcs of u sir 🙂

@bkrai

4 жыл бұрын

Thanks for comments!

Very good video sir it is very helpful

@bkrai

4 жыл бұрын

Thanks for comments!

Thank you sir...very clear and crisp explanation. In one video I got all the information. From the explanation in the video, I got how to find cutoff for maximum accuracy, by doing this only one class has got more weight in my dataset. but how to find a threshold value of cutoff(which gives maximum of sensitivity and maximum of specificity).

@bkrai

5 жыл бұрын

You can get that using the ROC curve. The color used on the curve changes from 0 to 1. You can identify a point on the curve that is closest to the ideal curve.

@raghavendras5331

5 жыл бұрын

Thank you sir

Thank you for this great video! And thank you for prompt reply. I have questions. If we are doing machine learning, we need to create ROC using predictive model created by test set, correct? (in your "Logistic Regression with R" video, you created predictive model using test set. We need to validate the accuracy of the model). Also, if I want to use which.max func to plot the highest values on the eval plot, what code should I use?

@bkrai

3 жыл бұрын

Seeing this today. But roc curves can be done for both train and test data.

Dear Sir, Your tutorial helps us all to learn about data science. I learn many thing from your tutorial. Now I want to learn how we can make ROC and AUC for multi-class? May you make another video to teach us about multi-class model performance? Thank you

@bkrai

6 жыл бұрын

Thanks for your comments, I'll add your suggestion to my list.

@santanumallik6992

3 жыл бұрын

@@bkrai Sir I have the same query my data set has three class in that case how will I get ROC & AUC curve

thx , very good video.

@bkrai

5 жыл бұрын

Thanks for comments!

Thanks !!!!

@bkrai

3 жыл бұрын

Welcome!

Thank you so much Sir..the video was really helpful in providing practical knowledge of dealing with predictive modelling problems in R..Can you please tell me how to apply weight of evidence/ fine classing in R - is there any ready made syntax?

@bkrai

5 жыл бұрын

Thanks for your feedback! I've also added your suggestion to my list.

Sir How does ROC curve work when the dependent variable is not binary in nature, in essence more than 2 factors for which we have to model the data(Note: but not continious in nature).

we an use Deducer package in R to directly run the ROC Curve library(Deducer) mymodel

@bkrai

6 жыл бұрын

thanks!

Hi Dr, when i do stacking of ensemble why do i get the roc curve in triangular shape?

Hi Sir, i have calculated cut off for accuracy for my data (~0.475). i would like to know where exactly i should replace default 0.5 with this .475 ?

Sir, I have ran the model using neuralnet package , is it necessary to calculate probability for predicted value or we can directly go with value obtained for test set. One more question sir is it any way to plot roc curve for two models.

Hello Sir , i'm an avid viewer of your videos which truly add value to our ML understanding. Have a quick question that once we determine the Best Value of Cut Offs post Model performance evaluation , should we go back and re-run the Model performance with Best Cut Off values and change the cut off of 0.5 that we considered as thumb rule.

@bkrai

4 жыл бұрын

Yes, that's correct.

Nicely explain. Sir, can you arrange to prepare a video on SVM of binary outcomes

@bkrai

2 жыл бұрын

Try this. kzread.info/dash/bejne/oodpybp-fseZkZc.html

Hi Dr. Bharatendra Rai, would you be able to make a tutorial on building a logistic regression model using training and validation sets, with performance checking via ROC curve as you have done here? I know you posted one on linear regression, but I thought a logistic model would be very helpful too. Thank you!

@bkrai

4 жыл бұрын

You can get logistic regression from this link: kzread.info/dash/bejne/c4qsmbmcYaa3Z7w.html ROC steps are already in the current lecture video.

Hi, Firstly, great video this really helped me to understand the ROC curve and implement it with my data in R. I am analysing diagnostic data for a masters degree research project. I wanted to know how to identify the cutoff value from the value that we take from the accuracy versus cutoff curve or the final ROC curve. The scale goes from 0-1 but my independent variable data ranges from 100 to 10^7 . In short, how do I take the best cutoff value that this analysis outputs and relate/convert this to my independent variable and an exact cutoff value? Thanks very much.

@bkrai

5 жыл бұрын

Cutoff value is for the dependent variable. If the dependent has two categories 'yes' and 'no', then default cutoff is probability = 0.5. ROC provides information on how prediction model performance will change if cutoff value changes from 0 to 1.

How to handle if my all data is categorical my predictor features are subject columns with 1 to 8 grades for each subject and response variable is subject where we have to predict response variable grades (1-8 ). Before applying model I converted all features and response variable into factors is this right step or should i only covert response variable into factors and keep predictors in numerical format

@bkrai

4 жыл бұрын

I'm seeing this today, but for categorical variables you can try random forest: kzread.info/dash/bejne/ln6XzrCCfpC6gNI.html

Hi Sir, Thanks for wonderful video. Could also make AUC video where dependent variable is continuous.

@bkrai

6 жыл бұрын

Thanks for the suggestion! I've added this to my list.

Im new to ur lecture and find it very interesting and useful.I have one question ?how do you get the cutoff .5 from the classification table @5:39mn of the video.Thanks

@kpakpomoevi1603

4 жыл бұрын

I figure out and see it should be a default value.However I'm still having issue with the performance object eval.when I tried to print eval,it is giving me this : A performance instance 'Cutoff' vs. 'Accuracy' (alpha: 'none') with 392 data points please help me have what you had on your screen.Thanks

@bkrai

4 жыл бұрын

You will see it better after plot.

Further to my earlier comment, also wanted to ask what software you used to create these videos on data analytics. Thanks

@bkrai

4 жыл бұрын

I used iMovie.

i am getting following error when i use : pred

@bkrai

4 жыл бұрын

Difficult to say anything without looking at code.

@juanmauricioarrietalopez2395

4 жыл бұрын

Same error. Could you solve it?

can you tell me the differences below: yourmodel

@davidserero9125

7 жыл бұрын

I think yourmodel

Hello. This looks like the cutoff would be the probability of a certain student getting admitted based on the multinomial model. If I am working with a dataset with one independent variable contributing to my multinomial model, and am wanting to obtain a cutoff value from that independent variable (ie what is the cutoff value of SAT score that will best tell me if someone is admitted to college), what would I be changing in the code? Thank you.

@lewsmash

5 жыл бұрын

I was also looking for this answer to this if you managed to find out?

Very nice and Excellent explanation. Could you please make another video to draw multiclass (more than 2 class) Roc curve. (one vs rest roc )?

@bkrai

Жыл бұрын

You can refer to these: kzread.info/dash/bejne/mKiesNeJm7O3cpc.html kzread.info/dash/bejne/aIeB1Mx-p6nPgbw.html

Hello Dr. Rai. Would you happen to know what code I can use to compare 2 ROCs and/or AUCs using R? Also, if there is a way to represent 2 ROCs in one graph. Thank you! ~ Tanya

@bkrai

3 жыл бұрын

You can calculate AUC for different models and compare them. It should be higher the better.

@sanjibphukan8921

2 жыл бұрын

@@bkrai Sir, in that situation how can we get the p-value of the compared statistic.

I have a question about the logistic regression model part. Does the code deal with the whole data? I thought when doing the logistic regression model, you have to divide the data into training set and test set. In the code you've used, does it divide training set and test set automatically?

@bkrai

6 жыл бұрын

I used full data as focus was more on ROC. But when developing a prediction model it is always good to partition data into training and testing data sets.

@wanglaoshuwang3831

6 жыл бұрын

Thank you so much! helped a lot! :)

Thank you so much sir, Just want to ask you whether type='response' is same as type='prob' when I am trying to give type='prob' , R is throwing an error like "Error in match.arg(type) : 'arg' should be one of “link”, “response”, “terms” ?

@bkrai

5 жыл бұрын

'Response' usually could be classes such as 'yes' or 'no'. But 'prob' gives probability values. And that could lead to errors that you are getting.

@abhilashiv3599

5 жыл бұрын

@@bkrai Thank you Sir

Please do a video on sentiment analysis using R in detail... Deep dive analysis

@bkrai

7 жыл бұрын

Thanks for the suggestion, I'll probably do it sometime this month.

Hi, I'm on R studio v. 1.1.423 now and nnet package isn't available and I can't seem to find an equivalent... any ideas what I can use to get the same results? Thanks.

@kamalpada1270

5 жыл бұрын

Please upgrade RStudio to atleast 1.1.463.. nnet works in this version.. Good luck.

@bkrai

4 жыл бұрын

This is old comment. I guess you must have already updated.

wooow just wooow!!!

@bkrai

3 жыл бұрын

Thanks!

Hello sir, can you make a video for ploting ROC curve for SVM. I am getting an error in my code. The error is i am getting is format of prediction is invalid. Thank you

sir, how to identify the optimal cut off value so that TPR (sensitivity) will be high ?

Sir, based on this miss classification problem for admit =1, how can u change prob to other value in the model? maybe with under 0.45 =0 and over = 1

@bkrai

3 жыл бұрын

ROC automatically tries probability values from 0 to 1 and then plots it on the curve.

I need some help please trying to submit my project but cannot get ROC work

Thanks for the useful information. I would like to ask you if I can use ROC to measure the effectiveness of the prediction model? And can I use ROC in R software?

@bkrai

4 жыл бұрын

For model effectiveness you can use AUC. Also yes you can do ROC in R, this video gives you all the steps.

@fatimabadi3335

4 жыл бұрын

Dr. Bharatendra Rai thanks

Explained very neatly sir, appreciate if you can pls add dataset and code for learning please....

@bkrai

7 жыл бұрын

email id?

@avinashsingh357

7 жыл бұрын

Thanks for your quick response sir, my emailid is avinashsinghemailid@gmail.com

@bkrai

7 жыл бұрын

all set.

Hi Professor, i love your videos ,it's very interesting.I'm a PhD student , sometimes i find difficult to have the link between my own variables ( concengrations of elements) and the variables that you work with, that's why ; I wish you have documents well explained concerning the data processing analysis to sends it to me I will be very grateful . Also i want that you sent me the data file.

@bkrai

6 жыл бұрын

email id?

I have applied the same functions in evaluation of my GAM model where I am not able to produce the confusion matrix. The results shows 2*132 table matrix instead of 2*2 matrix moreover I have 203 'Y" variable in validation data. Why its coming so. Plz help me. Thanking you.

very helpful video sir. thank you so much. I have a doubt how do you fix the threshold value as 0.5.

@bkrai

5 жыл бұрын

Default threshold is already 0.5, there is no need to do anything for this.

@nithinmamidala

5 жыл бұрын

Ok.. Thankyou sir.

Hi Sir, When I try using "prediction" function on a multinomial target variable and a matrix of predicted probabilities, I am getting the error below: Error in prediction(preTrainProb015, train015$Delq.Status) : Number of cross-validation runs must be equal for predictions and labels. In the above error: preTrainProb015

@bkrai

5 жыл бұрын

" Number of cross-validation runs must be equal for predictions and labels." Looks like you have mis-match between predictions and labels. Probably you have to review earlier lines of code.

hi sir, I have some confusion, please help me to resolve it. Your IV (admit) has two levels...0 and 1 and you performed multinomial logit? Is it obvious to plot a multiclass ROC rather than a typical ROC curve, when my IV has three levels ( i.e. 1,2, and 3). Thanks.

@bkrai

3 жыл бұрын

Multinomial logit works for 2 or more levels. However, ROC used here is only for situations where IV has 2 levels.

@lianjek5788

3 жыл бұрын

@@bkrai Would you please provide a lecture about multiclass ROC? Thanks.

@bkrai

3 жыл бұрын

Thanks, I've added it to my list.

I need to know how the medical data set are going to use in R studio programming and (example MIMIC, DCOM, etc) which library i have to use... pls if you know anyone inform...

@bkrai

4 жыл бұрын

You can try these: kzread.info/head/PL34t5iLfZddsQ0NzMFszGduj3jE8UFm4O

Unfortunately unable to proceed. An error message appears: pred

Hello Sir, are precision and recall same as sensitivity and 1-specificity respectively?

@puneetksidhu

5 жыл бұрын

if we would have to plot a PR curve for the same data that you used to plot ROC curve, how would we do that? Can you please send me the code at puneetkaursidhu@gmail.com

Hi Sir , why did you use multinom function here ? isnt multinom used only if target var have more than 2 categories ? while in this video we have only 2 categories , yes or no ?

@bkrai

5 жыл бұрын

Multi works for 2 or more, so using it here should be ok.

for those who are looking for the data: stats.idre.ucla.edu/r/dae/logit-regression/

@bkrai

5 жыл бұрын

Thanks for sharing!

In place of "tpr", will a numeric entry work?

@bkrai

4 жыл бұрын

I've not tried, but should work.

Hi Bharatendra, why don't you use glm()? I looked it up and it seems like multinom is used when the dependent has more than 2 levels. In your example, the dependent is admin(no, yes). That's why I'm confused why you chose multinom () instead of glm(). Thank you.

@bkrai

5 жыл бұрын

multinom works for 2 or more. So when dependent variable has 2 levels, it should work fine.

Would you be able to send me the dataset used for this please? awesome job done.

@ranjitshekdar1720

7 жыл бұрын

rshekdar@gmail.com my email id. sorry accidentally clicked send before could complete.

@bkrai

7 жыл бұрын

+Ranjit Shekdar all set.

@ranjitshekdar1720

7 жыл бұрын

you sir are very fast, much appreciated. Thanks again. I was not expecting it to be sent out so fast.

Sir, can you please also attach dataset files/link along with your videos. This would greatly help us in learning by practicing with same data set. Thanks for great videos sir.

@bkrai

5 жыл бұрын

I've added a link in the description area below the video.

@abhitest1

5 жыл бұрын

@@bkrai thanks.

Thank you for a great tutorial sir. Could you please share the dataset and the code ?

@bkrai

6 жыл бұрын

email id?

@hamsinisankaran2435

6 жыл бұрын

hamsini0992@gmail.com. Thank you

@bkrai

6 жыл бұрын

all set.

How to generate ROC AUC curve for multi class responsible variable? Thank You

@bkrai

3 жыл бұрын

You can use this method with two class at a time.

How do I do this AUC I have NA Values in my dataframe?

@bkrai

3 жыл бұрын

You need to first address missing values. See this link: kzread.info/dash/bejne/c6Jr0LKFepPMo8o.html

Sir, a)for multi-class, how you will will come with false positive, false negative b)how to compute ROC for multiclass

@bkrai

6 жыл бұрын

I'm adding this to my list. Thanks!

@SaranathenArunE

5 жыл бұрын

thanks so much sir

sir where do i get this dataset??

@bkrai

6 жыл бұрын

email id?

Hello Sir, Why are we using [[1]] in which.max function?

@bkrai

6 жыл бұрын

That's where the value we are looking for is stored.

Hello, can this be applied on CNN?

@bkrai

3 жыл бұрын

If there are two classes, then yes.

Hi Sir, One question here. Can ROCR curve be drawn in case of multiple classification. Ex:We have to classify the given data in 3 different classes? Thanks, Roopa

@bkrai

7 жыл бұрын

It is meant for only two categories. If you have three classes such as 1, 2, and 3. And if your interest lies more in correctly classifying say class "3", then you may still have two classes with 3 and others.

@roopashastri9908

7 жыл бұрын

ok..got it.Thank you sir!

why u wrote 'y.values' in slot function of AUC?

@bkrai

5 жыл бұрын

When you run 'eval' that contains that contains accuracy values for various cut offs, you can see different type of information are stored in various slots. And y.values contain data on accuracy.

Very nice explanation Could u plzz send me dataset and code

@bkrai

3 жыл бұрын

For data, see link below video. For code, see the pinned comment.

if am using 2 variables only GPA and Admit then what will be the Logistic Regression Model formula

@bkrai

8 ай бұрын

You can refer to this: kzread.info/dash/bejne/c4qsmbmcYaa3Z7w.html

Hello sir, Video was excellent but I have small question. Cut off value changed to 0.45 so do we need to again run model. If not then why and if yes then what changed need to be made in code so that pure classification can be made for improving accuracy level of model.

@bkrai

6 жыл бұрын

You don't need to run model again. The model gives predictions in terms of probabilities. When we use cutoff of 0.45, probabilities that are less than 0.45 are classified into first group and those above 0.45 are classified into 2nd group. So for any cutoff value, probabilities from prediction model are not going to change, they are only used for classification.

@tapangautam8746

6 жыл бұрын

Thank you sir for your reply and for guiding me I am want to know one more thing, as according to your reply I can assume probability less then 0.45 as yes and above 0.45 as no in terms of dependent variable. But where or on which sort of output I will come to know that due to variation in independent variable majority of time dependent variable occur (yes or no). Thanks

@bkrai

6 жыл бұрын

Independent variable will take only one value for each case. If a new student has GRE = 380, GPA = 3.61 and comes from a school ranked 3, then inputting these values in the prediction model will give probability = 0.18. Since p=0.18 is less than cutoff = 0.45, this student will be rejected and will not get admission.

@tapangautam8746

6 жыл бұрын

Good morning, now I understand answer. Thanks for helping me in compression of logistic regression

@tapangautam8746

6 жыл бұрын

Hello Sir I need little help from your side. In code where we create subset out of sample why == TRUE or FALSE is used. What is difference between = and == symbol in Logistic regression. Thank you sir

How could we cross validate these results?

@bkrai

2 жыл бұрын

You can refer to this for more detailed coverage: kzread.info/dash/bejne/mKiesNeJm7O3cpc.html

what does represent the color from green to red of the ROC?

@bkrai

4 жыл бұрын

It represents cutoff values between 0 and 1.

@fabioambrosini5265

4 жыл бұрын

Dr. Bharatendra Rai thanks

Hello Sir, Thanks for your every lecture for students like me. Respected Sir, i am working on Groundwater potential mapping, i have compete my analysis using ArcGIS, now i have to validate my results with ROC curve, could you please guide me how could it possible using R. I will be very thankful to you.

@bkrai

3 жыл бұрын

You can follow the steps in this video. If you get struck anywhere, let me know.

@umairrasool9580

3 жыл бұрын

@@bkrai Sir i have get problem here, please check, how to make model with my data? setwd("C:/Users/Umair/OneDrive/Manuscript Quetta GWP/ANN") .libPaths("C:/Users/Umair/OneDrive/Manuscript Quetta GWP/library ANN") install.packages("tidyverse") library(tidyverse) Wells

@bkrai

3 жыл бұрын

Which part has a problem?

@umairrasool9580

3 жыл бұрын

@@bkrai Sir problem to make model for ROC and need to compute the true + and true -.

Hello sir.. is possible to calculate auc metric for multiple class prediction

@bkrai

3 жыл бұрын

I'll look into this.

@subaganesh552

3 жыл бұрын

@@bkrai thank you sir...

@bkrai

3 жыл бұрын

welcome!

I can not find ant 'y.value' in eval

Great video, trying to plot ROC for random fors but it giving me the following error Error in prediction(p2, train$Dispute) : Number of cross-validation runs must be equal for predictions and labels

@bkrai

6 жыл бұрын

what are your previous lines before this line where you are getting error?

@francodjo

6 жыл бұрын

no error. thank you very much for quick response can you forward me the co

@francodjo

6 жыл бұрын

library(randomForest) TD_model

@bkrai

6 жыл бұрын

i've sent my code file.

@bkrai

6 жыл бұрын

note that the response variable here has two levels. If your data has more, then the codes shown in the video may not work as it is.

Hello, This is great video but I am slightly confused with the probability explanation. YOu mentioned that if the prob prediction score is less than .5 then chance are less than average but doesn't it depend on %age of events in the data on which model is based? If in the sample data, the event rate is 1 out of 4 then probability is .25, so any scores above .25 in the final output mean model is saying that this has higher chances. Not necessarily it has to be above .5.

@bkrai

5 жыл бұрын

In this data there are only two outcomes. Either a student is admitted or not-admitted. If you only calculate % as students admitted and use that as a probability, it will not be a very useful prediction. Because it will remain same for every students irrespective of their gpa, sat score or college they are coming from. The prediction in the form of probability between 0 and 1 here is based on all input variables.

@mashalnabh2747

5 жыл бұрын

@@bkrai Hello, Let me give an example: Lets say in your data set , there were 100 observations and 10 "events" of passing . So the probability of an event in your data set is 10/100= 0.1 . Now I build the prediction model based on different predictors like Gap, SAT, gender etc, and I get the final predicted probability scores. Lets say for some student, we get a score of .3,. All I want to say is that this student has higher chances than average. Not necessary the probability score has to be above .5. This depends upon the original percentage events in your data on which the model is built - which in this case is 0.1

@bkrai

5 жыл бұрын

For original %, let's consider your own data. Let's say 10 out 100 applicants get accepted giving a rate of 0.1. Now forget about any type of model and simply classify all 100 students as getting accepted. Here without any model, you will be correct 10% of the time, but incorrect 90% of the time. Do not mix-up this overall rate with individual probability. When you develop a prediction model, it should give overall accuracy better than 90% to be of any value.

@sachin01663

5 жыл бұрын

@@bkrai Thanks. Am I right to say that predicted probability score less than .5 (after building the final model) does not necessarily mean that, that event is less likely to happen.

@bkrai

5 жыл бұрын

That's correct.

cutoff means threshold?

@bkrai

6 жыл бұрын

That's correct.