You can get a copy of the code from the StatQuest GitHub, here: github.com/StatQuest/roc_and_auc_demo/blob/master/roc_and_auc_demo.R Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

@falaksingla6242

2 жыл бұрын

Hi Josh, Love your content. Has helped me to learn a lot & grow. You are doing an awesome work. Please continue to do so. Wanted to support you but unfortunately your Paypal link seems to be dysfunctional. Please update it.

@ryanmckenna2047

Жыл бұрын

The code would not run when I downloaded it from github

@statquest

Жыл бұрын

@@ryanmckenna2047 What part didn't run? I just re-ran it and worked fine.

@ashishdayal172

Жыл бұрын

did u make this in python too??

@statquest

Жыл бұрын

@@ashishdayal172 not yet

"The only man who never makes mistakes is the man who never does anything." Thank you ;)

@statquest

5 жыл бұрын

No, thank you! You're comment was very helpful and spared me a lot of future embarrassment. The video was only seen by 100 or so people (not 1,000s) before you pointed out the error.

Dude, your videos are great. I never found something so clearly on the internet. Congratulations!!!

Impressive video. Theory and examples with software are the best way to learn. There is much going on this video, one of the best of ever. Thank You, Josh, greetings form Italy for a happy new year for you, your beloved ones and for all the people which follow your amazing lessons.

@statquest

2 жыл бұрын

Wow, thanks!

Such an awesome channel I came across! ....gonna share it with everyone under my umbrella !!! You are doing really great bro!

@statquest

4 жыл бұрын

Thank you! :)

Complex things in simple and understandable language. I have never met a better teacher!

@statquest

11 ай бұрын

Thank you very much! :)

Your explanation of the process and logic behind each function and line are so helpful. I hope you'll make more of these videos. Thank you so much, this content is uniquely valuable.

@statquest

9 ай бұрын

Thanks!

Crazy how good you are at explaining. You explain the little things I always start to struggle with other teachers/tutors! Thank you so much for these Videos

@statquest

2 ай бұрын

Happy to help!

Hey Josh Ty again, while my studies I reproduced everything using R Colab (Really recommend for who is studying Josh's codes in R)

Best song ever, Josh. StatQuest keeps gettin’ better and better! Many thanks.

@statquest

4 жыл бұрын

Thank you so much! :)

Great explanation of everything including each parameter in the graphs. Loved it!

@statquest

Жыл бұрын

Thank you!

Thank you for helping me with my credit risk class :)

Good job and well-done. I like your style of teaching, it's great!!!

@statquest

3 жыл бұрын

Thank you! 😃

Thank you for this informative video. It helped me a lot. Great work!

@statquest

3 жыл бұрын

Glad it helped!

Wonderful tutorial!!.....thank you so much Josh :)

@statquest

4 жыл бұрын

Thanks! :)

Hey..... Love the way you present ❤️

@statquest

3 жыл бұрын

Thank you so much 😀

These are the best videos. When I need to relax, I watch your videos

@statquest

3 жыл бұрын

Glad you like them!

@happygolucky4350

3 жыл бұрын

@@statquest If you have two output neurons in a ANN (for a two class classification problem {1,0; 0,1}, it is okay to build the ROC just by comparing output of any one of those neurons with its corresponding target?

@happygolucky4350

3 жыл бұрын

Thanks Josh, I changed it to {1,0} as output as the AUC for the two neurons {1or0} in the {1,0;0,1} architecture were not the same.

Thank you sooooo much Josh! You are a life saver!!😄

@statquest

2 жыл бұрын

Happy to help!

Thanks for your wonderful and detailed videos!

@statquest

Жыл бұрын

Thank you so much for supporting StatQuest! BAM! :)

You are amazing man, thanks for the video and keep making more videos like these. BAM!!

@statquest

5 жыл бұрын

Double BAM!!!! Thanks for the encouragement! :)

@bitclear670

5 жыл бұрын

Double Bam!!

Incredibly helpful, thank you!

@statquest

3 жыл бұрын

Thanks!

Dear ,i haveenjoyed ur video ,very much clearity of thoughts

@statquest

4 жыл бұрын

Thank you so much 🙂

You solve my headache. Thanks a lot

@statquest

27 күн бұрын

Happy to help!

Great video! One quick question. Do you know how to plot ROC-AUC graph for SVM and adaboost?

Thanks man, very clear and helpful

@statquest

4 жыл бұрын

Thanks! :)

hey bro, i love your videos so much, please hang in and i will continue to support you!

@statquest

6 ай бұрын

Thank you very much!

Thank you sooooo much for your lessons. Super helpful

@statquest

3 жыл бұрын

Thanks!

This video is absolutely amazing! but how can i determine the threshold/cut off weight from threshold probability that decides whether the subject is obese or not using code and not by direct extrapolation from the logit curve?

thank you for such an informative tutorial

@statquest

2 жыл бұрын

Glad it was helpful!

Great again! I would be interested to see how to make combined ROCs for, say 2-4 different biomarker candidates. This would be to see if their combined use would result in higher AUCs than that of individual markers.

@statquest

3 жыл бұрын

Noted

Many Thanks Josh!

@statquest

5 жыл бұрын

You're welcome! :)

The silly songs, the calm voice and the bams gives this vibes as if the course is narrated by Forrest Gump. Love it.

@statquest

Жыл бұрын

Thanks! :)

Is there anyway to suppress plotting the top and right axes? I tried bty='n' and axes=FALSE to add them later using axis(1) and axis(2) but neither of those worked.

VERY helpful - thank you!

@statquest

3 жыл бұрын

Glad it was helpful!

Thank you very much!

I got an error, Error in roc.data.frame(trainData, fitModelTrai$votes[, 1], plot = TRUE, : 'response' argument should be the name of the column, optionally quoted. the only difference between your code and mine is that I have many parameters/columns/features (approx 35) not only one (weight)

Thanks a lot sir! You are very helpful!

@statquest

2 жыл бұрын

Most welcome!

You are amazing, man! Thanks!!!

@statquest

3 жыл бұрын

Thanks

Thank you so much for this video!

@statquest

Жыл бұрын

Glad it was helpful!

Keep up the good work .. Thank u🤩

@statquest

Жыл бұрын

Thanks!

YOU ARE MARVELOUS,EXTRAORDINARY .I WISH YOU COULD HAVE EXPLAINED IN PYTHON

@statquest

2 ай бұрын

One day I will.

THanks for the videos Josh! I have a question about AUC. Even though in this video AUC for random forest is lower than logistic, isn't forest a better alternative here as there exists a threshold that generates higher true positive rate for the same false positive rate compared to logistic. This makes the significance of AUC subjective in comparison

@statquest

3 жыл бұрын

What you have to do is pick a range of thresholds that are acceptable. Once you do that, you can compare the AUC between those thresholds to determine which method is best.

I mean, the stats tutorial is indeed very well done, but the intro song was already enough to make me immediatly click on the like button.

@statquest

Жыл бұрын

bam! :)

waaa.. i'm so thankful found this video. Thanks a lot. Stay healthy cool people :)

@statquest

2 жыл бұрын

Thanks!

Thanks for the video and explanations! What statistical test would you use to compare 2 ROC curves?

@statquest

2 жыл бұрын

There are a bunch of options. This tool (in R) implements them: bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-77

I thank God I found this channel 2 years ago... 😇

@statquest

2 жыл бұрын

bam!

@zainabkhan2475

2 жыл бұрын

@@statquest 😄😄😄

Thanks a lot !

Hey Josh, great videos on ROC curves, your teaching is refreshingly concise and clear. I just have one question that I hope you could expand on. When we first generate 100 samples from a normal distribution, why do we need to sort them from low to high? And what would the dangers be if we didn't do this? Thanks for the great content!

@statquest

Жыл бұрын

What time point in the video, minutes and seconds, are you asking about?

@user-io2em1ld4n

Жыл бұрын

@@statquest roughly around 2:55

@statquest

Жыл бұрын

@@user-io2em1ld4n Technically, you don't need to sort them, but it makes it easier to look at the data. When we print out the values for the "obese" variable at 4:11, the output is way easier to interpret because the values for weight were sorted.

Thank you so much! Do you consider make a video about limited dependant variables models (tobit, heckman...)? It will be very helpful for us! All the best.

@statquest

5 жыл бұрын

OK. I'll put it on the to-do list, but it will be a while before I get to it.

@Davidravaux

5 жыл бұрын

Thank you! This is a short bibliography about the topic: J. Scott Long, Regression Models for Categorical and Limited Dependent Variables Alfred DeMaris, Regression With Social Data: Modeling Continuous and Limited Response Variables Wooldrige, Introductory Econometrics I can share you the books if needed.

@statquest

5 жыл бұрын

@@Davidravaux OK. However, just know that my to-do list is huge (it has about 200 things on it - I get about 3 or 4 requests every day), so it might take me a long time to get to it. However, if a lot of people start asking for a certain topic, that topic gets moved closer to the top of the to-do list. So, if you know of a ton of people interested in this subject, you should have them add to this comment.

@Davidravaux

5 жыл бұрын

Ok, I totally understand, thank you for clarifying.

So good Thanks for the video

@statquest

2 жыл бұрын

Glad you enjoyed it!

Thank your great lectures! The thresholds that you derive here are between 0 and 1. Can we translate these thresholds to the actual cut-off values?

@statquest

3 жыл бұрын

In these examples, the thresholds are the actual cut-off values. In other words, if the logistic regression predicts that the probability that a mouse is obese is 0.9, then we would compare that to the threshold that we obtained from the ROC graph to make a final classification.

Love your content! Quick q: from a conceptual standpoint, are you just testing the hypothesis that the underlying distribution of the weights (which you defined as a gaussian) is not a uniform distribution

@statquest

3 жыл бұрын

ROC graphs give us a sense of how accurate or models are given different thresholds for making decisions. For more details, see: kzread.info/dash/bejne/Zp6GpLR9kq3LnbA.html

great video!!

@statquest

4 жыл бұрын

Thank you! :)

You are the savior of the little humans we are, thanke you god! I have a silly question, sometimes you use

@statquest

4 жыл бұрын

R is funny about the "

Thank you sir

Hi Josh, your videos are great! I have one question about choosing best method based on ROC overlapping graph. If we compare Logistic Regression and Random Forest we see that Logistic Regression is better because of bigger AUC. Bur does it make more sense here to choose Random Forest because one specific instance of Random Forest (with one specific threshold) gave us best confusion matrics? I assumed here that accurately classifyng positive and negative class are equally important.

@statquest

3 жыл бұрын

It really depends on your goals. In general, Logistic Regression performs better. However, depending on what threshold works best for you, you may still choose Random Forests if it performs better at that threshold.

I Have a problem with this. The ifelse function is giving me a different value each time i run it. ¿Do you have a clue why is this happening?

could you do the same in python too?

Hey Josh, is there a way to make inferences on more than two ROC and to perform multiple comparisons? (a generalization of DeLong's test? and maybe a method to adjust alpha for multiple comparisons too?)

@statquest

3 жыл бұрын

Good question! Off the top of my head I don't know if there is or not.

That ROC you had really tied the room together

@statquest

4 жыл бұрын

:)

Please make more Videos with R! :)

@statquest

4 жыл бұрын

:)

Hi and thanks for your great videos! Could you please elaborate about the obese variable and specifically about the "test" part in that code line. What if I already know who is obese and who is not (let's say based on some external medical profile, let's say "real") and I want to estimate the prediction of the model which is based on a some score (let's say "score") that each individual has. Would I just do glm(real ~ score).? What if I wanted to find the best score - the score that above it I classify someone as "obese" and below it "not obese". what's between the probability threshold in ROC curve and a thresholding of the score itself. Thanks!

@statquest

3 жыл бұрын

In order to draw this ROC graph, we have to know who is obese and who is not to begin with. So the situation in this video is no different from yours. If you want to find the "best" score, you have to then decide what percentage of false positives and false negatives you are willing to live with - the ROC graph will help you decide that. You can then find the corresponding value by looking at the thresholds and the probabilities predicted for from your model with different scores.

Great video thanks

@statquest

Жыл бұрын

Thank you!

Bam! Good tutorial.

@statquest

3 жыл бұрын

Thanks! :)

is it possible to print the cutoffs at seq(0.1, by=0.1) in this curve with roc() function? ...Awesome videos btw!!!!!!!!

@statquest

4 жыл бұрын

I'm not sure this would be easy to do, since the thresholds may not exactly equal 0.1, 0.2, 0.3 etc. For example, in this video, the thresholds start at 0, then the next one is 0.013, then 0.032, ..., 0.088, 0.1004, 0.119, etc. So you see, there is no threshold that is exactly 0.1. So you'd have to calculate the differences from different thresholds and print the one that has the smallest difference.

Thank you for the video. It was very easy to follow. May I know how do i obtain optimal cut off points using the ROC curve?

@statquest

4 жыл бұрын

I answer that question in my video that explains ROC and AUC: kzread.info/dash/bejne/Zp6GpLR9kq3LnbA.html

@mzw90

4 жыл бұрын

@@statquest Thank you for your reply! I was actually wondering how to interpret the threshold numbers seen on 09:51. After head(roc.df), you get a list of TPP, FPP and thresholds. For example in the 2nd row TPP 100 FPP 97.77, what does threshold of 0.01349 mean? I also have a separate question, I am curious if it is always necessary to always create a linear model first for the ROC curve? For example I am comparing the ROC curves of age and co-morbidities against non-cancer mortality, do I have to create a linear regression for age using glm()?

i am a big fan of you! can you make a survival anaylsis video?

@statquest

5 жыл бұрын

Yes! I will make one this spring. Many people have asked for this topic, so it is at the top of my to-do list.

You gotta stop saying BAM!!! it's really funny :D

Loved the video! How do you relate the threshold back to the data? I.e. make a statement like the threshold between obese and not obese is 140lb

@statquest

3 жыл бұрын

First, you find the threshold you are interested in (these are in roc.df), then we look at weight associated with the largest glm.fit$fitted.values < the threshold. For example, if the threshold is 0.5, then the weight is: max(weight[glm.fit$fitted.values < 0.5])

Hey josh great video. I'm having a small doubt. Is there any way to plot ROC graph for multiclass ? I ran a multinomial logistic regression model on my dry bean dataset which has 7 classes. Is there a way to plot ROC graph for this ?

@statquest

2 жыл бұрын

I don't know how to do that.

Great vedio! Very helpful. BTW, there is a discrepancy between this clip and the code shared in your website about the obj roc.df (line 78). Nothing has been assigned to the obj yet so when we run the line 78 gives an error msg. Overall, very clear and handy. Thank you!

@statquest

5 жыл бұрын

Thanks for catching that! The problem had to do with how wordpress interprets the the ">" and "

@farawayscity

5 жыл бұрын

@@statquest I see. Good to know! Thank you~ :>

Thank you!

@statquest

4 жыл бұрын

:)

Is there a way to increase the font size of the text and numbers? Great job btw

@statquest

4 жыл бұрын

Yes. See: stackoverflow.com/questions/4241798/how-to-increase-font-size-in-a-plot-in-r

What does par(pty=‘M’) do? You said it’s maximum but does it change the shape of the plot ?

@statquest

3 жыл бұрын

Yes. It uses up all available space to draw the plot, regardless of the shape of that space (so if that space is rectangular, your plot will be rectangular). In contrast, setting pty='s' forces the plot to be square.

Many thanks Josh, you are doing a great job. In my study, I would like to calculate and plot pROCs for a couple of maxent scenarios and glm model scenarios using 1000 iterations and a 5% omission error using pROC package in R, would be really grateful if you can guide me a bit. Thanks in advance.

@statquest

3 жыл бұрын

Let me know how it goes! :)

@rahulg1504

3 жыл бұрын

@@statquest May I get the R code for the scenario I mentioned? I am still trying to figure out how to prepare data from the maxent output and then use it with pROC package to calculate and plot AUCs. I am relatively a newbie in R. Theory wise I think I am pretty clear, but struggling with codes and commands to get this job done with pROC package.

@statquest

3 жыл бұрын

@@rahulg1504 The code for this video is here: github.com/StatQuest/roc_and_auc_demo/blob/master/roc_and_auc_demo.R

Hi Sir, your videos are very helpful. Hope that you can make a video on mean decrease Gini of Random Forest

@statquest

2 жыл бұрын

I'll keep that in mind.

@dorothymartin2477

2 жыл бұрын

thank you very much !!!! 😁😁

Hello! Any ideas on how to plot the Precision-Recall Curve?

@statquest

3 жыл бұрын

I'll keep that topic in mind.

Hello for the video, really useful, in this example you come up with a method to classify obese and not obese , what about when you don't know a threshold for the initial classification of obese or not obese ? Does the pROC function test different thresholds ?

@statquest

2 ай бұрын

That's the whole idea of an ROC graph to being with - it's used to determine the optimal threshold.

Ah, the pirate's favorite programming language!

@statquest

4 ай бұрын

:)

Tried to get this runing from predtictions from sparklyr...then after 4 hour of tourment i realize that predicted dataframe in not on my computer....tunel vision can be bitch sometimes.

Hey! Wonderful video. I had just one doubt- I used a similar code that you used in my Rstudio. And as the runif function is generating random numbers, I could have very well expected that the values in the obese variable is different from the ones generated in your machine. However, eerily enough, it came out to be exactly the same. What sort of sorcery is this? 😮

@statquest

9 ай бұрын

Did you set the seed of the random number generator? If so, we'll get the same random numbers every time.

thank you soooo much!!!

@statquest

3 жыл бұрын

:)

I have multiple logistic regression models and want to plot the ROCs of all those logistic regression models in one plot. How can I do that?

@statquest

4 жыл бұрын

I talk about how to add multiple curves to the same graph at 13:19

How d you calculate the p-value of the AUC (obtained from logistic regression model)?

@statquest

2 жыл бұрын

See: stats.stackexchange.com/questions/386468/does-auc-roc-curve-return-a-p-value

Is is possible to compute the significance level of AUC using the pROC package? Sth similar to the SPSS output

@statquest

3 жыл бұрын

As far as I know, you can do confidence intervals. For more details, see: www.rdocumentation.org/packages/pROC/versions/1.16.2

Excuse me, but is this the same thing as "Lift Chart"?

@statquest

4 жыл бұрын

They are similar, but not the same.

Thank you for another great video. I have a question, what if we have multiple problems for classifications? Not only two classifications (obese and not obese). For example, we want to classify 10 cell types (let's say cell type 1, cell type 2, ..., cell type 10) whether these cell types are present or not in the tissue sample? How can we use this roc() function to plot the ROC curve?

@statquest

2 жыл бұрын

To be honest, I don't know the answer to that off the top of my head.

@emkahuda776

2 жыл бұрын

@@statquest I have made my own function to plot the ROC curve with similar condition I mentioned. However, I need to make another function to calculate the AUC and was hoping I could use the roc() function which seems providing more information and can include much more information, such as AUC and partial AUC as well. 😰

So even if it has a lower AUC, it seems that random forest is a better choice. Right?

@statquest

4 жыл бұрын

It always depends on how important it is to avoid false positives or false negatives. Once you define those, you can figure out which curve makes more sense for your application.

Hi, Josh! A big fun of yours! Thanks for so many wonderful videos! Wonder if I can ask for help. I am using glmer function in R to fit a mixed effect logistic regression to my longitudinal data. However, I am having trouble extracting fitted value for my model to draw a ROC, like what you did with glm.fit$fitted.value. I have been searching about it but in vain. Appreciate it if you can give me a clue! Thanks very much!

@statquest

3 жыл бұрын

This might help: stats.idre.ucla.edu/r/dae/mixed-effects-logistic-regression/

@julieyananzhu1134

3 жыл бұрын

@@statquest Thanks for your kind reply! The web page didn't solve my problem directly, but it's very informative! Thanks!

I installed and loaded pROC, it says couldn't find function roc... Which editor u used? I used rstudio, help pleaseeee?

@statquest

3 жыл бұрын

I used RStudio as well. Sorry you'r having trouble.

i am following python for data science so far and got stuck after saw this video , best person like you using R language instead of python so what should i do and which one is best for data science and also in future purpose R program or python kindly let me know and enlighten me thanks in advance ..! little BAM

@statquest

4 жыл бұрын

They are both very useful. Python is a great language used in a lot of different situations and has a lot of good machine learning libraries. In contrast, R is very useful for doing statistics.... So I would recommend learning both if you have time.

Hello, thank you for your videos. If the thresholds refer to the "weight threshold", why are they expressed as 0.nr (zero point number). Can't understand the meaning. Can you hel with this please?

@statquest

4 жыл бұрын

Each weight corresponds to a probability of classifying someone as "obese", and traditionally the thresholds are in terms of these probabilities. Since probabilities go from 0 to 1, these thresholds go from 0 to 1. However, we could transform the thresholds to be in terms of weight if we wanted to.

@pierfrancescovisaggi7984

4 жыл бұрын

@@statquest Thank you very much for the answer, it's very kind of yours. Could I ask how to transform such probabilities in terms of weight? I only managed to do it without a linear model, just applying roc( myresponse, mypredictor). How could I transform all those thresholds into numbers corresponding to my predictor? Thank you

@statquest

4 жыл бұрын

@@pierfrancescovisaggi7984 Each value for weight has a corresponding probability stored in glm.fit$fitted values. See 4:40 . You can use the index for any weight to access the associated probability.

Excellent videio

@statquest

2 ай бұрын

Thanks!

Great video! I was wondering if it is possible to plot this graph in a Multinomial Logistic Regression?

@statquest

Жыл бұрын

Hmmm...I'm not sure.

@mathiasschmidt93

Жыл бұрын

@@statquest Ah okay, what about a multiple logistic regression? Any ideas about that one?

@statquest

Жыл бұрын

@@mathiasschmidt93 As long as your predicted value is binary, it shouldn't matter how many variables you use to make predictions - the process is the exact same as illustrated in this video. To see how it is done in R, see: kzread.info/dash/bejne/o5eqo9N6eJmWido.html

what is threshold?

hello i want to know how can i plot a roc curve from glm() function with excel data

@statquest

3 жыл бұрын

Maybe someone else with experience in this can help out.

AUC for Logistic regression is more than AUC for RF, but if you consider only corner most points for both, RF does better, so who is the winner in this case ?

@statquest

5 жыл бұрын

Which corner are you looking at? I don't see RF doing better in either one. Or are you looking at the very edges?

@animeshkansal7746

5 жыл бұрын

StatQuest with Josh Starmer At the north west corner Rf at a point has better tpp and fpp So isn’t rf better than logistic regression?

@statquest

5 жыл бұрын

@@animeshkansal7746 North east? You are right. RF is a little better up there. This is a good example of when a Partial AUC might be more informative.

@animeshkansal7746

5 жыл бұрын

Thank you so much, your videos are really great

@statquest

5 жыл бұрын

@@animeshkansal7746 Thanks!

thank you for your video. btw, can you make one for python?

@statquest

5 жыл бұрын

I'll work on it. I'm doing a lot more Python coding these days, so it makes sense.

@ccuny1

4 жыл бұрын

@@statquest A year later, I suddenly wake up to StatQuest. Python implementation please. Perhaps SciKit Learn also has built-in computations for these and other metrics. I'll check...

Is it expected that the number of sensitivity/specificity values determined by the roc function (that we stored in the data frame) may not match the number of predictor/response values that I input? For example, my input predictor/response vectors contained 46 objects, but the roc function returned only 12 sensitivity/specificity values.

@statquest

Жыл бұрын

I believe this is possible if there are fewer thresholds that make a difference. In other words, some thresholds might result in the same number of false positives, true positives etc. and in that case, those "duplicate" thresholds will be omitted.

@hannahhillman3593

Жыл бұрын

@@statquest Okay great this is exactly what I thought was happening--just wasn't sure if that was a possible outcome. Thanks so much for your reply and for all the great videos!!!

So do these thresholds correlate to the probabilities that are used to separate the obese vs. not obese? Is there a way to figure out how to convert the thresholds back to the actual weights themselves that are used as the cutoff?

@statquest

4 жыл бұрын

The thresholds, with the exception of -infinity and +infinity, are the exact same as the probabilities. -infinity corresponds to a probability of 0 and +infinity corresponds to a probability of 1. Thus, you can compare thresholds to the original glm.fit$fitted.values and match those to the original array of "weight" values.

@jarednesvet2826

4 жыл бұрын

@@statquest Great thanks for the help!

@redgreenskittles

4 жыл бұрын

@@statquest Many thanks for a great video. Could you kindly explain how exactly we can do this? I am looking to convert these threshold to actual cut-off values

@statquest

4 жыл бұрын

@@redgreenskittles First, I would look at the ROC curve to find my threshold. For the example, we might pick a False Positive Percentage of 20 to be the threshold. Then I would look in roc.info to find the threshold associated with that false positive percentage. We can do that by just printing roc.info to the screen and looking at it, or with the command... roc.df[min(which(roc.df$fpp

@redgreenskittles

4 жыл бұрын

@@statquest Wow that was a super quick response. Works like a treat! thank you