Welcome to my VLOG! My name is Yury Zablotski & I love to use R for Data Science = "yuzaR Data Science" ;)
This channel is dedicated to data analytics, data science, statistics, ML and AI! Join me as I dive into the world of data analysis, & coding. Whether you're interested in business analytics, data mining, data visualization, or pursuing an online degree in data science, I've got you covered. If you are curious about Google Data Studio, data centers & certified data analyst & data scientist programs, you'll find the necessary knowledge right here. You'll greatly increase your odds to get online master's in data science & data analytics degrees. Boost your knowledge & skills with my engaging content. Subscribe to stay up-to-date with the latest & most useful data science programming tools. Let's embark on this data-driven journey together!
If you wish to support me, please join the channel 🙏 kzread.info/dron/cGXGFClRdnrwXsPHi7P4ZA.htmljoin
Пікірлер
Hey there! I'm wondering where you get the information about the conventional thresholds for interpretation (like for p-values, Bayes, etc). There are so many different versions from different authors out there, which one should we trust? I'm really struggling to make up my mind! I already know about the effectsize package, but should we trust their frames of reference? Thanks in advance sir.
The code link in the description does not seem to be working anymore, do you have an updated one by any chance?
Absolutely amazing
Thanks so much 🙏 hope you enjoy other topics too!
Hello Sir. Is it possible to display only one category of the dichotomous variable provided with the "by" parameter. For example I only want to display the percentage of people who said Yes (sport-Oui). library(gtsummary) library(questionr) data("hdv2003") hdv2003 %>% tbl_summary( include = c("sexe", "relig", "relig"), by = "sport", percent = "row", statistic = all_categorical() ~ "{p}%" ) . The objective is that I would subsequently like to combine several tables where in the column I will have the percentages of several variables.
Yes it’s possible. Check out the arguments of the Funktion please yourself, I am away from my computer for a week. And you can combine several tables easily via tbl_merge
@@yuzaR-Data-Science Thank you
I found modify_column_hide function that can do it
Hi professor.Is there any way we can dynamically adjust line in quantile regression plot and it should affect other plots
Which line do you mean? And which other plots?
Amazing
Thanks 🙏
Amazing video. Thank you.
Glad you liked it! Thanks for watching!
Great video
Glad you enjoyed it! Thanks for watching!
Excellent video🙌 It would be great a video of bayesian analysis
Noted! Thanks for your positive feedback! 🙏
Thank you for this video. I just have two question. 1. You showed the plot of the variable importance ("plot('model_name', type = "s")). Is there a way to extract the names of the variables and/or interactions using a threshold (e.g. 75%)? I need them as a list, e.g. "variab_interactions_plus_75 <- ????". 2. I used the example data "mtcars". Is an interaction "hp:cyl" equal to an interaction "cyl:hp"?
Hi Hendrik, not that I know of. Once I needed to use for my paper, I created the data frame manually and only put the things inside, I wanted. But I think it's not that much more work, after algorithm done the whole heavy lifting ;) cheers
I also subscribed. Your videos are always very informativ and helpful. Thank you.
Thanks for the sub! And for watching! I am happy you like my content!
Wow! Thank you! So many important informations. I have to watch this video several times. But one question: In which order would you use the packages "janitor" and "dlookr". Would be interesting to teach people how to load and handle "dirty" excel table, fix some excel problems (e.g. date as numbers or entries like "no data" in numerical columns etc) and if those problems are fixed to use "dlookr" to diagnose, explore and repair the data.
Thanks a lot for your nice reply, Hendrik! I would use janitor first and dlookr on top. I guess you already have seen the janitor video on my channel. If not, feel free ;) I also have one video on tidy data, where I show the dirty table, but there is not much of R programming. Thanks for watching!
Wow! One of the best videos I have ever seen. Vwer informative.
Wow, thanks for such a generous feedback! If you know some folks who also would benefit from it, feel free to share it! I wish I had something like this video as I started to learn R. I hope the other videos are helpful too! Thanks again! Cheers!
Thanks for sharing your knowledge. How can I add the Yate´s correction into the plot?
Unfortunately, either not possible, or I don't know why and how. But thanks for good feedback!
Thanks for sharing your knowledge. How can I add the Yate´s correction into the plot?
Unfortunately, either not possible, or I don't know why and how. But thanks for good feedback!
This is an excellent video!! I was thinking, a nonparametric alternative for linear regression could be LOESS regression and boostrapp could be done without problem but, because LOESS is a nonparametric, instead of medians the means could be used properly or also in this case the medians should be used?
While resampling allows for a better use of means, I am a big fan of medians, because if the distribution of anything after bootstrapping does not get normal, like in the case of p.values, I would trust the median, but not the mean. So, I would use median as much as I can.
@@yuzaR-Data-Science Ok, I really thank you for answer!
you are very welcome!
@@yuzaR-Data-Science I had another question. Althoguh bootstrapping is not exactly an option to handle outliers, could be the case that the more resamples used, the more robust is the model to outliers?
Yes, because then you would resample the most frequent cases more often, so their distribution would be higher, and the outliers ... hmm, we would not get rid of them, but they will be resampled very rarely. hope that helps. cheers
man u just won another subscriber. This is one of the best videos about dplyr that i have ever seen :). Congrats dude!
Glad you enjoyed! Hope other videos are helpful too! Thanks for a nice feedback!
Is it possible that alongwith P value we also have Chi Square Statistics value & df value etc in GtSummary table..if yes what's the code
Not straight forward unfortunately. You could create a data frame and use flextable and the export it in word or png
I'm doing better analysis of my PhD data because of your videos, thanks a lot!
I am glad my content is helpful! Thanks for watching and commenting! That’s the best support 🙏
why not use the Chi-square test goodness-of-fit also?
you actually can, here is what I think about them: Close relatives: connect the dots Ironically, there is nothing exact about Exact Binomial test. It is called “exact” because it simply calculates the p-value directly from the probability, and not from any kind of statistics, like the Chi-Square. However, Chi-Square’s Goodness of Fit test is only approximation for a p-value, that is why the exact binomial test is recommended. Proportion test If you have lots of data (N > 30) or more than two outcomes, use a proportion test which is highly similar to the exact binomial test. In fact, the Exact binomial test is exactly the same as the proportion test with Yates continuity correction, which is used by the proportion test by default. Below, I explicitly wrote down such correction: prop.test(x = 7, n = 10, p = 0.5, correct = T) 1-sample proportions test with continuity correction data: 7 out of 10, null probability 0.5 X-squared = 0.9, df = 1, p-value = 0.3428 alternative hypothesis: true p is not equal to 0.5 95 percent confidence interval: 0.3536707 0.9190522 sample estimates: p 0.7 One sample Chi-Square test However, as you can see above, the proportion test calculates the chi-squared statistics, so it is actually calling a chi-squared test. And interestingly a proportion test without Yates continuity correction gives identical results to a Goodness-of-Fit One-sample Chi-Square test: prop.test(x = 7, n = 10, p = 0.5, correct = F) 1-sample proportions test without continuity correction data: 7 out of 10, null probability 0.5 X-squared = 1.6, df = 1, p-value = 0.2059 alternative hypothesis: true p is not equal to 0.5 95 percent confidence interval: 0.3967781 0.8922087 sample estimates: p 0.7 chisq.test(c(7,3)) # total is 10 and p = 0.5 for both numbers by default Chi-squared test for given probabilities data: c(7, 3) X-squared = 1.6, df = 1, p-value = 0.2059 (Simplest) Logistic regression If you are not overwhelmed yet, you can also go further and conduct the simplest logistic regression possible (don’t need to understand it now! I’ll cover it in different videos). Below you’ll find an log-odds output of the logistic regression (0.847) which can be expressed in probability using the plogis function. You’ll see that the probability is exactly 0.7, as in the tests above and the p-value is similar. By the way, the p-value represents the probability of observing a result as extreme or more extreme than the one you got, assuming the null hypothesis is true. m <- glm(c(rep(1,7), rep(0,3)) ~ 1, family = binomial()) broom::tidy(m) # A tibble: 1 × 5 term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 0.847 0.690 1.23 0.220 plogis(0.8472979) [1] 0.7
@@yuzaR-Data-Science Thank you for your attention and complete answer. This was very insightful. Also thanks for your video about regressions vs statistical tests, which I wondered about that for months or more (since I don't have a strong background on statistics, I'm a Biologyst who (tries to ^^) enjoy statistics). So, the proportion test calculates the pvalue from the Chi-square value as in the Chi-square test unlike binomial test? At least I can see a clear advantage given that proportion test gives the confidence intervals and I would expect that the Exact binomial test to be similar to the binomial glm but in fact it's similar to Chi-square pvalue. Is this due to the Yates continuity correction?, I found some sources saying is can be a bit conservative. For some reason, I tried to do the Chi-square test and it gives me the same result with continuity correction (which is the default it seems, and doesn't specify which) and not corrected, equal to the proportion test without correction. When i used pvalue by Monte Carlo simulation it gives something closer to the Yate's continuity correction Chi-squared test for given probabilities with simulated p-value (based on 2000 replicates) data: c(7, 3) X-squared = 1.6, df = NA, p-value = 0.3543 What is best? Using continuity corrections or alpha adjustments for multiple outcomes, or both?
Monte Carlo simulation is the best I think and adjustment for multiple comparisons is always a must.
Great video! Is it possible to only show the pairwise comparisons between one group e.g. 'Original' and Synthetic1, Synthetic2, Synthetic3 ... etc? It also does pairwise comparisons between those synthetic groups which I don't want to show and also don't want to conduct tests for except with the original one. Having a separate one by one figure takes up a lot of space so wondering if this is possible?
I think it’s difficult with this function, although not impossible. But it’s much more practical to model it, with for example quantile regression, and use tab_model function from sjPlot package 📦 I have videos on both if need some assistance for a start
Very nice video! It helped me a lot.🤓 I have a question here, when I try to build a function for mixed effect models and continue to the next step of glmulti, it warns me that "Error during wrapup: unused argument (REML = F) Error: no more error handlers available (recursive errors?); invoking 'abort' restart". Could you pls tell me how might this happen and how to solve this problem? That would be very appreciated!😳
hi, thanks for feedback! first, have you installed all the important packages (lme4, lmerTest ... etc.)? and secondly, have you tried this? glmer.glmulti<-function(formula, data, random = "", ...){ glmer(paste(deparse(formula),random), data = data, REML = F, ...) } mixed_model <- glmulti( y = response ~ predictor_1 + predictor_2 + predictor_3, random = "+(1|random_effect)", crit = aicc, data = data, family = binomial, method = "h", fitfunc = glmer.glmulti, marginality = F, level = 1 )
@@yuzaR-Data-Science Thanks for your reply! Yes, I followed these steps, and still got the error warning.
then I guess there are two many predictors. if I try it with ca. 20 it collapses and I need to restart rstudio. so, reduce the number of predictors to the most sensical ones and then run the glmulti
@@yuzaR-Data-Science I'll try this. Thank you soooo much for your kind suggestion！
sure, let me know whether it worked. by the way if you use "d" instead of "h", you can see how many models you are going to make, and if it goes over 6figures, I would reduce the number of predictors first and then use the glmulti
Thank you sir
So nice of you! Thanks you for watching and commenting!
I think your the best explaining statistics in such smooth way. I'm wondering if your blog is out of service?
Thanks man! Greatly appreciate your positive feedback. My blog was shut down, since they want me to pay. I refuse to pay, because I actually do something useful for the world for free. So, hope for your understanding. The good news is, youtube is still free and will stay free, so, when you just stop the video at any time and type the code it's free. However, when you want to see the whole code from any of the video, you could join the channel (kzread.info/dron/cGXGFClRdnrwXsPHi7P4ZA.htmljoin), because for the members, I do provide the whole code.
Grear work. I look forward to your videos on Survival Analysis in R
Thank you very much! Will definitely do. Just made some videos on linear regression in R. Logistic will follow and then the rest of models including Survival and ML one day. 3 years ago I've done two videos on survival already, but they are old, theoretical and low quality. I'll redo them in a more concise and R focused way. Thanks for nice feedback and for watching!
Great work. Please do videos on Survival Analysis in R as well😊
Thank you very much! Will definitely do. Just made some videos on linear regression in R. Logistic will follow and then the rest of models including Survival and ML one day. 3 years ago I've done two videos on survival already, but they are old, theoretical and low quality. I'll redo them in a more concise and R focused way. Thanks for nice feedback and for watching!
brilliant stuff. Very easy to follow. Can you create a dedicated playlist regarding machine learning models ?
Great suggestion! I'am on it, just made some videos on linear regression in R. Logistic will follow and then the rest of ML one day. Thanks for nice feedback and for watching!
@@yuzaR-Data-Science you have no idea how easy these are to follow. thanks again. keep it up.
thanks! do my best to upload more, but this summer is pretty busy.
Excellent content quality! You sir always find a way to keep us interestingly attached to the video. You are like our Statistics and Data Science dealer. Thanks for your labor. We'll keep growing up.
Wow, thank you! Your comments, my friend, are the most supportive and motivating! So, after reading them I just wanna jump straight into creating a new video on one of the 1000 ideas I have :) For instance, I am finally starting the logistic regression series. One of my favorite topics ;) Genuinely thankful for your continuous support! Cheers!
Thanks for this but is this not the classic case when p-value adjustments for multiple testing need to be applied? Why wasnt it applied? Also, in a situation where we have only 2 possible outcomes, when should the binomial test be used versus Chi-square? Would this be based on preference or one is objectively more appropriate versus the other?
Man, you ask good questions! ;) first, yes, you are absolutely right, p-values adjustment would be the right thing to do, but to the time I have done the paper, I used it irregularly. For the video I also try to keep the focus, so it's concise.
And since I try to keep videos short, some infos does not end up there, but was considered, while I was writing the script. For instance, check these parts below, I hope you'd find them useful: Intuition Ironically, there is nothing exact about Exact Binomial test. It is called “exact” because it simply calculates the p-value directly from the probability, and not from any kind of statistics, like the Chi-Square. However, Chi-Square’s Goodness of Fit test is only approximation for a p-value, that is why the exact binomial test is recommended. Proportion test If you have lots of data (N > 30) or more than two outcomes, use a proportion test which is highly similar to the exact binomial test. In fact, the Exact binomial test is exactly the same as the proportion test with Yates continuity correction, which is used by the proportion test by default. Below, I explicitly wrote down such correction: prop.test(x = 7, n = 10, p = 0.5, correct = T) One sample Chi-Square test However, as you can see above, the proportion test calculates the chi-squared statistics, so it is actually calling a chi-squared test. And interestingly a proportion test without Yates continuity correction gives identical results to a Goodness-of-Fit One-sample Chi-Square test: prop.test(x = 7, n = 10, p = 0.5, correct = F) chisq.test(c(7,3)) # total is 10 and p = 0.5 for both numbers by default (Simplest) Logistic regression If you are not overwhelmed yet, you can also go further and conduct the simplest logistic regression possible (don’t need to understand it now! I’ll cover it in different videos). Below you’ll find an log-odds output of the logistic regression (0.847) which can be expressed in probability using the plogis function. You’ll see that the probability is exactly 0.7, as in the tests above and the p-value is similar. By the way, the p-value represents the probability of observing a result as extreme or more extreme than the one you got, assuming the null hypothesis is true. m <- glm(c(rep(1,7), rep(0,3)) ~ 1, family = binomial()) broom::tidy(m) plogis(0.8472979)
❤
🙏
Nice lecture for resampling. Please make a video for simulation study
Thanks 🙏 Sunil, I’ll do. But it’ll take some time, because I first want to cover frequentists stats. Then come to simulation
Nice video and excellent explanation 👍
Thanks 🙏 Sunil! Glad you enjoyed it!
Nice video as usual, keep up the good work
Thanks, will do! Appreciate your continuous support!!! Commenting, watching and liking is really the best support! So, thanks again!
Your videos are intuitive. Can you start a playlist on machine learning in R?
thanks man! yes, that's the plan, but first I would cover stat models, like logistic regression etc. after that I would go full ML and AI ;)
Very insightful. Thank you vey much. Please can I get your email. I will like to ask you some stuff I find confusing since you are the expert :)
Hi Richmond, thanks a lot for your nice feedback! I do not share my email, but that's no problem, because you can ask anything here in the comments section of videos and I would do my best to answer as quick and as good as I can. The channel members get quicker and more insightful responses though, and the higher their level, the more time I can invest into answering questions, thus if you wish, join my channel: kzread.info/dron/cGXGFClRdnrwXsPHi7P4ZA.htmljoin
@@yuzaR-Data-Science Thank you very much for such informative videos. I spent several years in class and didn't understand all these concepts, but watching this video has made things easier for my comprehension. I have a few questions I would like to ask: When performing a statistical test, we use a parametric test if the data or variable in question is normally distributed, and a non-parametric alternative if the data or variable is not normally distributed. My question is: when does the central limit theorem come into play here? Also, a colleague of mine told me to always use parametric tests even if the data is not normally distributed. His explanation was that parametric tests are more powerful than non-parametric tests. So, should I straightforwardly use the non-parametric alternative when I observe that my data is not normally distributed, or should I take the CLT into consideration and use the parametric test?
I am not sure the CLT helps too much, but using parametric test for a highly skewed data is absolute nonsense. The power difference is minimum and is overrated. I also have colleagues who use non-parametric tests by default. Another extreme nonsensicality and laziness. Just for the sake of learning effect, please, take skewed data and calculate mean and median to see how much difference you'll get. And if your colleague would really care about power, he/she would use multivariable models, not univariable tests. And this is what I would recommend to you - the test are fine, in the beginning - but try to learn models and their assumptions when you want to go to the next level. Cheers and thanks again for a nice feedback!
@@yuzaR-Data-Science I'm really grateful for finding time in your busy schedule to reply me. So please when should I use the CLT or I shouldn't use it at all. Thank you
but what exactly do you mean by CLT? bayesian methods?
🙏👍💪😎
thanks!
best R content out-there!
Glad you still enjoy it ;)
Thanks but why is it called ‘exact’?
amazing question! Thanks :) I was thinking to put it into a video actually, but deleted it as "boring" and "less useful" part of the script :) Here is what I was going to say: Ironically, there is nothing exact about Exact Binomial test. It is called “exact” because it simply calculates the p-value directly from the probability, and not from any kind of statistics, like the Chi-Square. However, Chi-Square’s Goodness of Fit test is only approximation for a p-value, that is why the exact binomial test is recommended. Hope it answers the question :)
Sir please make a video lecture on Simulation study.
Thanks 🙏 Sunil, I’ll do. But it’ll take some time, because I first want to cover frequentists stats. Then come to simulation
@@yuzaR-Data-Science Sure Sir I will wait the video. Thank you so much Sir 🙏
You are very welcome! :)
🙏👍💪😎
thanks a lot!
🙏👍💪😎
thanks!
great practical content! thanks
Glad it was helpful!
top content, very concise and to the point! thanks!
waw, thanks for such a generous feedback!
I can't stop watching your videos ;) please produce more of them, It's really fun to learn from your content.
learning addiction is the best addiction ever! ;)
I have a question plz, Why did we put “temp” as a predictor to imputate missing values in Ozone variable ?
simply as an example of a predictor
@@yuzaR-Data-Science Am sorry, I cant get it !
sorry, what exactly can't you get?
sorry, what exactly can't you get?
@@yuzaR-Data-Science Do we have to put a predictor to impute missing values in a variable?
I love this channel. As always high quality content. I am happy I have also found you on LinkedIn.
Glad you enjoy it! And thanks sooo much for your positive feedback! It's very encouraging! 🙏🙏🙏
good quality content as always thanks! keep up a great work!
thanks for your continuous support! :)
Excellent demonstration in R.
thanks a lot Jeff, glad you enjoyed it! :)
I love your channel, I think it's one of the best channels in KZread right now
Wow, thanks! That's the best feedback ever! Greatly appreciate! Hope my future content will be useful too!