A/B Testing Fundamentals: What Every Data Scientist Needs to Know!

Data Scientists Must Know | Data Science Interview Must Watch: A/B Testing Fundamentals. I will dive deep into practical problems in A/B testing and share things I learned from the book Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing.
🟢Get all my free data science interview resources
www.emmading.com/resources
🟡 Product Case Interview Cheatsheet www.emmading.com/product-case...
🟠 Statistics Interview Cheatsheet www.emmading.com/statistics-i...
🟣 Behavioral Interview Cheatsheet www.emmading.com/behavioral-i...
🔵 Data Science Resume Checklist www.emmading.com/data-science...
✅ We work with Experienced Data Scientists to help them land their next dream jobs. Apply now: www.emmading.com/coaching
✔️ Cracking Hypothesis Testing Problems in Data Science Interviews part 1
• Mastering Hypothesis T...
✔️ Cracking Hypothesis Testing Problems in Data Science Interviews part 2
Practical significance and statistical significance
• A/B Testing Analysis M...
✔️ Cracking A/B Testing Problems in Data Science Interviews
• Crack A/B Testing Prob...
📚 Trustworthy online controlled experiments, a practical guide to A/B testing
www.amazon.com/Trustworthy-On...
// Comment
Got any questions? Something to add?
Write a comment below to chat.
// Let's connect on LinkedIn:
/ emmading001
====================
Contents of this video:
====================
0:00 Intro
1:52 What are A/B tests
3:39 Why A/B tests
5:05 Steps to run A/B tests
5:20 Experiment prerequisites
6:51 Experiment design
8:03 Running Experiments
8:22 Result to decision
10:48 Post-launching monitoring

Пікірлер: 39

  • @zenofall4455
    @zenofall44553 жыл бұрын

    Emma guided me through my interview prep, and I got 3 DS offers, she had suggested this book to me, this book is awesome.

  • @emma_ding

    @emma_ding

    3 жыл бұрын

    That's awesome news! Congrats on your offers!

  • @juanfelipehernandezmolina5185

    @juanfelipehernandezmolina5185

    2 жыл бұрын

    do u recommend it ? does it covers all she explained ?

  • @nicoleli8948
    @nicoleli89483 жыл бұрын

    谢谢Emma! 你的视频真的很有帮助~继续加油!

  • @surajbhala5298
    @surajbhala52983 жыл бұрын

    Thanks Emma , for making these videos. Looking forward to the next part 👍

  • @jeoffleonora4612
    @jeoffleonora46123 жыл бұрын

    Feel like I have a lot to learn about ab testing. Can’t wait to see the next few videos!

  • @zhenzhang3141
    @zhenzhang31413 жыл бұрын

    thanks for your videos! Plz keep updating these great contents and it is super helpful!

  • @lydiamai6861
    @lydiamai68613 жыл бұрын

    This video helps me in learning statistics and probability. Thanks Emma for making this video.

  • @zishenli4195
    @zishenli41953 жыл бұрын

    谢谢分享!视频条理清晰!期待更多视频!

  • @katekatebangbang2435
    @katekatebangbang24353 жыл бұрын

    Emma thanks for another amazing video! 希望能看到你出更多的product sense相关的视频:D

  • @tejasphirke3436
    @tejasphirke34363 жыл бұрын

    I would love to have more of this videos.. 😃

  • @yanliang79
    @yanliang798 ай бұрын

    Hi Emma, thank you for the helpful video! I found on the side when you were introducing Experiment Design, the bullet points number for "size of an experiment" and "how long to run an experiment" are both 2.

  • @jiehu1337
    @jiehu13373 жыл бұрын

    nice video, your blue sweater perfectly echoed with google color schema test!

  • @zheng9470
    @zheng94702 жыл бұрын

    Hey Emma, thanks for uploading this. It's very helpful. And I've a question here: what's the difference between sample size in experiments and the number of randomization units? Thanks in advance.

  • @thampasaurusrex3716
    @thampasaurusrex37163 жыл бұрын

    thank you. See you soon!

  • @hasnanhussain6512
    @hasnanhussain65129 ай бұрын

    Thank you!

  • @zeq9021
    @zeq90212 жыл бұрын

    Love these videos on AB Testing! Any chance we can get a video on methods of experimenting when AB-Testing is not possible? (i.e. Synthetic Control methods, switchback experiments, etc.)

  • @emma_ding

    @emma_ding

    2 жыл бұрын

    Hi Eze, thank you for the feedback! I'll definitely work on creating more helpful content. Stay tuned! :)

  • @mariademuri2855
    @mariademuri28552 жыл бұрын

    This is really helpful!! Not sure how easy this is to do but I was wondering if you would be able to make transcripts available for the video. (I personally have a learning disability so it really helpful for me to have a transcript to accompany a video. Thought this might be also helpful for people where english isn't their first language too)

  • @ankitranjan30
    @ankitranjan302 жыл бұрын

    Thanks for uploading this video, Emma. It' very helpful. I had a doubt regarding A/B/N testing and multivariate testing. I designed a full factorial experiment where I had 6 groups (factor A had 3 levels and factor B had 2 levels) without a control group. I checked that my residuals had constant variance and my data was normally distributed and so I was able to use Tukey's multiple comparisons. The result was that the best two groups (highest mean response indicates best in my use case) were not statistically different and even though it was practically significant as per an arbitrary self established metric, I would not recommend the winning group as it was not statistically significant (I believe this is the sanity check as the investment in winning group was worth it). I wanted to know if this problem falls under A/B/N testing or multivariate testing? Thanks in advance!

  • @imranm93
    @imranm932 жыл бұрын

    Hi Emma! Great video as always. You mention that in experiment design you use seasonality, day of week, primacy and novelty effect to determine how long to run the test for. But in your A/B testing interview video you only mention sample size. I must’ve missed something but are you able to explain why there’s that difference? Thank you!

  • @user-bq6ln4ne4h
    @user-bq6ln4ne4h4 ай бұрын

    Emma, Thank you for this helpful video! Could you please help me understand the difference between "have enough randomization units" and "size of an exp".

  • @rongzhang3242
    @rongzhang32422 жыл бұрын

    Hi Emma, I found a lot of case of A/B testing if about the website users. What is there is a national retail chain company that has 1000 stores around the country. And they want to test if a coupon can help increase sales. In this case, what is the randomization units here? the stores or the customer? Does A/B testing still apply in this case?

  • @MrJohn2brown
    @MrJohn2brown2 жыл бұрын

    Hi Emma, Any book you recommend to learn Root Cause Analysis or Failure Analysis ? Thank you

  • @manjariganapathy3574
    @manjariganapathy35742 жыл бұрын

    Hi Emma. Thank you so much for your video. Can you make a video on what to include in a resume to increase the chances of resume selection for DS interview?

  • @emma_ding

    @emma_ding

    2 жыл бұрын

    Hey Manjari, I have a video on the exact topic you are asking, here you go kzread.info/dash/bejne/qoeIpc-cqLuTgqw.html!

  • @matt95023
    @matt950232 жыл бұрын

    Thanks for the great video! How would you design an A/B test for a social feature on a social media site (e.g. facebook stories)? The users are interconnected with each other via posts, shares, likes, etc., rather than having their own isolated experience such as on e-commerce websites. If we randomly spilt users into 2 groups for the A/B test, users in the A group are going to be interacting with users in the B group. Could that bias the results?

  • @arunas1573

    @arunas1573

    2 жыл бұрын

    Absolutely! The way interaction effect is overcome in networks is by using cluster sampling. LinkedIn uses the concept of ego networks to identify clusters and randomization is done at the cluster level. This will minimize spill over effect. Metrics will be calculated at the cluster level and averaged at treatment and control group level.

  • @purushottamraj2430
    @purushottamraj24302 жыл бұрын

    @Data Interview Pro Please can you share all slides of A/B testing using google drive or any other application etc.

  • @anathemaconscience5666
    @anathemaconscience56662 жыл бұрын

    HI emma. I have a question, so why don't people do sanity check before running the experiment? So they can save energy on running an unqualified ab testing.

  • @emma_ding

    @emma_ding

    2 жыл бұрын

    Hi! Good question, we should do it before and after.

  • @Seth_majestus
    @Seth_majestus2 жыл бұрын

    An experiment. I still dont get it. How can I estimate how long to run A/B testing?

  • @liruisi2196
    @liruisi21962 жыл бұрын

    Hi, thank you so much for the great content! But I can not receive the "product case interview cheat sheet" even I registered my name and email through the link. How can I receive it? Thank you so much!!!

  • @emma_ding

    @emma_ding

    2 жыл бұрын

    Thank you for your interest in our product case interview cheat sheet! We have resent the email. Feel free to reach out to info@datainterviewpro.com if you have any other questions.

  • @Squiderify
    @Squiderify3 жыл бұрын

    Do you plan to make the slides available? It would be extremely useful. If so, you could also put a watermark on it because I bet it would be shared very often.

  • @emma_ding

    @emma_ding

    3 жыл бұрын

    That's a good suggestion. However the "slides" are actually video editing, i.e. part of the video. I will keep your suggestion in mind. Thanks.

  • @stabvdhg7671
    @stabvdhg76712 жыл бұрын

    COULD I GIVE YOU A SUGGESTION? IT IS BETTER TO ARRANGE YOUR VIDEOS IN ORDER.

  • @emma_ding

    @emma_ding

    2 жыл бұрын

    Hey Wei, this playlist may be helpful for you kzread.info/head/PLY1Fi4XflWSvgsaD9eXng6N5kxcMtcxGK :)

  • @CosmicNomad

    @CosmicNomad

    3 күн бұрын

    I agree with this suggestion. Emma, you have a lot of useful videos in the playlist but if you could sort in the order you think is the logical sequence of learning A/B testing, that'll make the playlist even more helpful. Right not this video is at the bottom of the playlist but it seems you think this is the starting point of learning instead.