AB Testing Case Study | Google and Spotify Data Scientists | Data Science Interview

Ғылым және технология

🚀 Land your dream data job using datainterview.com/.
====== ✅ Details ======
🤔 "How would you design an AB test to decide whether to change the color of the 'Buy Now' button from orange to green on Amazon.com?"
Dan (Ex-Google/PayPal Data Scientist) and Felicia (Spotify Data Scientist) partnered up to create a discussion video and technical design doc on how to approach an AB test problem based on Amazon's shopping experience.
The video is a preview of the entire discussion covering:
1. Problem Statement
2. Background (User Journey, User Target)
3. Hypothesis Statement (Success Metric, Guardrail Metrics)
4. Experiment Design (MDE, Power, Alpha, Sample Size)
5. Running an Experiment (Ramp Up, Validation Checks)
6. Launch Decision (Decision Tree, Post-Launch)
If you want to see the entire discussion and get access to AB testing courses, case course, product SQL course, Slack group and much more, make sure to check out datainterview.com/
👍 Feel free to subscribe, like and share!
====== ⏱️ Timestamps ======
0:00 Intro
06:11 Problem Statement
08:18 Background
14:15 Hypothesis Statement
====== 📚 Other Useful Contents ======
1. Principles and Frameworks of Product Metrics | KZread Case Study
/ principles-and-framewo...
2. How to Crack the Data Scientist Case Interview
/ principles-and-framewo...
3. How to Crack the Amazon Data Scientist Interview
/ crack-the-data-scienti...
====== Connect ======
📗 LinkedIn (Felicia) - / feliciarutberg
📗 LinkedIn (Dan) - / danleedata
📘 Medium (DataInterview) - / datainterview

Пікірлер: 23

  • @Bethechange766
    @Bethechange7662 жыл бұрын

    Hi, Please share the remaining part 4. Experiment Design (MDE, Power, Alpha, Sample Size) 5. Running an Experiment (Ramp Up, Validation Checks) 6. Launch Decision (Decision Tree, Post-Launch)

  • @brietje123456
    @brietje1234562 жыл бұрын

    Not really sure why we need the "first session only" constraint? If you just sum/average per user you can also get results "promptly", right?

  • @flyyang001

    @flyyang001

    2 жыл бұрын

    I agree, the first session only would also be impacted by the novelty effect.

  • @xxd3030
    @xxd30302 жыл бұрын

    Hey Dan, I really love your courses on KZread and purchase the monthly subscription a week ago. Now I could not login due to Google's prevention: "Attackers tried to steal your information"

  • @rafaBR
    @rafaBR2 жыл бұрын

    Playback Speed 1.5x recommended

  • @AlirezaAminian
    @AlirezaAminian Жыл бұрын

    You mentioned that CTR is not a good metric which is understandable, but how about CTP? In CTP, we are looking at unique page visits and unique clicks, therefore, if a user has multiple sessions and multiple visits, they will all be counted as one.

  • @samw2066
    @samw20662 ай бұрын

    Question on metric tested in the hypothesis test - avg. # of clicks per user. Why wouldn't it be avg revenue per user per day? I believe the user could click the "buy now" button and still exit, or they may buy but spend less money with the new button,etc....

  • @olhakyrychanska5254
    @olhakyrychanska52544 ай бұрын

    Incredible! Professional yappers, very useful for practicing corporate English✍️😁

  • @garyboy7135
    @garyboy71355 ай бұрын

    Use delta method to account for the repeated measures

  • @kristiyanto
    @kristiyanto2 жыл бұрын

    What's the purpose of having a guest speaker if you don't let her talk?

  • @squeeps33

    @squeeps33

    2 жыл бұрын

    Well they made the presentation together and she had quite a bit of engagement imo. He asked her to talk about the hypothesis statement and they had some solid dialogue... I guess the idea was that he would guide the presentation passively, and she'd give great insights/alternate. I learned quite a bit

  • @ankitbiswas8380

    @ankitbiswas8380

    Жыл бұрын

    thats your whole take away from the video ?

  • @raj-nq8ke
    @raj-nq8ke2 жыл бұрын

    Gold

  • @garyboy7135
    @garyboy71352 жыл бұрын

    On 24:30 about not using CTR due to violation of iid, I think you’re missing some important info, would love your feedback: 1. If you’re aggregating CTR to per user across session, your unit are users and still iid, assuming users are independent 2. Even if measuring at session level, you can still use delta method to estimate variance where otherwise variance would be underestimated due to sessions are not iid

  • @iqless7313
    @iqless7313 Жыл бұрын

    A user would be way more likely to click a 'buy now' button for a cheap, insignificant item. I think this definitely needs to be controlled for in the experiment.

  • @abhishekchandrashukla3814
    @abhishekchandrashukla381418 күн бұрын

    She is beautiful

  • @aravindkramesh
    @aravindkramesh2 жыл бұрын

    *God bless me, please. I wish I could get my dream job.*

  • @kjcooper17
    @kjcooper17 Жыл бұрын

    So wordy….. get to the points

  • @iizzcool

    @iizzcool

    Жыл бұрын

    some people love the sound of their own voice

  • @ASURAN24
    @ASURAN242 жыл бұрын

    Sorry, I think you don't know statistics. T-test is a parametric test.

Келесі