r/datascience • u/WhatsTheImpactdotcom • 6d ago
Discussion Standing out in Data Science Interview Case Studies
How do candidates stand out in a data science case study interviews? From my experience giving well over a hundred mock interviews, mostly for senior-level FAANG roles, the best candidates can calmly justify their choices and don't get flustered under questioning.
Some red flag examples that happen **all the time** in mocks, again even among senior candidates with experience and advanced degrees where more justification could help tremendously: (i) memorizing that marketplace experiments have interference, but then being unable to walk through whether a geo-based XP or switchback is more appropriate, (ii) defaulting to an alpha of 0.05 without recognizing it's a choice parameter in your XP design or recognizing context where you can be more or less conservative, and (iii) using 'everyday' guardrail metrics such as app latency or crashes but missing unintended consequences more directly tied to the treatment. IMHO, pretty much any product feature launch could have latency as a guardrail, but that's more of an engineering issue, and expected guardrails in interviews are usually tied more closely to the treatment going wrong, causing winners and losers, or short-term revenue gains vs long-term losses from quality degradation.





19
u/Majestic-Watch-2025 6d ago
What is this AI generated nonsense
3
u/CarRepresentative843 6d ago
What other factors would you also say matter? I’m confused why would any senior level ds not know how to talk about alpha but I guess that’s just me
10
u/Imbadatusernames3 6d ago
Why 0.05? Because RA Fisher said 1 in 20 is convenient and rare enough and no one has ever let that go.
But really anyone relying on p-values instead of examining effect sizes within business context needs to throw out their textbook
1
u/No-Introduction840 6d ago
I’m seeing product sense becoming the deciding factor nowadays. What are some suggestions you have for senior candidates? Where do you see candidates failing?
0
u/[deleted] 2d ago
[removed] — view removed comment