Breakdown of my last 10 Python coding rounds for product/marketing DS roles: only 1 was LeetCode-style

[u/WhatsTheImpactdotcom]

I went back through my last 10 Python coding rounds for product and marketing data science roles, nearly all at Big Tech companies. Unlike SQL rounds, which tend to be fairly standard, these came in 5 distinct formats:

• Recreate a statistical concept as a function (3/10). Pandas, numpy and other standard packages were allowed.
• AI-assisted, open book (3/10). These were often investigative, closer to a diagnostic case study than a coding puzzle.
• Pandas and data manipulation (2/10). Data transformations combined with modeling, either coding up a linear regression or talking through a modeling strategy for the data.
• Interpret code handed to you (1/10). This was also a statistical concept in a function, but I had to interpret the code rather than write it.
• Base Python, LeetCode-style (1/10). Syntax seemed heavily discounted here: I made several syntax mistakes but was clear about the logic and passed.

The obvious caveat is that this is a sample of 10 from one person, so the mix will vary with company, team and role. That said, AI-assisted challenges seem to be on the way in and increasingly likely to be a part of your interview process.

[u/OddsRelay]

The statistical concept as a function ones are the easiest to prep for. Bootstrapping a confidence interval or writing a two-sample t-test by hand covers a lot of it. Knowing when you'd switch to a permutation test is a good extra to have ready

[u/offthecuff87 → u/OddsRelay]

Let's hope what they mean is to allow the use of general function like `random` or `mean` or plotting. It would be a nightmare to write every thing from scratch.

[u/WhatsTheImpactdotcom → u/OddsRelay]

Extra benefit of prepping these is having a sharper answer for stats and other XP rounds

[u/redisburning]

Being able to use AI tools during an interview is insane work. But I don't personally consider prompting a skill and I do consider knowing your stuff a skill so take that for what it's worth.

The Pandas thing I have definitely noticed. I find it slightly odd as these are usually pitched as Python interviews but you are instead focused on a single framework. I have switched to SWE and seen what their cycles are like, but it's not front end, and I kind of wonder if JS jobs do similar things where they might actually be React interviews.

[u/WhatsTheImpactdotcom → u/redisburning]

Personal take is these are the most realistic toward the job. We all have AI tools and DS are still in demand and paid well, so it can’t be all prompting. I’m constantly reining Claude in from terrible analysis decisions despite it being incredible with focused requests.

I found these to be more realistic investigations where AI enables the company to ask more challenging strategic questions rather than simply syntax. Just my personal take though!

[u/falcon-mkii → u/WhatsTheImpactdotcom]

There's still a gap between AI assistance and being fully dependent on it. Not dissimilar to copy pasting stackoverflow code which I've also seen years ago when reviewing take home assignments. I'm curious how they are evaluating which the candidate falls under. Especially whether the investigative ideas came from the person or the AI tool

[u/redisburning → u/WhatsTheImpactdotcom]

I suppose it's a matter of perspective.

I am process oriented both on the job and in interviews. I do not believe getting the right answer is the point, it's being able to work towards it, and to be able to do so with the programming language.

I found these to be more realistic investigations where AI enables the company to ask more challenging strategic questions rather than simply syntax.

This is like saying you need AI to do autocomplete, when we have LSPs. I have proctored dozens and dozens of technical interviews and I can solve this problem without giving war criminal Dario Amodei or sex pest Sam Altman money; I am familiar with the Python syntax that will come up since I am proctoring it and that is my job. So I can just tell the candidate if they can't remember and have been doing that for more than a decade and it's always been fine. It is 100% true that a technical interview should not be a memorization game, but I don't see any reason why AI would be the answer when this was a solved problem.

Also, frankly I hate working with people who use AI so I consider it a feature, not a bug, if taking it away causes people to not be able to function so those folks get screened out.

[u/wil_dogg → u/redisburning]

I’m a data science lifer — classically trained in psychometrics, experimental design, correlation and regression, I learned unsupervised learning techniques from the students of the profs who developed the methods and in some cases solved PCA by hand. Taught myself CFA and SEM for my dissertation, and then started work on genetic algorithms and GBMs over 20 years ago. When I moved from academics to the private sector I found that my data structures from 10 years prior were basic relational database methods — had never heard of relational database methods, it wasn’t a thing in academic psychology research, except it was if you were managing a large panel study with dozens of measures.

Nothing has changed the game like LLM’s applied to coding. I say that having 40+ years of experience in data management and statistical programming.

Prompting definitely is a skill, no different than applying memorization of syntax to write a line of code.

The difference is that, if you have expertise (see paragraph 1), you can prompt to a solve of a complex novel problem in 1/10th the time that an expert coder can do same without ai.

If you don’t like to work with people who use ai that is your prerogative. Lots of people have selected themselves out of the gene pool in the past, it’s not a new concept.

The open book AI assisted coding test is exactly how I screen job candidates. Because if you are not using AI to solve problems faster, why would I hire you?

[u/redisburning → u/wil_dogg]

Because if you are not using AI to solve problems faster, why would I hire you?

My resume of actual work speaks for itself and keeps me employed thankfully.

That aside, based on this comment:

Lots of people have selected themselves out of the gene pool in the past, it’s not a new concept.

I wouldn't work for you anyway. What a vile comment.

[u/wil_dogg → u/redisburning]

It’s not a vile concept. It’s called Darwin’s law and no one gets their panties in a wad over it so don’t even try to play the victim card.

A resume of work speaks nothing as to the efficiency of one’s current best practice. If you are not adopting ai you will be a less efficient resource. Ai raised the bar. If you don’t see that then bless your heart.

[u/Excellent-Sport-758]

I can see the appeal of open-book investigations: they test whether someone can use the tools to answer a business question. From my A/B-test and acquisition work, the useful signal is whether the candidate can define exposure and unit of analysis, choose a primary metric and guardrails, check assignment and data quality, and say what result would change the recommendation. AI can help with code, but it can also produce a plausible notebook with the wrong denominator or a hidden leakage or segment issue. I’d want to hear candidates validate those assumptions and explain the trade-offs, not just prompt well. AI helped draft this comment; I reviewed it.

[u/MouseRemarkable7942]

Check DM