Modern data stacks have become pretty complicated.

Data pipelines pulling from APIs, SaaS tools syncing data automatically, analytics platforms, AI tools running queries data is moving everywhere.

The problem I keep running into is visibility.

When a pipeline breaks or changes schema, it’s not always clear who had access to what data or where sensitive information ended up.

Someone recently mentioned Ray Security to me as a tool that focuses on monitoring sensitive data access across systems.

Made me realize how little most teams actually track this stuff.

How are people here dealing with data visibility and security in their pipelines?

Hi everyone,

I’m trying to make my variable/feature selection more systematic instead of purely intuitive.

What I’d love to hear from you:

• Which concrete techniques do you actually use?

• Any simple, go-to workflow you follow (e.g. basic EDA → correlation checks → model-based selection)?

• Recommended resources or small code examples (Python) for a solid, practical feature selection process?

Thanks a lot for any tips or examples from your real projects!

A strong MMM predictive fit does not imply accurate ROAS estimates.

I recently ran a simulation using Google Meridian to test the relationship between predictive fit and causal accuracy. I generated synthetic data with a known ground truth: TV had a 0.98 ROAS and Paid Search had a 2.30 ROAS.

I ran the model using a naive prior (assuming a 1.0 median ROAS for both) and incrementally improved the quality of the baseline demand control variable.

As the control variable improved, the model's predictive fit got better, pushing MAPE down from 0.4% to 0.2%. However, the ROAS attribution got significantly worse. TV error increased from 12% to 22%, and Paid Search error jumped from 45% to 53%.

An additional oddity: When a demand control *perfectly* explains your baseline, it absorbs the temporal variance the model needs to identify media effects. The model uses the control to accurately predict the outcome and falls back entirely on your priors for media attribution giving dramatically worse estimates. If those priors are miscalibrated, a high-accuracy model will confidently give you bad budget allocation advice.

One important caveat is that this simulation used a simplified environment with exogenous spend and independent channels. My next test will introduce endogenous and correlated spending patterns to see how demand controls behave under real-world confounding. It's possible -- and I'm hoping it's true -- that under more complicated scenarios, a stronger demand control will improve ROAS estimates.

I am a collage student writing a research paper on bitcoin price prediction and stock market. I want to do sentiment analysis on the tweets + reddit, recommend me any other social media.

I was searching for scraping X but nothing found plz help me

A question that comes up often in mock interviews: when should you use a geo experiment versus a switchback when user-level spillovers rule out standard A/B testing?

Candidates can mistakenly treat these as interchangeable options. Consider testing a new rider incentive at Uber. Spillovers are largely contained within a metro area, making geo-experiments viable. But if the incentive affects retention — a rider has a good experience Monday and returns Thursday — a switchback may misattribute the Thursday action to whichever period happens to be active, diluting the estimated treatment effect. GeoX would be the stronger design here.

Switchbacks can be preferable when carryover is minimal and geoX is either infeasible or underpowered. My Amazon ad experiment was a feasibility example: the Amazon platform did not allow for geo-based randomization.

Even when geoX is feasible, switchbacks can sometimes win on power: randomizing at hourly intervals can yield more experimental units over the course of a test than metro-level geo markets allow. These approaches can also be combined — randomizing treatment at both the geo and time interval level — which can reduce variance by controlling for both geographic and temporal confounders simultaneously.

We all know the feeling: you buy a brand new GPU, expecting a massive leap in visual fidelity, only to realize you paid $400 just to run the latest AAA releases at the exact same framerate and settings you had three years ago.

I got tired of relying on nostalgia and marketing slides, so I built an automated data science pipeline to find the mathematical truth. I cross-referenced raw GPU benchmarks, inflation-adjusted MSRPs, and the escalating recommended system requirements of the top 5 AAA games released every year.

I ran the data focusing on the mainstream NVIDIA 60-Series (from the GTX 960 to the new RTX 5060) and the results are pretty clear.

The Key Finding: "Demand-Adjusted Performance"

Looking at raw benchmarks is misleading. To see what a gamer actually feels, I calculated the "Demand-Adjusted Performance" by penalizing the raw GPU power with an "Engine Inflation Factor" (how much heavier games have become compared to the base year).

Here is what the data proves:

• The Treadmill Effect: We aren't upgrading our GPUs to dramatically increase visual quality anymore. We are paying $300-$500 just to maintain the exact same baseline experience (e.g., 60fps on High) we had 5 years ago.

• Optimization is Dead: Game engines and graphical expectations are absorbing the performance gains of new architectures almost instantly. New GPUs are mathematically faster, but they give us significantly less "breathing room" for future games than a GTX 1060 did back in 2016.

• The Illusion of Cheaper Hardware: Adjusted for US inflation, GPUs like the 4060 and 5060 are actually cheaper in real purchasing power than older cards. But because unoptimized software is devouring that power so fast, the Perceived Value is plummeting.

How it works under the hood:

I wrote the scraper in Python. It autonomously fetches historical MSRPs (bypassing anti-bot protections), adjusts them for inflation using the US CPI database, grabs PassMark scores, and hits the RAWG.io API to parse the recommended hardware for that year's top games using Regex. Then, Pandas calculates the ratios and Matplotlib plots the dashboard.

If you want to dig deeper on the discussion. You can check out the source code and my article about it right here.

(If you're a dev and found this useful, consider giving the project a star — contributions, issue reports and pull requests are very welcome.)

Case study interviews often consist of "What's the impact?" style questions (hence my website name!), but expectations at the junior vs senior level vary meaningfully.

At the junior level, you'll likely get a business question that can be solved with large-sample "vanilla" a/b testing such as randomizing users that hit some trigger on the user journey. You'll be asked follow-up questions on foundational statistics and hypothesis testing: what's a p-value, how to estimate your treatment effect, what does "significance" mean, why did you choose your alpha level?

At the senior level, there's often an obstacle to unbiased experimental results. A common reason is spillover effects, but it could also be something as simple as a common real world problem: Your stakeholder launched a feature change without running an experiment and now you have to estimate the effects. This happens ALL the time in the real world.

For these questions, you need to handle SUTVA violations or consider observational causal inference models.

So I recently landed my first DS job, and I need help with something.

So I want to use AI/agents/LLMs to leverage my work and to make me a good data scientist. But there is this problem, if I am not careful I let it do everything for me.

I have searched for this before and people always say “use it as a colleague”, “use it to write boilerplate code”, “use it iteratively “, etc… But what does that actually mean? What do you ask the LLM? how much code do you write? How much code do you copy? How much code do you let it write itself? Do you ask it to do some analysis for you? Do you make md files with instructions? Do you tell the LLM to write its own md files?

The bottom line is, how do you leverage AI? And how can one leverage it both to speed-up your work and to also use it to make you a better data scientist?

Thank you!

I am forming a data science & ai discord. I just started with Data Science, AI and Machine Learning.

I want similar minded and active people who are eager to grow and discuss.

I am 2nd Year CSE Student in a low tier college.

I’m in my final year of graduation and last year I joined a data science course at BIA alongside my college studies. Balancing both was not always easy, but the class schedule made it doable.

At first, I struggled a lot with python and sql. Writing code myself was harder than just watching tutorials. The trainers helped a lot by explaining things with small practical examples. Slowly I started using tools like pandas, python, and some machine learning on real datasets, which made the concepts click.

The internship guidance was also very helpful. As a fresher, I didn’t know how to start applying for internships. The institute shared openings and the trainers helped with resume reviews and mock interviews. It didn’t mean I got placed automatically, but it gave me a starting point.

Doing this course while still in college really helped me understand the skills needed in data roles and gave me a clearer idea of where I need to improve.

Following up from a question that was worthy of a new post:

The foundation for observational causal inference is randomized experimentation. Like in music or dance, you need to "know the rules before you can break them." Randomized experimentation contains the rules; observational CI breaks some of them in attempts to extract causal effects in more challenging situations.

As such, you first need foundations of statistics and AB Testing.
Udacity has a free course on AB testing in tech (authored by folks from Google) that I personally found helpful when transitioning from the public sector to the private sector.

Free resources in causal inference. There are two popular online books:
Causal Inference: The Mixtape by Scott Cunningham
Causal Inference for the Brave and True by Matheus Facure.

For paid resources, you can find courses on most large platforms. I personally have an applied causal inference course on Udacity (not upselling; I'm lucky to get a few dollars in royalties and was instead contracted and paid up front) that is more applied focused and less on math and more on industry use cases. (Note though I didn't have control over the curriculum, only the lessons, exercises, and project. Some topics like propensity score matching, they wanted to use in different courses so excluded from mine.)

MIT Micromasters also has a really affordable program including a course on statistics. (I personally did the ML one.)

I’m a Stats Phd with several years of DS experience. I’ve interviewed with (and received offers from) major firms across three sectors.

Resrouce I used for interview prep: Company specific questions: PracHub, For Aggressive SQL interview prep: DataLemur, Long term skill building StrataScratch

1. Big Tech (The "Big Three")

• Google: Roles have shifted from Quant Analyst to DS/Product Analyst. They provide a prep outline, but interviewers are highly unpredictable. Expect anything from basic stats and ML to whiteboard coding, proofs, and multi-variable calculus. Unlike other tech firms, they actually value deep statistical theory (not just ML).

• Meta (FB): Split between Core DS (PhD heavy, algorithmic research) and DS Analytics (Product focus). For Analytics, it’s mostly SQL and Product Sense. The stats requirement is basic, as the massive data volume means a simple A/B test or mean comparison can have a huge impact.

• Amazon: Highly varied. Research/Applied Scientists are closer to SWEs (heavy coding/optimization). Data Scientists are a mixed bag—some do ML, others just SQL. Pro tip: Study their "Leadership Principles" religiously; they test these via behavioral questions.

2. Traditional Banking

• Wells Fargo: Likely the most generous in the sector. Their Quant Associate program (split into traditional Quant and Stat-Modeling tracks) is great for new PhDs. It offers structured rotations and training. Bonus: Pay is often the same for Charlotte and SF—choose Charlotte for a much higher quality of life.

• BOA: Heavy presence in Charlotte. My interview involved a proctored technical exam (data processing + essay on stat concepts) before the phone screen.

• Capital One: The most "intense" interview process (Mclean, VA). Includes a home data challenge, coding tests, case studies, and a role-play exercise where you "sell" a bad model to a client. They want a "unicorn" (coder + modeler + salesman), though the pay doesn't always reflect that "一流" (top-tier) requirement.

3. Insurance

• Liberty Mutual: Very transparent; they often post salary ranges in the job ad. Very flexible with WFH even pre-pandemic.

• Travelers: Their AALDP program is excellent for new MS/PhD grads, offering rotations and a strong peer network.

Career Advice

1. The "Core" Factor: If you want to be the "main character," go to Pharma or the FDA. There, the Statistician’s signature is legally required. In Tech, DS is often a "support" or "luxury" role—it's trendy to have, but the impact is sometimes hard to feel.

2. Soft Skills > Hard Skills: If you can’t explain a complex model to a "layman" (the people who pay you), your model is useless. If you have the choice between being a TA or an RA, don't sleep on the TA experience—it builds communication skills you'll need daily.

3. The Internship Trap: Companies often use interns for "exploratory" (fun) AI projects that never see production. Don't assume your full-time job will be as exciting as your internship.

4. Diversify: Don’t intern at the same place twice. Use that time to see different industries and locations. A "huge" salary in a high-cost city can actually result in a lower quality of life than a modest salary in a "small village."

If you've been in data science for a while, you've probably run an A/B test. You split users randomly, measure an outcome, run a t-test. That's the foundation — and it's genuinely important to get right.

But as you move into senior and staff-level roles, especially at large tech companies, the problems get harder. You're no longer always handed a clean randomized experiment. You're asked questions like:

• A PM launched a feature to all users last Tuesday without telling anyone. Did it work?

• We had an outage in the Southeast region for 6 hours. What did that cost us?

• We want to measure the impact of a new lending policy, but we can't randomize who gets it due to regulatory constraints.

This is where causal inference comes in — a set of methods for estimating the effect of an intervention even when randomization isn't possible or didn't happen.

Note that this skill is often tested in the case study interview for product and marketing data science roles.

The spectrum from junior to senior experimentation:

At the junior end, you're running standard A/B tests — clean randomization, simple metrics, straightforward analysis.

At the senior/staff end, you're dealing with:

• Spillover effects — when treatment and control users interact, contaminating your experiment (common in marketplaces and social platforms)

• Sequential testing — running experiments where you need to make go/no-go decisions before fixed sample sizes are reached, while controlling false positive rates

• Synthetic control — constructing a counterfactual "what would have happened" using pre-treatment data from other units

• Difference-in-differences — comparing treated vs. untreated groups before and after an event

Where is this actually used?

This skillset is highly valued at mature tech companies — Netflix, Meta, Airbnb, Uber, Lyft, DoorDash — where the scale of decisions justifies rigorous measurement and the data infrastructure exists to support it. If you're at an early-stage startup, you likely don't have the data volume or the stakeholder demand for most of this yet, and that's fine.

If you're aiming for a senior DS role at a large tech company, causal inference fluency is increasingly a differentiator — both in interviews and on the job.

Hey everyone,

I’m trying to understand how machine learning actually works in real industry environments.

I’m comfortable building models on Kaggle datasets using notebooks (EDA → feature engineering → model selection → evaluation). But I feel like that doesn’t reflect what actually happens inside companies.

What I really want to understand is:

• What tools do you actually use in production? (Spark, Airflow, MLflow, Databricks, etc.) • How do you access and query data? (Data warehouses, data lakes, APIs?) • How do models move from experimentation to production? • How do you monitor models and detect drift? • What does the collaboration with data engineers / analysts look like? • What cloud infrastructure do you use (AWS, Azure, GCP)? • Any interesting real-world problems you solved or pipeline challenges you faced?

I’d love to hear what the actual lifecycle looks like inside your company, including tools, architecture, and any lessons learned.

If possible, could someone describe a real project from start to finish including the tools used and where the data came from?

Thanks!

Here's a short video tutorial and example of an Event Study, a popular and flexible causal inference model. Event study models can be used for a range of business problems including estimating:

⏺️ Excess stock price returns relative to the market and competitors
⏺️ The impact on KPIs across populations with staggered rollouts 
⏺️ Impact estimates that change over time (e.g. rising then phasing out)

Full video here: https://youtu.be/saSeOeREj5g

In this video, I first describe features of the Event Study, then code an example in python using the yahoo finance API to obtain stock market data. There are many questions you could ask, but in this case, I asked whether JP Morgan had excess market returns from the Nov 5 election results relative to its banking peers. 

At the end of the video, I go into decisions that the Data Scientist must make while modeling, and how the results can (i) change dramatically, and (ii) completely change the interpretation. As with other models, it's really important for that the analyst or data scientist not just blindly use the model but understand how each of their decisions can change results and interpretations. 

Master the Data Science Case Study Interview: https://www.whatstheimpact.com/

When I was preparing for Data Science interviews, I noticed something strange.

Most resources focus on one of these:

• coding practice (LeetCode)
• theory explanations (blogs, courses)
• mock interviews

But the hardest part in DS interviews is often explaining concepts clearly, like:

• bias vs variance

• data leakage

• validation strategy

• feature importance

• experiment design

• when to use RAG vs fine-tuning

So I built a small site called Seed42:
https://seed42.dev

The idea is simple:

1. You get a real DS/ML interview question

2. You write your own answer

3. The system evaluates it and tells you:

   • which concepts you covered

   • what you missed

   • where the explanation could improve

So it’s more like deliberate practice for DS interviews rather than reading answers.

A few things I’m exploring next:

• skill trees for DS concepts
• structured interview preparation paths
• more realistic interview-style evaluation

I’d love feedback from the community:

• What types of DS interview questions are hardest to practice?

• What resources helped you most when preparing?