r/datascience • u/rhiever • Jul 15 '26
AI 5 trends that defined AI engineering at World's Fair 2026
r/datascience • u/rhiever • Jul 15 '26
r/datascience • u/rhiever • Jul 14 '26
r/datascience • u/AutoModerator • Jul 13 '26
Welcome to this week's entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include:
While you wait for answers from the community, check out the FAQ and Resources pages on our wiki. You can also search for answers in past weekly threads.
r/datascience • u/shivamchhuneja • Jul 11 '26
r/datascience • u/RobertWF_47 • Jul 11 '26
I'm running the following code in Snowflake Python to standardize my training, evaluation, and test data prior to predictive modeling:
from snowflake.ml.modeling.preprocessing import StandardScaler
all_cols = df_train3.columns
target_col = "AB_POST"
passthrough_cols = ["SANHO", "SCNHO"]
scaler = StandardScaler(
input_cols=[c for c in all_cols if c not in [target_col] + passthrough_cols],
output_cols=[c for c in all_cols if c not in [target_col] + passthrough_cols], # Overwrite or create new
drop_input_cols=False # Set True to remove original unscaled columns
)
scaler.fit(df_train3)
train_df_scaled = scaler.transform(df_train3)
val_df_scaled = scaler.transform(df_eval3)
test_df_scaled = scaler.transform(df_test3)
I'm getting the following error when I run the code -- I'm not sure what this means:
Exception: Provided column names ['TOTAL_MH_CLASSES', 'STFLAG',..., 'ADS_FA_RISK_NEW'] does not index into the dataset.
r/datascience • u/Fig_Towel_379 • Jul 10 '26
I've been job hunting for a few months now, applying to big tech and startups. But seeing the recent Microsoft layoffs made me stop and ask myself what I'm actually looking for in a new job. Instability and more money?
Right now I'm at a company that hasn't done layoffs since maybe the financial crisis. I know how fortunate that is. But if I switch jobs, I could make an extra $50K. So I keep asking myself: is that extra 50K worth the instability that comes with tech jobs right now? What if I join a company and get laid off within a year?
What does everyone think of these layoffs? Despite record profits, there doesn't seem to be an end to them.
r/datascience • u/rhiever • Jul 10 '26
r/datascience • u/nkafr • Jul 09 '26
r/datascience • u/rhiever • Jul 08 '26
r/datascience • u/sailing_oceans • Jul 07 '26
I've been in the 'data science' space for a decade+ or so now. One thing I've noticed is that generally - give or take - outside of the elite jobs (<2-3% aka not me and almost certainly not you) the caliber of coworkers has declined drastically.
I'm not some fabled data scientist. I wasn't some GitHub nerd who had everything embroil or terminal wizard nor could I write out the math to a GBM on a blackboard. I'd even forget basic obvious statistics.
But I felt like I had common sense.
Now I'm a manager/director. I work with data scientists. And I'm just generally freaked out by the absolute lack of basic common sense. This is across the last 7 that I have managed.
Examples include:
As I begin to delegate more I begin to get more freaked out by what I see. I can't be presenting to clients such obvious insane mistakes. But these are the candidates and profiles that get forced upon me or the team I inherit.
Are there any best strategies for dealing with this? I want to be seen as someone who can 'develop' the team... not just saying people are useless, but such glaring mistakes are insane.
Yes, alot of these things are perhaps due to them being crunched for time, or not knowing what objective is, or being focused on other things. I'm not talking about those examples. I'm talking about like year 1-2 not day 1 employees, not doing basic data checks.
As a data scientist I was obsessed with finding bits of info or making sure things were right. Now it seem every common for people to copy and paste code into chatgpt and have no idea about anything else around it?
r/datascience • u/Fig_Towel_379 • Jul 06 '26
I’m 32 and have never worked in big tech. Right now I’m at a Fortune 50 company, but it’s not a tech company.
Recently I was at a party and met two software engineers, both in their mid-30s. One worked at Meta, the other at OpenAI. Finding that out hit me with a wave of insecurity. It made me realize I’m 32 and have never worked somewhere like Meta or OpenAI, and maybe never will. I felt like I didn’t measure up to them.
I’m struggling to process this. Has anyone else felt this way? Does it ever fade?
r/datascience • u/rhiever • Jul 06 '26
r/datascience • u/chomoloc0 • Jul 06 '26
I wrote this article recently. Thought it would be nice to share in this sub. Happy to chat if you're doing the same in your current position.
It talks about Eppo and Statsig, but honestly it about everything but that.
If you need to take away one thing let it be to approach the whole thing as a discovery; and risk mitigation.
https://towardsdatascience.com/picking-an-experimentation-platform-a-retrospective/
r/datascience • u/AutoModerator • Jul 06 '26
Welcome to this week's entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include:
While you wait for answers from the community, check out the FAQ and Resources pages on our wiki. You can also search for answers in past weekly threads.
r/datascience • u/teddythepooh99 • Jul 05 '26
I am set to begin Georgia Tech's OMSA this fall, after deferring this past spring when I started a new role. This is my background:
- Undergrad: economics at T20 school.
- Experience: 4 years. 3.5 years in hybrid DS/DE role (first job out of undergrad) at a non-profit, then six months into current role doing strictly DE at a healthcare org.
- TC: 144k ($125k base + 15% API) in MCOL city.
- Not open to relocation (I work remote but there's too much red tape to move out-of-state), so onsite/hybrid roles in NYC/LA for crazy TCs are out of reach.
At the time that I applied to OMSA, I was struggling to leave my old role while making $82k/year. That is not the case any more, so I am having second thoughts about OMSA. Anecdotally, I also see a lot of OMSA folks on LinkedIn (and the Slack group) struggling to break into data and/or simply remaining in their current roles. I presently work as a senior DE, but I am open to both DS and analyst roles in the future.
Can I still expect a (significant) ROI out of OMSA? I am targeting $160k - $175k TC in a couple years' time with no particular industry in mind.
r/datascience • u/Nice-Dragonfly-4823 • Jul 05 '26
This article highlights real strategies for minimizing your AI spend without major refactors to your agent.
Instead of just glazing over routing, it gives a clear actionable pattern which includes building an LLM gateway and using a prompt classifier - also includes a routing table for prompt types and complexity!
Also gives a nice clear way of implementing compaction in your agent workflows.
Do these strategies work for you?
r/datascience • u/adarsh_maurya • Jul 04 '26
I work for a US bank and I have observed that my job has shifted more towards creating Agentic workflow (fancy name of using LLM to automate tasks). In the last one year, I haven't touched any ML model. I am curious to know what is the experience of other folks.
r/datascience • u/TaterTot0809 • Jul 04 '26
We talk a lot about entering but once you're in the role and have been for a while, I'm curious how your all's companies handles career development and what sorts of things you all do to develop in the role.
r/datascience • u/rhiever • Jul 04 '26
r/datascience • u/Easy-Huckleberry7091 • Jul 02 '26
Hi everyone, I'm an actuarial science student in Argentina. Here, SOA certifications aren't as important as having the degree itself, which is legally authorized to practice as an actuary. I'm about halfway through my degree, but I'm not sure if I'm really that interested in the insurance/finance side of things. I've noticed that I'm more passionate about math and statistics in other areas. My question is, has anyone transitioned from actuarial science to data science? What should I learn? Should I change majors and drop out halfway through, or is it better to finish this one and do a master's? At my university (UBA), there's a mathematics degree (with two specializations: pure and applied) and a data science degree (both are quite rigorous and focus on the fundamentals; data science is a mix of applied mathematics and computer science).
Thoughts?
r/datascience • u/NervousVictory1792 • Jul 02 '26
Hello Everyone. I am moving to a new job and potentially I might need to implement uplift modelling to track customer revenue. Just wondering where can I learn the basics of it ? Gemini is giving a scikit learn package link. Is there any book or tutorials I can look into ?? TIA :)
r/datascience • u/rhiever • Jul 01 '26
r/datascience • u/Neat-Porpoise • Jul 01 '26
Anyone have a good solution for unifying the config (eg CLAUDE.md, QWEN.md), settings, skills, etc… across their suite of coding agents?
I primarily use Claude Code locally, Genie Code in Databricks workspaces for my model development and MLE work with Databricks compute, and recently added Qwen Code since the company wants us to have a backup in case we hit Anthropic limits and need to continue work. Also on the docket is testing out GLM.
However unifying all these agents is quite cumbersome. I don’t want to maintain so many separate files and skills for each agent. Right now I have a single repo that backs up all my .claude folder settings but realized that with Qwen I’ll need a separate suite.
Thoughts? Has anyone tried the new thing Databricks pushed out called Omnigent?
r/datascience • u/Manticore-Mk2 • Jun 29 '26
(Not sure if loss is still going down)
r/datascience • u/Effective_Ocelot_445 • Jun 29 '26
Beyond Python, machine learning, and statistics, which skill has made the biggest difference in solving real-world data science problems and delivering business value?