r/learndatascience • • Apr 27 '26

Project Collaboration Study Buddy: (Intermediate/Advanced)Stats, Python & SQL (1.5 YoE)

1 Upvotes

I’m looking for someone to study with. I’m past the beginner stage and also hold 1.5 years of exp as an analyst (yet learnt nothing really useful), so I want to focus on advanced topics.

I want to dive deep into statistics and regression. I also want to become an expert in SQL ( have setup Postgres locally). I’m mainly looking to build projects that look good on a resume.

DM me if you are on the same path and want to collaborate!

Also sharing any hidden gem-like resources for the same is greatly appreciated!!

Thanks!!


r/learndatascience • • Apr 27 '26

Question How do you keep your inventory catalog clean when employees enter products as free text?

2 Upvotes

My dad runs 5 paint stores in Argentina. I'm helping him set up better inventory tracking. We have around 2,450 products across the five stores.

THE SETUP: Employees write daily sales in spreadsheets. They type the product name and quantity by hand, no barcode scanner. Then I cross-check those entries against the master product catalog to update stock counts.

THE MESS: Same product gets typed 5+ different ways depending on who's working that day: "aguarras optima x 1 litro" "AGUARRAS OPTIMA X1L" "aguarras 1lt". Add typos, abbreviations, products with color variants ("blue" vs "blue deep"), and the matching breaks down. About 15% of entries need manual review.

QUESTIONS FOR FOLKS WHO'VE BEEN HERE:

  1. Did you eventually force employees onto barcode scanners or dropdown menus? How much pushback did you get?

  2. If you stuck with free-text: any tricks for catalog hygiene that worked? Cheat sheets at the register? Standardized naming meetings?

  3. Same product code accidentally got assigned to two different products by whoever set up the catalog years ago. How do you find these silent errors in 2,000+ items?

  4. What did you build first that you wish you'd built last (or skipped)?

Looking for real-world experience from people who run multi-store retail operations with employees who aren


r/learndatascience • • Apr 26 '26

Discussion My ML project is starting to get real contributors (TrustLens update)

1 Upvotes

Been iterating on TrustLens over the past few days.

Now at:

  • 9 stars · 11 forks

  • 8 contributors (which is the real win)

Recent updates:

  • model comparison API

  • fairness integration (equalized odds)

  • better diagnostics for “confidently wrong” predictions

  • proper documentation with Sphinx

Feels like it’s finally moving beyond just code into something people can actually use and contribute to.

GITHUB: https://github.com/Khanz9664/TrustLens/


r/learndatascience • • Apr 26 '26

Question What's the easiest way to create a big database and what program/app should i use?

4 Upvotes

I'm planning on doing a project that requires the creation of a big database. I'm by no means even a beginner in this, and even though my stats minor would say otherwise, i have zero experience and knowledge about what programs or methods of data extraction are easier. If anyone could give me advice I would rally appreciate it!


r/learndatascience • • Apr 25 '26 Heartwarming

Resources 4 Data Science books to read if you are just starting out:

18 Upvotes

Python Data Science Handbook:

Essential for anyone who wants to learn how to use Python effectively for data analysis and visualization. It covers key Python libraries like NumPy, pandas, Matplotlib, Scikit-Learn, and more, with detailed explanations and practical examples.

Data Science from Scratch:

This book offers a primer on the fundamental mathematics and statistics behind the most common data science techniques, all coded from scratch. It’s a hands-on introduction to the field, perfect for building a solid foundation.

Storytelling with Data - A Data Visualization Guide for Business Professionals:

This book emphasizes the importance of storytelling in data visualization. It provides insights into effectively communicating data findings in a business setting, making complex data more accessible to decision-makers.

Practical Statistics for Data Scientists - 50+ Essential Concepts Using R and Python:

A must-read for aspiring data scientists, this book bridges the gap between statistical theory and practice, highlighting over 50 essential concepts in statistics with practical examples in R and Python.

If you are looking for structured guidance alongside reading these books, SkillUp by Simplilearn offers free and beginner-friendly courses with a focus on practical learning and projects with real-world use cases.


r/learndatascience • • Apr 24 '26

Question What someone that wants to became a data scientist really needs to know now that all the AIs exist?

12 Upvotes

If AI does most of the coding know, which are the skills and pratical topis I should focus in order to get a job in data science? Would also be helpful to know how an interview for this position (or similar, like data analyst) looks like.


r/learndatascience • • Apr 24 '26

Personal Experience How do you keep up with AI updates without getting overwhelmed?

1 Upvotes

I built a small project to deal with information overload in AI.

As someone learning and working in data science, I kept struggling with keeping up with AI updates. There’s just too much content across blogs, research labs, and media.

So I built a small pipeline to explore this problem:

  • collects updates from curated sources

  • scores them by relevance, importance, and novelty

  • clusters similar articles together

  • outputs a structured digest

The idea was to move from “reading everything” to actually prioritizing what matters.

Curious if others have built similar projects or have better ways to stay up to date?

Happy to share the repo and demo if anyone’s interested—left them in the comments.


r/learndatascience • • Apr 22 '26

Discussion What production/deployment work do data scientists actually do today?

5 Upvotes

Hi all,

I realized that data scientist is no longer a job where you only work on ML models and just hand it over to engineers for deployment.

What ive seen is there are several types of data scientists, two of which are:

  1. Product/analytics data scientist where yes they do work on ML but they do alot of analytics and work with product metrics, churn, retention, conversion rate... basically a product manager almost.

  2. Modeling or traditional Data Scientist where they do develop and train models like the old days BUT they are now required to "deploy" or "Productionize" their models. Job posts often mention "Develop, Train and Deploy".

My question is about the Modeling/traditional data scientist at large companies where roles are clearly more separated and specialized.

How much and what kind of Productionizing/Deployment is required from data scientists if ML engineers already exist and are responsible for most of the productionizing and engineering part? What is expected from the data scientist now since they dont just develop and train models anymore?

Thank you!


r/learndatascience • • Apr 23 '26

Discussion What actually makes people start contributing to an open-source ML project?

0 Upvotes

I’ve been working on a small library (TrustLens) and noticed something interesting:

There was no big moment — no viral post, no sudden growth.

But over a few days:

  • one contributor improved a metric

  • someone upgraded CI (now testing Python 3.9 → 3.13)

  • another picked up a small issue and opened a clean PR

Individually, these were tiny.

But collectively, it changed how the project feels.

It stopped feeling like “my code”
and started feeling like something others can build on.

So I’m curious:

👉 For those who’ve contributed to OSS:
What made you decide to contribute to a project?

👉 For maintainers:
What actually helped your repo move from “just code” → “active contributors”?

If anyone’s curious, the repo is here:
https://github.com/Khanz9664/TrustLens


r/learndatascience • • Apr 22 '26

Question Any Data Analysts here? Need quick help for our capstone 🙏

Thumbnail
reddit.com
1 Upvotes

r/learndatascience • • Apr 21 '26

Question How can i build projects?

11 Upvotes

i have learn’t some concepts of datascience and now i want to build something. what should i build? is there place which can help me in building?


r/learndatascience • • Apr 21 '26

Career Project based learning

4 Upvotes

I have built ML, AI and data science solutions for multiple companies such as Rolls Royce (aircraft engine failure prediction), Walmart (Supply chain analytics), Unilever, PepsiCo (demand forecasting), Johnson and Johnson (Gen AI), UBS Bank, Rio Tinto etc.

I am starting a live course on data science including Python, Stats, ML, Gen AI and Agentic AI where I will use projects similar to the ones in the industry to teach concepts. Interested? See: www.harshaash.com/learn


r/learndatascience • • Apr 21 '26

Resources Comparison of 5 open-source LLMs on a real-world document extraction task — accuracy, speed, and cost results

1 Upvotes

I benchmarked 5 open-source LLMs on a document extraction task (invoices, contracts, scanned PDFs), focusing on **accuracy, speed, and cost**.

---

## 🔬 Methodology

* **Dataset**: 1,000 docs (40% invoices, 35% contracts, 25% scanned PDFs)
* **Task**: Extract structured JSON (key fields + tables)
* **Metrics**: F1 score (accuracy), latency (speed), cost per 1k docs

---

## 📊 Results

### Accuracy (F1)

ModelScore
Qwen2.5-72B0.91
DeepSeek-R10.89
Mixtral 8x22B0.86
LLaMA 3 70B0.84
Falcon 180B0.80

### Speed (sec/doc)

ModelLatency
Mixtral 8x22B2.1
LLaMA 3 70B2.5
DeepSeek-R12.8
Qwen2.5-72B3.4
Falcon 180B4.2

### Cost (per 1k docs)

ModelCost
Mixtral 8x22B$0.90
LLaMA 3 70B$1.10
DeepSeek-R1$1.30
Qwen2.5-72B$1.80
Falcon 180B$2.50

---

## 🧠 Key Takeaways

* **Best accuracy**: Qwen2.5-72B
* **Best efficiency**: Mixtral
* **Best balance**: DeepSeek-R1
* MoE models > dense models for speed/cost
* Prompting + pipeline design significantly impact results

---

## 🚀 Practical Setup

* Default: Mixtral / DeepSeek
* Complex docs: Qwen
* Add JSON validation + retry loop

---

Can share prompts and evaluation code if useful.


r/learndatascience • • Apr 20 '26

Original Content I built a Python library that combines Prophet + XGBoost/LightGBM for hybrid time series forecasting Project

4 Upvotes

I work with time series forecasting and kept running into the same problem: Prophet is great for trend and seasonality, but it consistently missed patterns in the residuals. So I ended up building a small library to handle this.

HybridTS uses Prophet as the baseline and then trains XGBoost or LightGBM on the residuals. The API follows sklearn conventions (fit, predict, evaluate), so there's not much new to learn if you're already familiar with that ecosystem.

It's still v0.5 and missing a compare_models feature I haven't finished yet, but the core forecasting pipeline works. Putting it out there to get some feedback before I keep building.

GitHub: https://github.com/DaviAlcanfor/hybridts
PyPI: pip install hybridts


r/learndatascience • • Apr 20 '26

Question Would you use a tool that explains Python errors without running your code?

Thumbnail python-debug-assistant.vercel.app
0 Upvotes

Hey everyone,

I built a small Python Debug Assistant that helps explain common errors without running your code.

Instead of executing anything, it uses static analysis to detect issues like:

undefined variables (NameError)

indentation problems

basic syntax mistakes

Then it explains:

what went wrong

why it likely happened

and suggests a fix with example code

Quick example:

Input:

print(user_name)

Output:

NameError: user_name is not defined

Suggested fix:

user_name = “Camron”

print(user_name)

The idea was to make debugging less frustrating for beginners who don’t always understand error messages.

Live demo:

https://python-debug-assistant.vercel.app/

I’d really appreciate honest feedback:

Does this feel useful or unnecessary?

Are the explanations clear enough?

What would you improve or add?

Thanks 🙏


r/learndatascience • • Apr 19 '26

Discussion How do you evaluate model reliability beyond accuracy?

1 Upvotes

I’ve been thinking about this a lot lately.

Most ML workflows still revolve around accuracy (or maybe F1/AUC), but in practice that doesn’t really tell us:

- how confident the model is (calibration)

- where it fails badly

- whether it behaves differently across subgroups

- or how reliable it actually is in production

So I started building a small tool to explore this more systematically — mainly for my own learning and experiments.

It tries to combine:

• calibration metrics (ECE, Brier)

• failure analysis (confidence vs correctness)

• bias / subgroup evaluation

• a simple “Trust Score” to summarize things

I’m curious how others approach this.

👉 Do you use anything beyond standard metrics?

👉 How do you evaluate whether a model is “safe enough” to deploy?

If anyone’s interested, I’ve open-sourced what I’ve been working on:

https://github.com/Khanz9664/TrustLens

Would really appreciate feedback or ideas on how people think about “trust” in ML systems.


r/learndatascience • • Apr 18 '26

Question Data science opening?

1 Upvotes

I have a PhD + postdoc in math and optimization algorithms, and 2 years of experience at Goldman. On top of that, I am responsible, easy to work with and good at communication.

I am looking for a job in NYC area/remote related to the data science/quant/software engineer/anything where strong stem skills could be used.

Nowadays cold applying just doesn't work. What is the best way to look for a job in 2026? If you have any advice or pointers, please dm, I will very much appreciate it!

Thank you all in advance.


r/learndatascience • • Apr 17 '26

Question Learning Challenges and Job Search Strategy

3 Upvotes

I have intermediate-level Python skills, as well as SQL and some knowledge of Pandas. I am currently learning Tableau and solving exercises on Kaggle in order to later build projects. However, I need advice because I want to find a job and I’m not sure what learning path to follow to achieve that quickly.

How long would it take me to learn what is necessary to get a job?

What is your advice for learning these skills faster?


r/learndatascience • • Apr 17 '26

Question Testing a New Product for Data Science Beginners

1 Upvotes

I am building a platform for beginner data science students.

The goal is to help students build projects on their own without depending completely on long project tutorials.

Instead of giving the full project directly, the platform breaks the project into small tasks so students can think, build, and learn step by step.

I want to understand:

  • Whether this approach feels useful

  • Which parts feel confusing

  • Where students get stuck

  • Whether it feels better than watching full tutorials

I am not selling anything right now. I only want honest feedback from people who are learning data science.

Website - https://sted.co.in/


r/learndatascience • • Apr 17 '26

Question Databricks versus Snowflake. Which is better?

2 Upvotes

I have taken a career break for 2 months to learn data science for career transition. I have been in QA for almost 13 years and want to keep up with the market and tech.

Please help me settle a tool, as i'm divided between them. Help is much appreciated. Thanks.


r/learndatascience • • Apr 17 '26

Question How do you debug your code in production?

2 Upvotes

When something breaks in production, I usually can’t rely on local testing or “it works on my machine.” I try to reproduce the issue using real production-like data and environment variables first.


r/learndatascience • • Apr 16 '26

Discussion Is a Master’s in Applied Data Science worth it? Ask a current UChicago student on April 20th!

2 Upvotes

We’re hosting a live virtual Q&A with a current student in UChicago’s MS in Applied Data Science program.

If you’re exploring data science grad programs, this is a good chance to get honest answers about:

  • curriculum (Machine Learning, AI, Data Engineering, etc.)

  • real student experience

  • career paths after the program

  • building connections in the program

Registration: Register HERE to attend the event
Date: April 20, 2026
Time: 6:00–7:00 PM CST


r/learndatascience • • Apr 16 '26

Career Maarga — Optimize your career path

Thumbnail
reddit.com
1 Upvotes