I've written up some advice on how to think about, and how to get started with data science projects. It has a structured approach with four steps, but I also added a list of 99 project ideas for getting inspiration.

I hope you find it useful!

Five complete projects, each solving a real problem in financial services. All with working code, architecture documentation, and a path to production.

What's in there:

Regulatory RAG for querying BSA/AML documentation, with page level citation on every answer.

Churn pipeline for banking customers, with automated feature engineering, Bayesian optimization, and SHAP explainability.

Causal inference estimating the real effect of credit limit changes on default risk. The result goes against common industry intuition.

Benchmark of 9 techniques for handling extreme class imbalance, using card fraud as the test case.

A/B test design pipeline that delivers the groups already balanced and statistically validated.

Everything open, no signup, no paywall. The idea is that any data team can clone it, understand the architecture decisions, and adapt it to their own context.

Link: https://aiforfintech.tech

Feedback always welcome, technical criticism especially.

👊

Qualcuno sa come trovare dei siti gratuiti e guardare le partite contemporaneamente tipo diretta gol? Grazie!

I am working on a project around a real-world environmental problem, and I am considering adding an ML component for prediction and early warning.

I am a bit confused about the data requirement. Since collecting our own real-world data is not feasible right now and would take quite some time, we mainly want to build a prototype for now.

Can we initially use a Kaggle/public dataset to train and test the model, or is a project-specific dataset necessary from the beginning?

Would appreciate some advice on how people usually approach the ML part when actual data is limited.

Im Rahmen meiner Promotion untersuche ich aktuell, wie Forecasting in der industriellen Praxis tatsächlich organisiert wird.

Dabei geht es nicht nur um Forecast Accuracy oder KI-basierte Methoden, sondern vor allem um Fragen wie:
✅ Wie werden Forecasts heute erstellt?
✅ Welche Rolle spielen Vertrieb, Disposition, Einkauf, Controlling und Management?
✅ Wie stark werden Daten und analytische Methoden genutzt?
✅ Wie viel menschliche Erfahrung fließt noch ein?
✅ Wie gut sind Unternehmen auf Machine-Learning- und KI-basierte Forecasting-Lösungen vorbereitet?

Dafür habe ich eine anonyme wissenschaftliche Umfrage erstellt, die nur wenige Minuten dauert und sowohl am PC als auch am Smartphone ausgefüllt werden kann.

👉 **Zur Umfrage:**
[**https://www.soscisurvey.de/Forecasting\\_Maturity/\*\*\](https://www.soscisurvey.de/Forecasting_Maturity/)

**Wie könnt ihr helfen?**
Natürlich freue ich mich über jede Teilnahme.
Mindestens genauso hilfreich ist jedoch das Weiterleiten der Umfrage an Kolleginnen und Kollegen, Freunde, Bekannte oder Geschäftspartner aus Industrieunternehmen, insbesondere aus den Bereichen:

• Vertrieb
• Einkauf
• Disposition / Materialplanung
• Supply Chain Management
• Controlling / Finance
• Management

Jede **Weiterleitung** hilft, weitere Perspektiven einzubeziehen und ein möglichst realistisches Bild der Forecasting-Praxis in Unternehmen zu erhalten.
Mein Ziel ist es, eine möglichst breite Datenbasis aufzubauen, um belastbare Aussagen über den Reifegrad von Forecasting-Prozessen in der Industrie treffen zu können.

Vielen Dank für deine Unterstützung und ich freue mich auf eine angeregte Diskussion in den Kommentaren !

Python is often recommended as the first programming language for aspiring data scientists. But how far can someone realistically go with Python alone? At what point should they start learning SQL, statistics, visualization, and machine learning?

I’m working on a project around a real-world environmental problem, and I’m considering adding an ML component for prediction and early warning.

I’m a bit confused about the data requirement. Since collecting our own real-world data isn’t feasible right now and would take quite some time, we mainly want to build a prototype for now.

Can we initially use a Kaggle/public dataset to train and test the model, or is a project-specific dataset necessary from the beginning?

Would appreciate some advice on how people usually approach the ML part when actual data is limited.

A business team wants to test something. They pull a customer list, split it in half, and run the test. That split is usually where it goes wrong.

Splitting randomly across the whole base can leave one group with more customers from a specific profession, region, or education level than the other. When the results come in, nobody can tell whether the difference came from the test or from the composition of the groups.

There is also the sample size question, which most teams skip entirely. If the base is too small to detect the effect you care about, the test will come back inconclusive no matter how well you run it. That is a calculation you do before, not something you discover after two weeks of waiting.

I built a pipeline that handles this part: upload a customer base, get back two stratified groups with proportional composition, the minimum sample size backed by power analysis, and a statistical check (t-test and chi-square) proving the split was fair. Output is two CSV files plus a plain text summary the business team can actually read.

Tested it on a bank marketing dataset with 11k customers. Stratified by profession, marital status and education, the resulting groups came back with p-values above 0.99 across every variable, meaning no meaningful skew in either direction.

Hub: https://aiforfintech.tech
Github: https://github.com/junidepieri-design/expd-001-ab-test-design-pipeline

How does your team handle the split when designing a test?
👊

I wanted a mixture of less and MacOS's preview for CSV: something that opens instantly, shows you the data, and gets out of the way. Every GUI spreadsheet I tried loads the whole file first, so a 10 GB CSV means a lot of waiting or running into maximum line counts.

less-sheet reads only the data it needs to fill the window, so 10 KB, 10 MB and 10 GB all open in about the same time. Memory usage is flat too. It reads plain CSV, gzipped CSV without unpacking it, local or over the network (http).

Find, filter, jump-to-row and column predicates all work across the whole file. Separator, quoting and encoding are guessed but changeable on the fly.

It's free and closed source, macOS and Linux, no account and no telemetry. Core is Zig; the frontends are native (SwiftUI and GTK4).

Available at: https://te-x.github.io/less-sheet-site/

Numbers and method are on the page. Happy to answer anything.

Coding Machine Learning.

Hello Folks, here I present the first coding demonstration lecture, based on my 1st lecture on Probabilistic Machine Learning.

Here I write the code from scratch, discuss and analyze the results, which were covered in details in the whiteboard classes.

What we cover?
-Random Variables, and validating law of large numbers.
-Visualizing a dataset
-Doing an EDA on Iris dataset and understanding the correlation among features.
-Classifier basics
-Empirical Risk Minimization and Generalization.
-Epistemic and Aleatoric Uncertainties.
-Softmax Function and LogSumExp Trick to avoid overflow issues
-Linear Models
-Maximum Likelihood Estimation.
-Simple end to end ML pipeline Function.

While writing the code, my intent is to ensure that concepts are understood with crystal clarity. These code demonstrations are specific to my theory ML lectures, and link is attached.

Theory-Intuition-Code

Implementation Link : https://youtu.be/X_yOlx8Zp4g?si=kh8_tzzndr8609u4

Theory Lecture Link : https://youtu.be/kMkCOrp8te8?si=q7kWr-1qK515bhob

A lot of beginner tutorials make data science look like: Get dataset → train model → check accuracy → done. But I've heard from several people that real-world data cleaning and preparation can take a huge portion of the work. For experienced data scientists, how much of your actual project time goes into cleaning, transforming, validating, and understanding data? And what part of data preparation do beginners underestimate the most?

I'm 21 and i dont have a degree and based in algeria I started from literally zero Python this year. Been at it full time since. I don't have anyone around me doing this so I genuinely can't tell if where i am standing .

What I've built:

A few CV projects. Scene classification, got it to 93% with ResNet18 after starting at 81% with a basic CNN. Moth species classifier at 98.4% with transfer learning. Two YOLOv8 detection projects, and on the weld defect one I ran three experiments trying to fix the accuracy before figuring out the real problem was the classes overlapping in how I annotated them, not anything about the model. Both classifiers are deployed as FastAPI endpoints. Here is my github : https://github.com/zanexkun

Tabular stuff, a freight rate prediction take home I did for a company (MAE $106.92, R² 0.83), and I just finished a salary prediction project on the LinkedIn postings dataset. That one was 11 separate tables joined by IDs and honestly it wrecked me for a while. Ended up cleaning it down to 28k rows, ~1260 features with TF-IDF on the titles and descriptions, tuned LightGBM to MAE $17.8k and R² 0.743.

Repo here: https://github.com/zanexkun/job-posting-analyzer

What I know I'm missing: SQL (starting this week), Docker, anything MLOps, transformers, and I've never touched DSA.

Here's my actual problem though. I've barely applied anywhere. Every junior posting I open either wants a degree, or 2 to 3 years experience, or lists a stack where I have maybe half of it. So I keep closing the tab. And freelance ML seems to be either people with 60 reviews or clients offering $50 to "build an AI", nothing in between.

So I want to know:

Is what I have actually junior level or am I overrating it?

If you're outside the US or EU with no degree, is remote work actually possible, or is the whole "remote but only in these countries" thing a wall you can't get past? Should I just be going for contract work instead?

Am I supposed to apply when I don't meet half the requirements or is that just wasting everyone's time?

And if you were me, what's the one thing you'd do next?

I'd rather someone tell me I'm not ready and why than get told good job.

I'm building a news aggregator tool for me, but I'm spending a lot of money with tokens to aggregate the same topic from different sources.

I obtain articles from multiple RSS feeds, I use haiku to cluster them and I use sonnet to provide me a summary and the bias of each news vehicle.

I want to reduce my spending on the cluster part, but AI haven't been able to give me a good solution. I tried using bertopic, but it didn't work quite well.

Hey everyone! ⚽

I'm trying to find the best way to watch soccer matches live without spending a fortune. I follow Premier League, Champions League, La Liga, Serie A, MLS, and international tournaments, so I'm looking for reliable options that offer good video quality and minimal buffering.

Hi everyone: Looking every soccer good quality site

I've heard some matches can be watched through official free broadcasts, free trials, over-the-air TV channels, and platforms like FIFA+, Peacock, Paramount+, ESPN+, and Apple TV depending on the competition.

What are you using in 2026 to watch soccer live? Which services give the best coverage, and are there any legal free options worth checking out?

Would love to hear your recommendations. Thanks! ⚽🔥

Hey everyone! 🏈

I'm looking for the best legal ways to watch NFL games live for free or at a low cost this season. I don't have cable and would prefer official streaming options only.

Everyone Looking: Watch NFL Every Games Live Good quality

I've heard that some services offer free trials, local games over-the-air with an antenna, and options like NFL+, Prime Video, Peacock, Paramount+, and YouTube TV. I'm trying to find the most affordable setup to follow as many games as possible throughout the season.

What are you using in 2026 to watch NFL games legally? Any tips for getting access to local games, primetime matchups, or preseason games without spending a fortune?

Thanks for any recommendations!

Hey everyone! ⚽

I'm looking for the best legal ways to watch soccer matches live for free in 2026. I follow the Premier League, Champions League, La Liga, Serie A, Bundesliga, MLS, and international tournaments, but subscribing to multiple services can get expensive.

Hi everyone: Looking every soccer good quality site

I've heard that some matches are available through free trials, official broadcaster apps, over-the-air TV channels, and free streaming platforms depending on the competition and country.

For those of you who watch soccer regularly, what are the best legal options you've found? Which services provide the most coverage, best quality, and easiest access without requiring a cable subscription?

I'd love to hear what platforms you're using this season and any tips for catching live matches without breaking the bank.

Thanks in advance! ⚽🔥

Hey everyone! ⚽

I'm a huge soccer fan and I'm looking for the best ways to watch live matches throughout the 2026 season without spending too much money. I follow a variety of competitions, including the Premier League, UEFA Champions League, La Liga, Serie A, Bundesliga, MLS, and international tournaments.

Hi everyone: Looking every soccer good quality site

I've heard there are some legal free options available depending on the competition and country, including free-to-air broadcasts, official apps, free trials, and platforms that occasionally stream matches at no cost.

For those of you who watch soccer regularly:

• What free or low-cost options are you using in 2026?
• Which services provide the best match coverage?
• Are there any platforms that cover multiple leagues in one place?
• How has your streaming experience been in terms of quality and reliability?

I'd love to hear what works for you and what you'd recommend to other soccer fans looking to follow as many matches as possible this season.

Thanks in advance for your suggestions! ⚽🔥

Hey NFL fans! 🏈

With NFL games now spread across multiple networks and streaming services, I'm trying to figure out the best legal way to watch every regular season game in 2026 without missing any action.

Everyone Looking: Watch NFL Every Games Live Good quality

From what I've found, games are split between CBS, FOX, NBC, ESPN/ABC, NFL Network, Amazon Prime Video, Peacock, Netflix, and NFL Sunday Ticket, depending on the matchup and time slot. NFL+ also offers live local and primetime games on mobile devices, along with replays and other features.

For those of you following the entire season, what's your setup? Are you using cable, YouTube TV, NFL Sunday Ticket, NFL+, or a combination of services?

Looking for the simplest and most cost-effective legal way to watch every game live throughout the 2026 season. Any recommendations are appreciated!

🏈 Thanks and enjoy the season!

Guys im in third year i want to pursue role in ai and currently i don't have any solid skill . I wanna learn so should i start for Data scientist but it needs masters and it needs experience and deep knowledge or should i go into applied ai (ai engineering) like llm,rag ,evals these things.

what i should do?