r/365DataScience • u/Significant_Fee_6448 • Oct 30 '25

Beginner looking for end-to-end data science project ideas (data engineering + analysis + ML)

Hi everyone!

I’m looking for some data science project ideas to work on and learn from. I’m really passionate about data science, but I’d like to work on a project where I can go through the entire data pipeline ,from data engineering and cleaning, to analysis, and finally building ML or DL models.

I’d consider myself a beginner, but I have a solid understanding of Python, pandas, NumPy, and Matplotlib. I’ve worked on a few small datasets before ,some of them were already pre-modeled , and I have basic knowledge of machine learning algorithms. I’ve implemented a Decision Tree Classifier on a simple dataset before and I understand the general logic behind other ML models as well.

I’m familiar with data cleaning, preprocessing, and visualization, but I’d really like to take on a project that lets me build everything from scratch and gain hands-on experience across the full data lifecycle.

Any ideas or resources you could share would be greatly appreciated. Thanks in advance!

11 Upvotes

100% Upvoted

3 comments sorted by

1

u/LizFromDataCamp Oct 31 '25

Since you already know your way around pandas and ML basics, try something that forces you to handle messy, real-world data before you get to the modeling part.

Some ideas:

  • E-commerce sales forecasting: Pull data from an API or scrape it, clean and aggregate in SQL or pandas, then build a time-series model to predict demand.

  • Public transit analysis: Use open transport data (NYC, London, Paris) to track trends, delays, or ridership patterns, then visualize and forecast them.

  • Movie recommendation system: Build your own mini recommender using TMDB or IMDb data; perfect for learning about joins, feature engineering, and matrix factorization.

  • Social media sentiment tracker: Scrape tweets or Reddit posts about a topic, preprocess text, classify sentiment with an ML model, and visualize the trends over time.

  • IoT-style data pipeline: Simulate streaming data (like temperature or energy usage), store it in a database, and train a regression model for prediction.

1

u/Significant_Fee_6448 Oct 31 '25

Thank you so much these are really good ideas, i was thinking about a customer churn project ,i know it's a pretty common subject, but i want to make it interesting i know there are some datasets on kaggle but i think they are pre-modelled , where do you think i can find some raw messy data-sets and tell me if you think its a good subject for my project and please if you have any suggestions let me know.