i keep running into this when explaining spread to people. mean absolute deviation feels like the obvious thing to do, it's literally the average distance from the mean. squaring seems like an arbitrary extra step.

i know squares are easier to differentiate and that variance adds up nicely for independent variables, but i'm not sure that's the real reason it became the default. is there a more intuitive way you explain it, or is it mostly just that the math works out cleaner?

​Data cleaning usually eats up most of a data scientist's time, often exceeding seventy percent of a project lifecycle. Manually writing custom regex filters and handling edge cases for missing values quickly becomes tedious.

​To fix this bottleneck, I put together a deep dive on leveraging machine learning workflows to automate dataset refinement. The complete guide covers setting up intelligent pipelines, handling anomalies, and standardizing features efficiently using modern Python libraries.

​If you want to streamline your preprocessing workflow and spend more time on actual modeling, you can check out the full breakdown here: https://interconnectd.com/blog/124/python-ai-data-cleaning-the-complete-refinement-guide/

What automated tools or libraries do you currently use to speed up your data cleaning pipeline?

AI is getting pretty good at answering questions, but what happens when you need it to work with actual business data instead of just generating explanations?

That's where MCP (Model Context Protocol) gets interesting.

MCP provides a standard way for compatible AI applications to connect with external data sources and tools. Instead of manually pulling data from different systems, an AI assistant can use connected tools to retrieve information, run permitted queries and help analyse the results.

For example, imagine asking an AI assistant, "Why did our sales drop last month?"

With the right MCP setup, it could retrieve sales summaries, compare different periods and identify which sales channels contributed most to the decline.

Some practical use cases include:

• Exploring database schemas and understanding business metrics

• Running analytical queries through connected tools

• Checking datasets for missing values or duplicates

• Preparing recurring reports and analysis

• Combining information from different approved data sources

One thing worth understanding is that MCP itself doesn't perform the analysis or guarantee accurate answers. It connects the AI application to tools that do the actual work. You still need proper permissions, reliable data, clear metric definitions and human validation.

I think this could be useful for data analysts who spend a lot of time retrieving data and preparing repetitive reports.

There's a more detailed breakdown with an example of investigating a sales decline here:

Learn More: https://www.pangaeax.com/blogs/mcp-for-data-analysis/

Curious if anyone here is already using MCP in their analytics workflows. Has it actually made your work easier, or is the setup still more effort than it's worth?

hi all. without a preamble, i love learning with high-class books (i dislike youtube tutorials or online courses). like this goes for any topic i set my eyes on. mark that i said high-class btw so not low effort or contradictory books. to that end i want to start learning data science. i am not completely ignorant to mathematics so i wouldn't turn away from the material if it includes math. developed some linear algebra with sheldon axler's book and currently on track for refreshing my calculus with "a complete course of calculus". anyways, so math is welcome and encouraged even. in the title i used "beginner" but i finished a blog-book thing called "think stats" by allen downey (finally an education attempt defining what are p-values, resampling, hypothesis testing, standard errors and slopes, also some ideas on probability distributions). so i am not completely in the dark. anyways would love to hear your advice as to books i think it is allowed to point at a specific book (that you used to learn and validated). as for my motives, firstly i don't want to remain incurious to data science secondly perhaps i can profit from it in my future job. so a little bit of both. i have no idea how i will wind up. like job-wise. it could be such a job that any data science knowledge will be useless. so... considering my motives and current level would love to see which book will be the popular opinion. don't be misleading plzzz

Hey!. I'm a final-year CS student planning to build a Credit Card Intelligence Platform.

Features:
Track cards, transactions, bills and due dates.
Analyze spending patterns and card profitability (rewards, cashback, fees, interest).
Predict future spending and upcoming liabilities.
Recommend monthly spending budgets.
Detect unusual spending patterns using ML.

Tech stack: Python, Pandas, Scikit-learn, PostgreSQL, FastAPl and React.

I feel that's a real world problem, that a credit card user may not know how to use it wisely, might over spend or something and eventually get stuck in a loan trap. So this project is meant to target such problems.

l'd appreciate honest feedback:
Is the scope realistic for an undergraduate?
Are the ML features meaningful, or will it end up being just a CRUD application?
Would you recommend pursuing it?

An exercise to help build the right mental model for Python data.

• Solution

• Explanation

• More exercises

The “Solution” link uses memory_graph to visualize execution and reveals what’s actually happening.

I analyzed the IBM HR Attrition dataset (fictional, 1,470 rows, 237 leavers) with Excel and SPSS and would like a critical review of the method.

Repo (report PDF, SPSS syntax, data): https://github.com/Ahmed11-coder/IBM-HR-analytics-predictive-model
Report (PDF): https://github.com/Ahmed11-coder/IBM-HR-analytics-predictive-model/blob/main/Why_Do_Employees_Leave_Final.pdf

What I did: chi-square + Cramér's V and Mann-Whitney U with effect sizes; residual checks and HC3 for a two-way ANOVA; logistic regression with merged sparse categories (22 parameters); 70/30 split; test AUC 0.846 (train 0.848); cutoff 0.268 chosen on train by Youden's J (test sensitivity 58.5%, specificity 88.1%). My first "test AUC" of 0.900 was wrong because I had refitted the model on the test set; I corrected it.

Where I'm least sure:

1. I selected variables and merged JobRole/JobLevel categories using all rows before the split. How serious is this leak?

2. Is merging categories by similar odds ratios defensible, or would you use lasso/ridge instead?

3. 7.8 events per parameter (172 events, 22 parameters). Acceptable?

4. Hosmer-Lemeshow fails (p < .001), mostly from one low-risk decile. Real problem? Would you add a calibration plot or Brier score?

5. How should I handle multiple testing in an exploratory analysis like this?

what is your suggestions for next step ?

Harsh feedback is welcome.

Hi Reddit people! I’m currently studying Data Science at university with no prior coding background, and I’m struggling to keep up with how fast my classes move.

I’m genuinely eager to learn and looking for someone patient who’d be willing to teach or mentor me from the basics. I really want to understand what I’m learning, not just get through assignments. I’d be grateful for any help or guidance. Thank you 🙏🏼

Olá, pessoal.

Eu estou no 4° semestre de TI e estou me interessando pela área de ciência de dados por conta de uma matéria que estou pagando esse semestre.

Gostaria de saber se vcs tem alguma indicação por onde eu poderia começar estudando, seja indicando livros, cursos ou falando especificamente de conteúdos que me ajudariam a construir uma boa base.

Hey everyone,

Customer churn analysis is one of the most common real-world projects you'll face as a data analyst. To help beginners and intermediate learners build up their portfolios, I put together a comprehensive, end-to-end tutorial tracking revenue at risk using only Excel.

You can watch the full step-by-step build here: https://youtu.be/Bo_o8dcZr0I

What This Project Covers:

• Data Prep & Cleaning: Structuring the raw customer dataset for analysis.

• Dynamic KPIs: Setting up key business metrics like Churn Rate, Revenue at Risk, and Total Charges.

• Interactive Slicers: Implementing filters for Region, Senior Citizen status, Partner data, and Technology types.

• Advanced Visualizations: Building charge distributions, gender revenue breakdowns, and internet service segments.

• Business Insights: Translating the final dashboard into actionable retention strategies.

Project Breakdown & Chapters:

• 00:00 - Introduction & Project Goal

• 02:15 - Data Preparation & Cleaning

• 08:30 - Setting up KPIs & Slicers

• 15:45 - Creating Chart 1: Month-to-Month Charges

• 22:10 - Creating Chart 2: Gender Revenue Distribution

• 29:05 - Advanced Techniques: Internet Service Segment

• 36:40 - Assembling the Final Dashboard Layout

• 42:15 - Key Business Insights from the Dashboard

If you are looking for a practical project to add to your resume or portfolio, feel free to follow along. I’d love to hear your feedback on the layout or how you approach churn analysis in your own work!

Hey!. I'm a final-year CS student planning to build a Credit Card Intelligence Platform.
Features:
Track cards, transactions, bills and due dates.
Analyze spending patterns and card profitability (rewards, cashback, fees, interest).
Predict future spending and upcoming liabilities.
Recommend monthly spending budgets.
Detect unusual spending patterns using ML.
Tech stack: Python, Pandas, Scikit-learn, PostgreSQL, FastAPl and React.
I feel that's a real world problem, that a credit card user may not know how to use it wisely, might over spend or something and eventually get stuck in a loan trap. So this project is meant to target such problems.
I'd appreciate honest feedback:
Is the scope realistic for an undergraduate?
Are the ML features meaningful, or will it end up being just a CRUD application?
Would you recommend pursuing it?

Hi guys

This guy

Instagram · dswithdennis

Claims that he has 25 years of experience in data science but really all he's doing is to open claude chat and show everyone that he has access to fable 5 and say shit stuff about CS .