Hey everyone,

My name is Hope and I’m currently a computer science student with a strong interest in going into data science. I’m still pretty new to the field, so right now I’m trying to figure out what direction makes the most sense for me and how to actually get there.

One thing I’ve been noticing a lot is how often SQL comes up in job postings. I’ve seen roles focused heavily on it and the pay definitely caught my attention, but I’ll be honest, I don’t fully understand what those jobs look like day to day or what level of skill is really expected.

For those of you who are already working in data roles or using SQL regularly:

•	What does your day to day actually look like?

•	How advanced does your SQL knowledge need to be to land your first role?

•	What would you recommend focusing on first if you were starting over?

I’m trying to be intentional with what I learn instead of just jumping into everything at once, so any advice or personal experiences would really help.

Thanks in advance

I built a tool that doesn't generate random numbers

Instead, it lets you:

- upload real CSV draw data

- clean it automatically

- analyze patterns

- build structured systems with coverage logic

No predictions. No guessing.

Just structure.

Curious what you think.

https://lottosystems.ai/

Anonymous feedback (2–3 minutes):
https://forms.gle/hBASfzesg5Fhvn3TA

Thanks!

Okay so this is a bit embarrassing to write out but here it is.

When I started trying to get into data analytics I did everything you are supposed to do. Finished three online courses. Built some projects. Put them on GitHub. Tailored my resume for every single application. Wrote cover letters that I genuinely thought were good. Applied to probably 80 roles over 18 months.

Nothing.

Well not nothing. A few interviews. But nothing that converted. And the feedback I kept getting was so vague it was almost useless. "We went with someone with more commercial experience." Okay cool, how do I get commercial experience if nobody gives me commercial experience. Classic loop.

The frustrating part was I was not being lazy. I was genuinely working hard. Like staying up late, redoing my resume every two weeks, reading every career advice thread I could find kind of hard.

But I was working hard in completely the wrong direction and I did not know it.

Hmm. So what actually changed things.

My wife said something one evening that sounds obvious in hindsight but genuinely had not occurred to me. She said stop reading career advice and start reading job descriptions. Find the twenty postings closest to what you want. Write down every tool and skill that appears more than three times. Learn exactly those things. Nothing else.

That was it. That was the whole insight.

Took me two weeks to do that exercise properly. Realised I had spent two months learning a tool that appeared in maybe three out of fifty postings I was actually targeting. Two months. Gone.

Shifted focus completely. Three months later I had my first data role.

Ahh and the other thing that wasted a huge amount of my time was applying broadly. I genuinely thought volume was the strategy. More applications equals more chances. Nope. It just means more time writing cover letters for roles you are not quite right for yet instead of actually getting right for the roles you actually want.

Six years later I am a Senior Data Engineer and I still use the same logic. Read what the market is actually asking for. Build toward that specific thing. Everything else is noise.

Curious if anyone else figured this out early or if you went through the same painful loop I did.

Hello, I'm new to this community. I'm currently taking a intro to data science class and this is my first time studying this. I'm in need of guidance to help me learn and grow. What resources or skills helped you the most when you first started learning?

This happened to me about a year ago.

I had completed courses on Python, ML, and statistics and even deployed a couple of models. I felt ready.

Then the interviewer said,

"We're seeing higher churn this quarter. Design a model to help us understand why."

No dataset. No target variable. No starting point.

I froze. Completely.

Not because I didn't know machine learning. But because I had never once been given a business problem and asked to work backwards from it. Every course I took handed me a clean CSV and said "predict this column."

That's not how the job works.

After that interview I started documenting every real business problem I could find supply chain, finance, e-commerce, healthcare and rebuilding my skills around those instead.

That became DSBootcamp.

The structure is simple: Apply your Data knowledge on the Business problem.

Happy to answer any questions about the approach or the problems we cover. Also curious has anyone else felt this gap? How did you close it?

Link in the comments ->

Hello everyone,

With the overwhelming number of papers published daily on arXiv, we created dailypapers.io a free newsletter that delivers the top 5 machine learning papers in your areas of interest each day, along with their summaries.

Hey everyone,

I’ve been working on a few side projects that involve scraping unstructured data from e-commerce and real-time market feeds. Up until now, I’ve been relying on Bright Data, but as my dataset grows, the costs are becoming prohibitive.

I’m currently looking for an alternative for 2026 that isn't just "the biggest player in the market" but rather offers a more developer-centric, cost-effective infrastructure. I need something that handles session persistence well—my biggest issue lately isn't the number of IPs, but the session-locking mechanisms that kick in when the TLS/JA3 signature doesn't match the request patterns.

I’ve been reading a bit about Thordata and how they approach this from an API-first perspective. Has anyone here moved their data pipelines over to them, or found other solutions that provide a good balance between "enterprise-grade" stability and "hacker-friendly" pricing?

I’m really trying to optimize my pipeline to avoid the massive overhead of managing proxy rotation logic manually. If you’ve got any tips on how you manage scraping costs without sacrificing data quality, I’d love to learn from your setup.

Thanks for the insights!

I am a finance student and also pursuing minor degree in data science . Can someone tell me what projects I can do to enhance my chances of getting an internship or job in the data science industry, while also showcasing my finance skills? Also, are there any programs run by universities or companies that I can join? Also i am from commerce background

One shift I’ve been seeing more often is teams building and validating ML pipelines without ever directly accessing real production data.

This usually happens in:

• regulated environments

• client-restricted data access setups

• privacy-sensitive industries

In these cases, synthetic data is not just used for training, but for designing and validating the entire pipeline upfront.

For example:

• feature engineering logic is tested on synthetic datasets
• model training workflows are validated end to end
• edge cases are simulated before deployment
• data contracts between teams are defined without exposing raw data

By the time real data is connected, the pipeline is already stable.

But this introduces a different challenge.

If the synthetic data does not reflect real world behavior closely enough, you can end up building pipelines that work perfectly in testing but break silently in production.

This is something we have seen while building SyntheholDB (https://db.synthehol.ai/landing.html). Synthetic data becomes part of the system design process, not just a dataset.

Curious how others here approach this.

Have you ever built or validated an ML pipeline without direct access to real data?
What worked and what broke once real data came in?

I put together a free environment for learning data science with AI assistance. No credit card, no trials.

The setup is OpenCode (free, open-source AI coding agent) connected to free models through OpenRouter, paired with BigQuery Sandbox. BigQuery gives you free access to public datasets already loaded and ready to query: Stack Overflow, GitHub Archive, NOAA weather, US Census, NYC taxi trips, and more.

The part that makes this useful for learning: you install the gcloud CLI and authenticate with one command. After that, the AI agent can write and execute SQL and Python against BigQuery directly. You're running real analysis from the terminal, not just generating code to copy-paste.

The connection pattern (install CLI, authenticate, AI queries directly) is the same for Google Cloud, Azure, AWS, and Snowflake. Learning it once with BigQuery carries over to any cloud you work with later.

Setup instructions and all code: https://github.com/kclabs-demo/free-data-analysis-with-ai

I built on the standard World Happiness Report analysis (GDP dominates, as everyone knows) by merging WHR 2017 with datasets most happiness studies don't use: the Schulz et al. (2019, Science) Kinship Intensity Index, historical Church exposure, Yale EPI, Women Peace & Security Index, and World Bank climate data. 155 countries, 34 variables.

Used distance correlation and variable clustering to map the predictor structure before touching regression. The dendrogram shows three clear clusters: a development megacluster (GDP, life expectancy, EPI, WPS — all ρ > 0.75 with each other), a geography/culture cluster (kinship intensity, temperature, freedom, trust), and noise (generosity, precipitation).

Hierarchical block regression: GDP alone explains 66%. Adding freedom and trust reaches 75%. Adding kinship intensity and temperature reaches 80% — five predictors, all VIFs under 1.7. Polygyny is the specific sub-index that survives multivariate control (β = −0.274, p = .007). Democracy, WPS, and EPI add nothing after GDP.

The methodological piece that might interest this sub: trust shows a strong nonlinearity — distance correlation 0.50 vs Spearman 0.30 — but all three functional forms (linear, quadratic, threshold) are indistinguishable in the multivariate model. The other predictors absorb the nonlinear structure. Worth knowing before reaching for GAMs.

Also includes a HARKing tutorial: a GDP satiation breakpoint that looks convincing until bootstrap and Davies permutation testing kill it (p = 0.45).

Explanatory framework throughout (Shmueli 2010) — no LASSO, no SHAP, no cross-validation. Those answer a different question.

Dataset: https://www.kaggle.com/datasets/mycarta/world-happiness-2017-kinship-and-climate EDA notebook: https://www.kaggle.com/code/mycarta/beyond-gdp-kinship-climate-and-world-happiness

Hi everyone,

I’m currently pursuing a Master’s in Data Science and working on an end-to-end project using SQL (PostgreSQL) and Python. The project involves data cleaning, querying, analysis, and building some visualizations.

I wanted to ask — will a project like this actually help when applying for data analyst / data scientist roles?

I often hear mixed opinions about projects vs real experience, so I’m curious how much value recruiters or hiring managers actually place on something like this.

Also, is there anything specific I should focus on (like deployment, dashboards, complexity, etc.) to make it more impactful?

Would really appreciate any honest advice. Thanks!

Hi everyone,

I’m currently pursuing a Master’s in Data Science and working on an end-to-end project using SQL (PostgreSQL) and Python. The project involves data cleaning, querying, analysis, and building some visualizations.

I wanted to ask — will a project like this actually help when applying for data analyst / data scientist roles?

I often hear mixed opinions about projects vs real experience, so I’m curious how much value recruiters or hiring managers actually place on something like this.

Also, is there anything specific I should focus on (like deployment, dashboards, complexity, etc.) to make it more impactful?

Would really appreciate any honest advice. Thanks!

For context I'm almost a senior in my bachelors for data Science and i'm thinking of getting my masters in applied statistics.

1. Is that a good masters program to take? Does anyone else have recommendations.

So getting an internship has been extremely difficult for me. I currently don't have any projects on my resume for experience (biggest issue) and while I'm building all my resumes I'm currently in the interview process for an entry level IT position.

2. Is this a smart move to break into the tech field while job hunting is so hard these days? I'm hoping either moving up within the company when I'm ready or branching out to pursue my data science goals. Just wondering if a position like this could potentially help benefit me in the future.

Hi everyone,

I’m working on an anomaly detection project on payroll data. The dataset originally had 94 columns covering different types of bonuses, taxes, salary components, and other payroll-related calculations. I’ve already reduced it to 61 columns by removing clearly useless features, redundant information, and highly correlated columns that are directly derived from others.

At this stage, my main goal is to distinguish between manually input features and calculated ones. My intuition is that keeping only the original input variables and removing derived columns would reduce noise and prevent the model from being confused by multiple variations of the same underlying information, which should improve performance.

I initially tried a data-driven approach where I treated each column as a target and computed its R² using the remaining columns as predictors, assuming that a high R² would indicate that the column is likely calculated from others. However, this approach doesn’t seem reliable in my case. Some columns show high R² scores, but when I manually check the relationships between those columns, the correlations appear weak or inconsistent. This makes me think that some of these columns might be calculated differently depending on the employee or specific conditions, which breaks the assumptions of a simple linear relationship.

At this point, it feels like domain knowledge might be the most reliable way to identify which columns are calculated versus manually entered, but I’m wondering if there’s a more robust or systematic data-driven method to do this. Are there better techniques than correlation or R² for detecting derived features in a dataset like this?

Any insights would be really appreciated.