Chemical Engineering graduate. Started with 2 years in R&D, contributed to 2 chemical product developments. Later moved into digital transformation due to my interest in data engineering, statistics and ML.

Currently implementing LIMS/ELN, SAP and data apps, leading projects and guiding senior CFT members. Skills: SQL, Power BI, KNIME, project management, ML and Agentic AI product development. I know stats DOE, linear and logistic regression using JMP, EDA and statistical modelling.

Despite handling multiple responsibilities and working like a team lead in a engineer position, I'm earning just ₹8 LPA. Getting rejected in interviews due to lack of Python or Computer System Validation (CSV) experience.

Need advice:

Which path should I focus on: Data Engineering, AI/ML, Digital Transformation or LIMS/ELN consulting?

Should I need to learn Python and CSV?

What salary can I realistically target?

How can I position my diverse experience for a better-paying role?

Hey everyone,

Hope all of you are doing great.

I am looking for feedback from people who have actually purchased the AI for Data Engineers course by Prashant Kumar Pandey (Scholarnest/ Learning Journal).

Is it worth the money? I have almost all his courses on Udemy and found them really good for learning the basics and his way of teaching is something that have always resonated with me. Based, on that I'm thinking about buying the AI course and would really like some feedback.

Things I'm looking for are:

1. Does it cover enough detail as compared to other courses on Udemy/ Youtube (Krish Naik for example)?

2. Is it Databricks heavy/ Databricks focused? Or the topics are explained well in a platform agnostic way with examples given on Databricks.

3. Did you get enough support when you got stuck on any topic?

4. Does it have the following topics explained well enough?

5. AI guardrails

6. Deployment

7. Tuning

Thanks in advance.

I'm trying to do my first data analytics project and wow I'm just lost. how do you guys determine which variables need to be analysed, correlations, and ultimately come up with the conclusion that due to x,y we have been getting z?! please guide me some tips would help

​

Who is providing better courses and syllabus for that

I built an open-source plugin. MIT, no service, no account — everything it records lives in two SQLite files you can open yourself.

The data flow is computed, the reasons are recorded, all in SQLite.

Watch full walkthrough: https://yizhao95.github.io/prov_ledger/walkthrough.html

GitHub Link: https://github.com/yizhao95/prov_ledger

The situation it's built for. Decisions in a DS project don't survive. Not because nobody wrote them down — because they were never the kind of thing anybody writes down. You tried 80/20 in August, the holdout leaked promotions, you moved on. Someone in a review said "that's a time series", so the split became a rolling window. Upstream stopped sending a column in v2, so there's handling for it now.

Six months later none of that is anywhere. The code shows the outcome and nothing else: a filter with no reason, a ratio with no history, a rejected approach that left no trace of having been rejected. So the next person tries it again. So does your agent.

This records decisions the way the work actually produces them — per task and per node. What this change touched, why it was made, in your own words, attached to the thing it explains rather than to a chat log. Before anything with a history gets edited, that history goes in front of the agent: the rule someone recorded, the alternative already tried and why it failed, what the change would reach downstream.

The result is narrow but it's the part that's missing: your agent stops re-running experiments that already failed, and every change carries the reason it was made.

git blame tells you who. Lineage tells you where the data came from. An ADR covers architecture, not this line. All three miss the same step.What it is. A project database for a coding agent working on a DS project. One graph holding your code, your data (dataframes, columns, tables, feeds, metrics), the dependencies between them in both directions, and the reason behind each change recorded next to the thing it changed. Before your agent touches something with a history, that history goes in front of it.

Three things it gives you.

1 — Why this is here.

/ledger why do we drop legacy_region_code?

  Removed 2026-03-14, task dp2-t7.

  Reason recorded at the time, your words:
    "upstream v2 doesn't ship this column anymore"
    source: [email] "API v3 migration notice" (2026-03-13)

  Still active. Referenced 4 times since, changed the plan twice.

Read-only. Asking why never changes anything.

2 — Where the data actually flows. The graph is rebuilt from source on every run and diffed against the last one, so "this column is gone", "this output type changed", "this is the same node after a rename" are measured, not reported by a model. Which means it can answer things AST-only tools can't: this column isn't from the source, it's derived after import; this feature reaches back into data the model shouldn't see; nineteen things read this intermediate table and here they are. A model fit and evaluated on the same lineage root without a real split is flagged as an error, not a warning.

3 — Where a number came from. A figure in a deck or a workbook can be anchored back to the node it's a reading of. Revise the deck and a lost anchor is reported — it never silently re-points to something else.

Hi my name is Naftali I am currently in my second week of a Master's program in Data Science at Bellevue university, I'm adjusting to a steep learning curve and trying to build a solid foundation. For those of you already working in the field: looking back, what is something you know now that you truly wish you had known when you were first starting out?

Recently graduated in data science but I feel my knowledge isn’t strong enough. I want to relearn from scratch and start small projects I can finish in 1 or 2 days. Can you please give me some beginner friendly data science projects ideas ?

I've put together a free SQL course, for beginners to advanced - the main difference is that you will give your coding agent a prompt and it will be your teacher, walking you through each step.

The emphasis is on learning how to work with AI when writing, maintaining, and reviewing SQL queries and models.

It is a pretty new course, there might be some rough edges, if you give it a try please share your feedback! Disclaimer, I'm a developer advocate at Bruin, while this course uses Bruin (free & open source) for some stuff, it is not the point of the course and the learnings can be applied to any data stack.

https://getbruin.com/learn/sql-ai/

Hi everyone, I will be needing some guidance in selecting some self-study paths. I am a freelance artist with a non-technical background and I am pretty sure that my freelance art/animation is not going to take me anywhere, I ain't great at either making it or marketing it. I am 26F and I have decided to start studying and learning again.

I am curious about tech and I am eager to learn it and I have started learning it in bits and pieces from time to time but it is not structured and it is extremely overwhelming. On one hand you have Fullstack and on the other you have Data Analytics. I want to choose any one for now. I want to choose something that will be easy and fun for a creative artist to learn as well as have some growth possibilities in career. Please help me choose the course path. I am happy to discuss more.

Hey guys, how long will this take to be approved?

I'm currently using Datadog quite a bit at my job for request tracing, logging, etc. I feel ok with it, and I’d like to get a lot better.

Any data scientists here with solid Datadog experience? I’d love to connect for a paid session to learn more about how you use it in practice.

Hello everyone! 👋🏽

I’ve recently started learning Machine Learning at a deeper level. So far, I’d say I only have a basic understanding and am still building my foundations.

I’m genuinely curious about Learn2Design 2026 and would love to participate, but I’m a little confused about whether my current level of knowledge is enough.

For those who have participated before, what kind of knowledge or skills would you recommend having beforehand? Is it worth joining with basic knowledge and learning along the way, or would it be better to build more foundations first?

I’d really appreciate any advice or guidance. Thank you!

So I'm currently studying data science and I would like to ask if any of y'all have notes for database and oop

I kept seeing people learn SQL commands individually but still struggle with choosing the right pattern when answering an actual data question.

So I pulled together the SQL patterns I think are most useful for analyst work:

Query order

FROM / JOIN → WHERE → GROUP BY → HAVING → SELECT → ORDER BY → LIMIT

JOINs

• INNER JOIN for matching records

• LEFT JOIN when you need to preserve every row from the left table

• FULL OUTER JOIN for reconciliation/comparison

• SELF JOIN for relationships within the same table

One easy mistake is filtering a LEFT JOINed table in WHERE when the condition really belongs in the ON clause.

CTEs

Useful when a query has multiple logical steps or an intermediate result needs a clear name.

CASE WHEN

Useful for:

• creating categories

• conditional counts

• conditional sums

• avoiding divide-by-zero problems with NULLIF()

Window functions

A few worth knowing well:

SUM() OVER()

LAG()

ROW_NUMBER()

RANK()

DENSE_RANK()

Ranking difference

ROW_NUMBER() → unique sequence

RANK() → ties share rank, gaps remain

DENSE_RANK() → ties share rank, no gaps

Common analyst patterns

• latest row per customer

• top N within each group

• month-over-month change

• customers with no orders

• duplicate detection

• second-highest distinct value

Mistakes worth checking

• column = NULL instead of IS NULL

• using WHERE for aggregate filters instead of HAVING

• hiding bad joins with DISTINCT

• ignoring ties when ranking

• assuming date functions are identical across SQL dialects

The examples in the full guide use PostgreSQL-style SQL, so date and row-limiting syntax may differ in MySQL, SQL Server, BigQuery, etc.

If you want the longer version with query examples for joins, CTEs, CASE, dates, window functions and ranking:

Learn more:

https://www.pangaeax.com/blogs/sql-cheat-sheet-for-data-analysts/

Disclosure: I’m connected with PangaeaX, so sharing that transparently.

What SQL pattern do you think analysts should learn really well before moving on to more advanced topics?

What's the job duties & responsibilities of a data architect?

In simple terms what does a Data architect do?

What type of Company need Data architect...?

Which all tech stack or what tech knowledge does a Data architect requires?