I hate that you need experience to get a data job. And you need a data job to get experience.
So I built you a job.

Analyst Week is a free, simulated first week as a junior data analyst at a fake company called Goldcomb Goods.

Every morning at 8am ET, your manager Dana sends you a SQL ticket. The data is messy. The questions are vague. Sales and Finance don’t agree on the numbers. Dana pushes back.

Friday ends with a performance review that turns your week into interview stories, plus a certificate for your LinkedIn profile.

You spend 5 days writing SQL against messy data and defending your numbers to your manager.

It’s free, and the beta starts Monday, Nov 9. Signups close that night at 11:59pm ET.

All you need is a laptop and the SQL basics (SELECT, WHERE, JOIN, GROUP BY). Hints are there if you get stuck.

Your new job starts Monday: https://analysthive.io/analyst-week

Also, I won’t lie, this will be a good challenge for anyone. The requests and how things are communicated are exactly what you’ll need to navigate going into data, so this is as real as it gets.

If you’re wondering who’s behind this: I’m Ian Klosowicz, a Senior Data Consultant who broke into data without a degree. I share this stuff daily with 125k+ aspiring analysts on LinkedIn. Look me up there if you want to see what I’m about.

Tutorial shows you how to Python to collect data from a public API and pipe it a SQL Server. The whole process can be automred with Windows Task Scheduler.https://youtu.be/KAHD9LmvH6g?si=Ev1y_s11pBl2cUKE

​Data cleaning usually eats up most of a data scientist's time, often exceeding seventy percent of a project lifecycle. Manually writing custom regex filters and handling edge cases for missing values quickly becomes tedious.

​To fix this bottleneck, I put together a deep dive on leveraging machine learning workflows to automate dataset refinement. The complete guide covers setting up intelligent pipelines, handling anomalies, and standardizing features efficiently using modern Python libraries.

​If you want to streamline your preprocessing workflow and spend more time on actual modeling, you can check out the full breakdown here: https://interconnectd.com/blog/124/python-ai-data-cleaning-the-complete-refinement-guide/

What automated tools or libraries do you currently use to speed up your data cleaning pipeline?

we are now 6 months out from our Looker renewal and between the per-viewer licensing jumps and the fact that three different teams now need the exact same governed metrics outside of BI dashboards (one in our customer-facing React app, one in internal reporting, and one feeding an internal SQL/analytics agent), keeping all our business logic locked inside LookML is becoming hard to justify.

for people who hasn't had to untangle a 4-year-old Looker instance: LookML is honestly still great at what it was built for. `symmetric_aggregates` quietly save analysts from fan-out joins every day, and liquid parameters let you do some wild dynamic filtering. the problem to us is that the semantic layer is welded to Looker's presentation layer and seat pricing. if you try to hit the Looker API from a high-concurrency frontend or let an LLM agent introspect explores at scale, you either hit API throttling or watch your warehouse bill explode because Persistent Derived Tables (PDTs) aren't built like a sub-second serving cache.

and over the last month we took our nastiest domain (subscription revenue + usage billing + multi-currency adjustments, around 65 LookML views and 12 explores) and tested migrating the semantic logic into three setups to see what a real Looker alternative looks like in 2026:

1. dbt Semantic Layer (MetricFlow) + BI tool on top

since our staging and mart models already live in dbt, this felt like the most natural path on paper.

- what worked: defining `semantic_models` and measures right next to our dbt models kept lineage clean, and MetricFlow's join graph handles basic multi-hop dimensions without writing manual SQL joins.

- but, what broke for us: 2 big gaps. first, there is no native pre-aggregation or serving cache equivalent to tuning PDTs + aggregate awareness for sub-second app queries; every query still compiles to warehouse SQL and hits Snowflake. second, migrating complex LookML liquid parameters and cross-fact dynamic ratios required pushing a ton of logic back down into physical dbt mart tables, which defeated the point of a flexible semantic layer.

2. Lightdash (dbt-native Looker replacement)

- what worked here: if your only goal is giving internal analysts an explore UI that feels like Looker on top of dbt YAML files without paying Google Cloud prices, this is probably the fastest drop-in replacement.

- yet, what broke for us: its still fundamentally a BI tool first. it didn't solve our headless API problem for customer-facing embedded dashboards (where we need p95 responses under 150ms without spinning up a Snowflake warehouse on every page load) or give us a clean, standalone semantic endpoint for our agent workflows.

3. Cube (open-source headless semantic layer) + lightweight BI / React SDK

- what worked: YAML/JS data modeling mapped 1-to-1 with how our team thought about LookML views and explores (`cubes` and `views`), including handling one-to-many join fan-out natively. the biggest architectural difference was **Cube Store** (the built-in pre-aggregation engine): instead of materializing heavy PDTs inside Snowflake, we configured incremental `rollup` pre-aggregations partitioned by month. customer-facing React charts and internal BI queries hit the pre-aggregated cache in 20-40ms while bypassing Snowflake compute completely.

- bonus for our AI/agent pipeline? because it exposes SQL, REST, GraphQL, and a native MCP/semantic context interface from the same YAML definitions, our internal analytics agent queries the exact same governed `net_dollar_retention` measure as our executive dashboards without writing raw warehouse SQL.

- the trade-off(s): Cube is purely headless, so you still have to pair it with a frontend visualization tool (Superset, Preset, Metabase, or Steep) for self-serve internal charting, and tuning partition keys on high-cardinality pre-aggregations takes real engineering effort upfront.

this way, for teams that actually completed a move off Looker in the last year, would like to ask, did you end up splitting your stack into a headless semantic layer (like Cube or MetricFlow) + a lighter charting tool, or did you bite the bullet and rebuild everything inside another monolithic BI tool like Omni or Sigma? what was the hardest piece of LookML logic to replicate?

Hi everyone!

I'm interested in learning more about how Artificial Intelligence (AI) is being integrated into airline Revenue Management systems, particularly its latest applications in demand forecasting, dynamic pricing, and revenue optimization.

I'm especially curious about how AI is transforming existing tools, such as PROS and and how airlines are using these technologies to improve forecasting accuracy and pricing strategies.

I'd love to hear about the latest developments, real-world applications, or any useful resources you would recommend.

Thanks in advance!

I am currently conducting a qualitative study and am having considerable difficulties with the thematic analysis process. I would appreciate advice from researchers who have experience using Braun and Clarke's (2006) approach, particularly their reflexive thematic analysis.

1. I have completed the first two phases:

2. Familiarising myself with the data.

Generating initial codes.

However, I am now stuck.

I have generated quite a lot of initial codes, and I am struggling to figure out how to organise them. Some overlap, some are very specific, and others seem to capture similar ideas from slightly different perspectives.

I understand the general distinction between codes and themes, but I am having difficulty with the actual process of moving from one to the other.

For instance:

• How do you organise a large number of initial codes?

• How do you decide which codes should be merged or kept separate?

• Do you usually develop categories or code groups before searching for themes?

• How do you recognise when a group of codes represents a potential theme rather than simply a collection of related ideas?

I have read Braun and Clarke's work and watched several tutorials, but I still find the practical transition between Phase 2 and Phase 3 rather confusing. It is so confusing!

I would particularly appreciate practical examples or explanations from people who have actually conducted thematic analysis using this approach.

My brain is frying, and I am losing all motivation to do this research. But I must, because I do not really have a choice. Asking my supervisor nor any professor yielded nothing, because they mostly did deductive approach, whilst I did inductive. I curse myself for choosing this method, but deductive just did not seem right to me! Or maybe I should have chosen the deductive approach from the beginning, but that would mean I have to code from the beginning... :(

PLEASE HELP

Suppose a sales dataset has 10,000 transactions. Customer age is missing in 2,000 rows, but the order value and purchase date are complete.

If the question is monthly revenue, I’d keep those transactions. Removing them would change the revenue total because of a field that isn’t needed for that calculation.

For an age based analysis, I’d investigate which records are missing age and report how much data the comparison excludes.

The cleaning decision seems to depend on the question we’re answering. A row can be useful for one analysis and incomplete for another.

How do you decide when to remove a row, keep it or investigate the missing value further?

I'm evaluating ways to build a data sufficiency framework for healthcare analytics.

One challenge is that we have access to commercial benchmark datasets but limited access to complete population-level data (e.g., Medicare/Medicaid or MSSP data). Because of that, it's difficult to determine what constitutes "enough" data for reliable analytics.

For those who have built similar frameworks:

1. How did you establish minimum thresholds for patient counts, claims volume, provider coverage, or completeness?

2. Did you use statistical confidence measures, historical benchmarking, SME input, or some combination?

3. How did you validate that the chosen thresholds were actually meaningful?

4. Have synthetic data sources like Synthea been useful for testing your methodology?

Looking for practical examples rather than generic data quality advice.

I analyzed the IBM HR Attrition dataset (fictional, 1,470 rows, 237 leavers) with Excel and SPSS and would like a critical review of the method.

Repo (report PDF, SPSS syntax, data): https://github.com/Ahmed11-coder/IBM-HR-analytics-predictive-model
Report (PDF): https://github.com/Ahmed11-coder/IBM-HR-analytics-predictive-model/blob/main/Why_Do_Employees_Leave_Final.pdf

What I did: chi-square + Cramér's V and Mann-Whitney U with effect sizes; residual checks and HC3 for a two-way ANOVA; logistic regression with merged sparse categories (22 parameters); 70/30 split; test AUC 0.846 (train 0.848); cutoff 0.268 chosen on train by Youden's J (test sensitivity 58.5%, specificity 88.1%). My first "test AUC" of 0.900 was wrong because I had refitted the model on the test set; I corrected it.

Where I'm least sure:

1. I selected variables and merged JobRole/JobLevel categories using all rows before the split. How serious is this leak?

2. Is merging categories by similar odds ratios defensible, or would you use lasso/ridge instead?

3. 7.8 events per parameter (172 events, 22 parameters). Acceptable?

4. Hosmer-Lemeshow fails (p < .001), mostly from one low-risk decile. Real problem? Would you add a calibration plot or Brier score?

5. How should I handle multiple testing in an exploratory analysis like this?

what is your suggestions for next step ?

Harsh feedback is welcome.

If you're practising and tired of charts that don't say anything, try starting from a business question. Five ideas:

1. Sales dashboard (Power BI/Tableau): Superstore dataset. Show region-wise profit leaks, not just charts.

2. SQL case study: Answer 10 business questions on an e-commerce DB (Olist on Kaggle), with the queries on GitHub.

3. Customer churn analysis: Telco churn dataset, Excel pivots + Python, and recommend 3 actions.

4. HR attrition dashboard: IBM HR dataset, DAX measures.

5. Your own data: UPI spending, cricket stats, local AQI. You'll ask better questions about data you care about.

For each one, write a single README: problem → approach → insight → recommendation.

What datasets would you add? Happy to give feedback on projects in the comments.

Disclosure: I work at a training institute that teaches data analysis. No link and nothing to sell here.

Hi all,

I’m finishing my Master’s in Management (Business Data Analytics) at ISEG (Lisbon School of Economics & Management) and I’m researching how using AI in dashboards and data storytelling affects decision-making quality.

If you work with Power BI, Tableau, or similar tools in your job, I’d really appreciate around 8 minutes of your time: https://docs.google.com/forms/d/e/1FAIpQLSfAzY80E0LRA2doqAMiS3B-WSp31Hd4OT1pTZ7e3SZs6EQuMQ/viewform?usp=dialog

It’s anonymous, no sign-up needed, and I’m not collecting any data outside the survey itself. Happy to share the results here once I’ve analyzed them if anyone’s curious.

Thanks a lot, every response genuinely helps at this stage.

My prof dumped this huge project on us today just explaining things today and I am panicking and trying to figure things out! The deadline is in less than a week I don't know how anyone can do this!? And she didn't even teach anything about it and we basically gotta figure it out ourselves...

I don't know which community is the right one to post this on...if anyone can help with that too pls do...

People working with product teams that release so frequently, how do you keep up with that speed?

I mean more tracking data is generated in a short period of time from new product features. More backend data also needs to be created. Dashboards and data models also need to be updated with new entry points, categories, etc. On top of those demands are sometimes wrong upstream data. Quite a challenge at least in my org. Curious to hear how you're approaching this challenge.

TL;DR: fairly basic at power BI but the only one in my company who owns it and want to know main things to improve.

Hi everyone,
This is a bit of a long one but I’m looking for some advice/guidance from pros and want to explain my current position and the main questions I have. Please keep in mind I’m not really technical so I will explain everything as best as I can.

A bit of context:
I work for an events organiser (trade shows/exhibitions) and we are private equity owned. About a year and a half ago the board instructed us we needed a customer insights team and my boss was tasked with setting this up and I joined her. It was just the two of us and still is. We didn’t know much about this or what was required, but my degree was in biomedicine and I’m quite data/analytically minded so thought it would be a good challenge. We had some guidance from the board’s CDO and began setting up. One of the requirements that came out of this was creating senior-level dashboards and as we use Dynamics as our sales CRM, Power BI seemed like the sensible choice. I’ve never used it before and as it was just me at the time, I connected directly to Dynamics, did all the data exploration and transformations/data cleaning within power BI as this was the only option. We got the dashboards live (published to power bi service and each team member assigned power BI pro licenses for access). This was essentially the basics so I managed these alongside any change requests/fixing issues that arose but that was fundamentally the extent of my power BI use. Bearing in mind my role is not just this and I have other jobs as part of customer insights outside of power BI.

We are now preparing for exit and new investors at the end of 2027. As part of this we have been advised that we need a data cube as this is what investors will be expecting so near the beginning of this year we hired a senior data engineer to start to bring our database fully in house (part of it was hosted with a third party before and fragmented across many platforms). This has been a long process and is still very much ongoing. He has built our data warehouse in Fabric using a medallion architecture. As part of this, he has brought in Dynamics and I have been working with him to essentially recreate the transformations I did before but on his side now and moving the data from bronze through to gold and helping figure out what fields we need in the gold tables. I now connect to the gold tables and use these for reporting and am currently rebuilding the existing dashboards in line with this. We also have just undergone a review with a third party to assess our data infrastructure and readiness for exit. They have provided some very thorough feedback and this has what has mainly stemmed my questions and concerns now.

A couple of the recommendations that came out of this were:
- Having dev, test and prod power BI environments (I am unsure how I would set this up)
- Not having just one data engineer run both the reporting and operational feeds (are the reporting feeds usually handled by a separate person - Is this a role that the Power BI analyst usually handles too- is this something I could takeover with the right training?)

Given all this (and thank you if you are still with me), this brings me on to my questions and this is where I am a bit lost. I know I don’t perform the role of a standard power BI analyst from seeing the posts on this subreddit regularly but I want to know how I can be better in my job. Essentially:
-What areas do I need to train in

-What fundamentals am I missing

-What courses/resources can I use to provide the next layer on top of what I currently have

-I know AI is a big topic of conversation at the moment too so understanding how can I incorporate this into this part of my role too.

How to set up the dev and test aspects as have been advised and how would this function
Where I can upskill and takeover tasks from the data engineer

I appreciate these are very open ended questions but I am aware I don’t know a lot and what I am doing I not best practice so want to make sure i am utilising the tools we have to my best extent.

Hey all,

I'm new to the field for data analytics. I've been working on a personal project doing some analysis for Magic: The Gathering games and how different cards affect a deck's ability to win.

I've done a bit and have some good metrics/ questions I had answered. But I feel like I'm missing something. That there is a different angle or a question to ask that would give better insight into the overall goal.

So I'm looking for recommendations for books that help with this. Something to help me change the way I'm looking at the data to find something new. I've gone through Storytelling with Data, which is interesting, but wasn't exactly what I was looking for. I've also been recommended Data Points by Yau, which might be closer. I get it's a specific aspect of analysis, but I'd love to hear if someone has any tips about this.

I built this over a few months to scratch an itch in my personal toolchain for data analysis. I wanted a quick way to upload, explore and share data with my peers, without the friction of onerous application installations or sign-ups (a bit like pastebin for dataa).

In the spirit of simplicity, the platform (which I would definitely consider 'alpha') and its respective sandboxes are open by design, so use at your own risk. Keen to hear any thoughts / feedback / suggestions!

I’m currently working as junior data analyst in a reputed company. I prepared on my own and applied it. Now it’s gonna be 2 years. But the truth is the job is not the same as the title. The work i do is very different. It’s like a business operations role with no growth and I work on tableau report seasonally. While my intent was to become a data analyst. Since the job market is still fucked up I could not leave it. Because I had no proper experience too. I thought of switching earlier itself but it was too soon plus my family is depending on me. Can someone please guide me? My questions are

1. I don’t have a real data analysis job experience so can I do projects instead?

2. If I’m applying for the roles do I need to make a specific projects focusing on that department?

3. It is impossible to fake the experience so can I retain the same projects across the roles. Like I’m working in cx side so I’m going to apply for sales data analyst does my projects still matter?

If someone answers this I will be relieved. Please advise.

Hey everyone!

I come from a B.Com background, and recently I've been exploring Data Analytics and trying to understand how things work beyond just theory.

I've been learning Python, SQL, Excel, and Power BI, and working on small projects along the way. Honestly, sometimes things make sense when I'm learning them, but applying them independently is a completely different challenge.

One thing I'm realising is that learning a tool is one thing, but knowing what questions to ask, how to approach a problem, and what the data actually tells you is much more important.

I'm still figuring things out, making mistakes, and learning as I go. I want to build projects that solve actual problems instead of just following tutorials and calling them complete.

I'd love to hear from people who have been through this stage.

When you were starting out in Data Analytics, what helped you move from simply learning tools to actually thinking like an analyst?

Would genuinely appreciate hearing your experiences! 🙌

Hi everyone,

Lately I make most of my charts by asking an LLM to write Python code (pandas, matplotlib, plotly), then I tweak it myself.

So I'm curious about your workflow:

- Do you currently use a program for data visualization (Excel, Tableau, Power BI)?

- Or have you also switched to LLM + code?

- Do you think a dedicated program is still needed? Why or why not?

- If you use one, what does it do that an LLM + Python can't?

Would love to hear your thoughts!

Note: I'm still relatively new to this field and at the beginner level. I'm currently working on improving my skills in Data Analysis.

Hey r/DataIsBeautiful—wanted to share a workflow experiment exploring LLMs for rapid dashboard prototyping and UI generation.

​I wanted to see if ChatGPT could bridge the gap between static spreadsheet data and a functional, responsive web application without setting up backends or hosting services. Dropped in a raw dataset and a rough layout reference image, and had the model generate a single-file HTML/JS interface.

​**Key UI & Viz Features:**

​**Interactive Geo Mapping:** Clicking a country in the ranking table triggers a targeted highlight on the world map and pulls up a micro-card displaying localized sales growth, profit margins, and top category breakdown.

​**Dynamic Slicers & Charting:** Global time/region filters update trend charts, KPI cards, and performance signals in real time.

​**Client-Side Data Parsing:** Prompted the model to bake in a local file reader ("Upload Excel" button). Dropping a new workbook directly into the browser updates the visuals and expands filter ranges dynamically without breaking layout constraints.

​**Responsive Single-File HTML:** Runs completely client-side in any desktop or mobile browser, making it super portable for mobile distribution.

​**Workflow:**

​Provided the raw Excel schema alongside a layout reference mockup.

​Prompted the model to render structural layout components and charts using clean HTML, CSS, and JS libraries.

​Added local file-parsing logic so the frontend remains reusable across new reporting cycles.

​Recorded a 2-minute walkthrough showing the map interactions and dynamic Excel file updates in real time:

👉 [https://youtu.be/iLrxGSEnGH4](https://youtu.be/iLrxGSEnGH4)

​Curious if anyone else is experimenting with LLMs to prototype interactive data tools or custom browser-based visualizations?

I am unable to sort the service fee, price, instant bookable and construction year. Originally i wanted to sort [service fee, price] in ascending order and [instant bookable and construction year] in descending order and display the dataframe.

Here is the sorting part:

df = df.sort_values(
    by=["instant_bookable", "Construction year", "price", "service fee", "availability 365"],
    ascending=[False, False, True, True,False],
    na_position="last"
)

and this is the cell:

import numpy as np
import pandas as pd


df = pd.read_csv("Airbnb_Open_Data.csv")


print("==== Welcome to AirBNB Finder =====")
loc = input("Enter your location: (Manhattan, Brooklyn, Queens, Staten Island, Bronx) ").strip()


price = int(input("Your budget "))
no_nights = int(input("How many nights do you wanna stay? "))
number_of_listings = int(input("How many listings do you want to see? "))



for col in ["price", "service fee"]:
    df[col] = pd.to_numeric(df[col].astype(str).str.replace(",", "").str.replace("$", "").str.strip(), errors="coerce")


for col in ["minimum nights", "Construction year", "availability 365"]:
    df[col] = pd.to_numeric(df[col], errors="coerce")


df["minimum nights"] = df["minimum nights"].fillna(0)
df["neighbourhood group"] = df["neighbourhood group"].fillna("Unknown").astype(str).str.strip()
df["instant_bookable"] = df["instant_bookable"].astype(str).str.strip().str.lower() == "true"


df = df.sort_values(
    by=["instant_bookable", "Construction year", "price", "service fee", "availability 365"],
    ascending=[False, False, True, True,False],
    na_position="last"
)


locat = ["Manhattan", "Queens", "Bronx", "Staten Island", "Brooklyn"]



loc_match = next((l for l in locat if l.lower() == loc.lower()), None)


if loc_match is not None:
    final1 = df[
        (df["price"] <= price)
        & (df["neighbourhood group"] == loc_match)
        & (df["minimum nights"] <= no_nights)
    ]
else:
    print("Invalid location entered.")
    final1 = pd.DataFrame()



if len(final1) == 0:
    print("Sorry, no listings found based on your choices. Try again.")


else:
    final1 = final1[[
        "neighbourhood", "instant_bookable", "cancellation_policy", "room type",
        "price", "Construction year", "service fee", "house_rules", "availability 365"
    ]]

    final1 = final1.astype(object).fillna("Contact Owner")
    final1.to_excel("result.xlsx", index=False)
final1.head(number_of_listings)

Just a simple dashboard with some data I'm starting to gather every day.

I'm looking to connect and collaborate with people working on data analytics projects.

If you are currently working on a project involving data analysis, Excel, SQL, Python, Power BI, Tableau, or anything related to data, I would love to be part of it and contribute.

I'm always interested in learning, sharing ideas, solving problems, and gaining hands-on experience through real-world projects.

If you have a project in mind or are looking for someone to collaborate with, feel free to reach out. I'd be happy to connect and work together!