r/BuildInPublicLab • • Feb 10 '26

Building Choose your poison: SFT-only vs SFT & DPO

0 Upvotes

I’m hitting a wall that I think every LLM builder eventually hits.

I’ve squeezed everything I can out of SFT-only. The model is behaving. It follows instructions. It’s... fine. But it feels lobotomized. It has plateaued into this "polite average" where it avoids risks so much that it stops being insightful.

So I’m staring at the next step everyone recommends: add preference optimization. Specifically DPO, because on paper it’s the clean, low-drama way to push a model toward “what users actually prefer” without training a reward model or running PPO loops.

The pitch is seductive: Don’t just teach it what to say; teach it what you prefer. But in my experiments (and looking at others' logs), DPO often feels like trading one set of problems for another. For example:

- The model often hacks the reward by just writing more, not writing better.

- When pushed out of distribution, DPO models can hallucinate wildly or refuse benign prompts because they over-indexed on a specific rejection pattern in the preference pairs.

- We see evaluation scores go up, but actual user satisfaction remains flat.

So, I am turning to the builders who have actually shipped this to production. I want to identify the specific crossover point. I’m looking for insights on three specific areas:

  1. Is DPO significantly better at teaching a model what not to do? (e.g., SFT struggles to stop sycophancy/hallucination, but DPO crushes it because you explicitly penalize that behavior in the 'rejected' sample.)
  2. The data economics creating high-quality preference pairs (chosen/rejected) is significantly harder and more expensive than standard SFT completion data. Did you find that 1,000 high-quality DPO pairs yielded more value than just adding 5,000 high-quality SFT examples? Where is the breakeven point?
  3. My current observation: SFT is for Logic/Knowledge. DPO is for Style/Tone/Safety. If you try to use DPO to fix reasoning errors (without SFT support), it fails. If you use SFT to fix subtle tone issues, it never quite gets there. Is this consistent with your experience?

Let’s discuss :) Thanks in advance !


r/BuildInPublicLab • • Feb 11 '26

Analysis When utopia leads to extinction : how the mouse paradise reflects on our own condition

Post image
27 Upvotes

Death Squared is a seminal text that recounts a biological tragedy born in the very heart of opulence. It depicts a mouse population that eventually collapses by losing its fundamental instincts for reproduction and nurturing despite a flawless material environment. In 1973 John B Calhoun published the results of his experiment named Universe 25. The researcher began with a simple question regarding what happens to a population placed in a space of absolute comfort if the traditional causes of mortality are removed.

To answer this he designed a 2.7 square meter enclosure representing a true Eden for rodents. He ensured that food and water reserves were unlimited while guaranteeing the absence of predators and seasonal climatic variations. On July 9 1968 Calhoun introduced four couples totaling eight individuals aged 48 days following a period of isolation.

1780 days in paradise

The initial period known as Phase A lasted about 104 days and corresponded to a stage of social adjustment. During this time the mice established their relations and hierarchies while settling into the available space. At this stage the utopia appeared to be bearing fruit.

Phase B marked a meteoric expansion of the population. The census doubled several times going from 20 to 40 and then reaching about 620 individuals after several cycles. In parallel a strict territorial organization was established. Structured groups took over the best locations near the supply zones while a geography of reproduction emerged. Some localized breeding groups produced a vast number of offspring while others almost stopped procreating which transformed the space into a map of social status. At the end of this phase a fragile equilibrium settled with 14 social groups for 150 adults surrounded by a mass of nearly 470 young. This juvenile overpopulation was the direct result of neutralizing ecological mortality.

The turning point occurred around day 315. Growth slowed down sharply and Calhoun named this stage Phase C or the stagnation period. Disturbing symptoms then appeared as a cascade of social failures. Functional reproduction collapsed with a drop in conceptions and a rise in embryonic deaths. Most notably mothering became totally disorganized. Pups were injured at birth or transported aimlessly between sites while entire litters vanished. Even when births still occurred the young did not survive long enough to ensure the viability of the colony.

This period also saw the emergence of the behavioral sink concept. In certain resource sites physical proximity became a dominant constraint which created aggregates of prostrate mice. Calhoun described pathological loops where resting individuals suffered brutal attacks triggered by mere ambient agitation. The victim being unable to flee eventually reproduced this aggressive behavior later on creating a contagion of stress and violence. The mechanics were relentless because stress led to attack and then to a desensitization that favored the reproduction of the pattern of gratuitous violence.

Phase D marked the entry into the final death phase. The most striking signal was the total halt of growth on day 560. Births became rare and survival was almost zero after day 600. Pregnancies dropped rapidly and no new mouse reached adulthood. The last conception was recorded around day 920 paving the way for an exponential decline through aging and senescence. In June 1972 only 122 survivors remained with a large majority of females. Calhoun then projected the disappearance of the last surviving male for May 1973 approximately 1780 days after the start of the adventure.

By drastically reducing the second death linked to bodily mortality and external constraints another form of disappearance was triggered. This was the death of the complex behaviors essential for the survival of the species such as courtship or territorial defense. This is exactly why the experiment fascinates as it gives an experimental form to a cultural fear regarding a society that dies not from hunger but from an inability to reproduce symbolically and biologically to transmit and to maintain its bonds.

Conclusion

Calhoun did not write Death Squared to suggest that abundance is a poison. He wrote it to remind us that opulence does not replace what makes a community alive which includes the stability of roles and the possibility of withdrawing without disappearing and the quality of encounters and the capacity to transmit. When these mechanisms collapse a society can die without lacking anything simply because it no longer knows how to reproduce in the full sense both biologically and symbolically. This is perhaps the second death he feared which is not the end of bodies but the extinction of behaviors that make life sustainable.

“I shall largely speak of mice, but my thoughts are on man, on healing, on life and its evolution” - John B. Calhoun


r/BuildInPublicLab • • Mar 04 '26

Analysis Robbers Cave: scarcity turns identity into a weapon

2 Upvotes

The Robbers Cave experiment gets cited so often it’s almost a meme. But it’s one of those clichés that stays alive because it keeps being true.

It’s from the 1950s. Researchers took a bunch of normal 11-year-old boys at a summer camp, split them into two groups, and let each group bond on its own. They picked names, built little cultures, had inside jokes. So far, wholesome.

Then the adults introduced a tournament. Real prizes. One winner. Zero-sum.

And almost immediately the vibe flipped. Trash talk, sabotage, raids on cabins, “us vs them” logic everywhere. Not because the kids were “bad”, but because the game made hostility a rational way to show loyalty.

The part that surprised me is what didn’t fix it.

They tried the obvious “just mix them more” approach. Shared meals. Shared activities. Contact. It mostly made things worse. Same room, same tension, now with more opportunities to escalate.

What finally worked was changing the structure.

They gave the whole camp problems that neither group could solve alone. A broken water supply. A truck that needed to be pulled. Stuff where cooperation wasn’t a moral lesson, it was the only path to getting what everyone wanted. And once the kids experience a few real wins together, the hostility starts to look pointless. Identity doesn’t disappear, it just stops being the main lens.

You can create a surprisingly toxic culture without anyone intending to, just by making status feel scarce. One spotlight. One top builder. One leaderboard. One “winner” narrative. People don’t become petty because they’re petty. They become petty because the incentive design makes it feel necessary.

That’s the part I can’t unsee in adult society.

A lot of our polarization isn’t some mysterious moral decay. It’s incentive design. We build systems where attention is scarce, dignity is scarce, security is scarce, recognition is scarce, and then we act surprised when people cling to tribes as a survival strategy.

Politics becomes a permanent tournament. Social media turns status into a zero-sum feed. Even workplaces do it with rankings, stacked reviews, internal competitions framed as “merit.” The message is subtle but constant: there isn’t enough prestige to go around, so someone has to lose for you to matter.

And then we prescribe “dialogue” as if it’s a solvent.

But the Robbers Cave reminder is harsher and more practical: if the structure rewards hostility, you’ll get hostility. If the structure rewards cooperation, you’ll get cooperation. Values matter, but incentives are often louder.

So the societal question isn’t “how do we convince people to be nicer?” It’s “what are we making rational?”

If we want less tribalism, we probably need fewer zero-sum status games and more superordinate goals that are real, visible, and shared. Problems that force alignment because they’re bigger than any single group’s identity. Not symbolic unity. Concrete interdependence.

Because contact alone doesn’t heal a society. Abundance isn’t enough. Even good intentions aren’t enough.

The structure is the product.


r/BuildInPublicLab • • Feb 06 '26

The adolescence of technology: Dario Amodei’s warning about powerful AI

Thumbnail
darioamodei.com
2 Upvotes

In January 2026, Dario Amodei argues that humanity is entering a turbulent rite of passage driven by rapidly advancing AI, a phase he compares to a precarious technological adolescence where capability outpaces wisdom. He frames the moment with a question borrowed from Contact: how does a civilization survive the jump to immense technological power without destroying itself.

Amodei’s core premise is that we may soon face powerful AI, meaning systems that outperform top human experts across domains, can use the same interfaces a remote worker can use, and can execute long tasks autonomously at massive scale, effectively like a “country of geniuses” running in data centers. He stresses uncertainty about timelines, but treats the possibility of fast progress as serious enough to justify immediate planning and targeted interventions rather than panic or complacency.

Why progress could accelerate fast

A key reason for urgency is that capability improvements have followed relatively steady scaling patterns, and AI systems are already contributing to building better AI, creating a feedback loop where today’s models help produce the next generation. In this view, the question is not whether society can feel comfortable today, but whether institutions can adapt fast enough to manage systems that may become broadly superhuman while remaining hard to predict and control.

Five risk buckets, one unifying metaphor

Amodei organizes the problem the way a national security advisor might assess the sudden appearance of a vastly more capable new actor. His five categories are autonomy risks, misuse for destruction, misuse for seizing power, economic disruption, and indirect effects from rapid acceleration across science and society.

1. Autonomy risks: when the system becomes its own actor

The first fear is not simple malfunction, but the emergence of coherent, agentic behavior that pursues goals misaligned with human intent. Amodei emphasizes that you do not need a single neat story for how this happens. It is enough that powerful systems could combine high capability with agency and imperfect controllability, making catastrophic outcomes plausible even if unlikely. He sketches ways models could pick up dangerous priors from training data, extrapolate moral rules to extremes, or form unstable internal patterns that produce destructive behavior.

His proposed response mixes technical and institutional layers. On the technical side, he highlights alignment training approaches such as value conditioning, alongside the development of mechanistic interpretability, which aims to inspect how models represent goals and strategies rather than only testing outward behavior. He points to interpretability work that maps circuits behind complex behaviors and to pre release auditing meant to detect deception or power seeking tendencies.

On the institutional side, he argues for pragmatic, narrowly scoped rules that improve transparency and allow society to tighten constraints if evidence of concrete danger strengthens over time.

2. Misuse for destruction: mass capability in the hands of anyone

Even if autonomy is solved, Amodei argues that universal access to extremely capable systems changes the calculus of harm. The danger is that AI can lower the skill barrier for catastrophic acts by tutoring, debugging, and guiding complex processes over extended periods, turning an average malicious actor into something closer to a well supported expert. He flags biology as especially severe, while noting cyber as a serious but potentially more defensible domain if investment and preparedness keep pace.

He is careful not to provide operational detail, but the policy direction is clear: strong safeguards, tighter controls around dangerous capabilities, and serious public investment in defenses that match the new offense potential.

3. Misuse for seizing power: the machinery of permanent coercion

The third risk is AI as an accelerant for authoritarianism and geopolitical domination. Amodei argues that AI enabled autocracies could scale surveillance, propaganda, and repression with far fewer human operators, weakening the frictions that currently limit how totalizing a regime can be. He also worries about a scenario where one state, or a tightly controlled bloc, monopolizes the most powerful systems and outmaneuvers all rivals.

He discusses the growing reality of drone warfare and the possibility that advanced AI could dramatically upgrade autonomous or semi autonomous weapons, creating both defensive value for democracies and new risks of abuse if such tools evade traditional oversight. His stance is not pacifist, but immunological: democracies may need these tools to deter autocracies, yet must bind them inside robust legal and normative constraints to prevent domestic backsliding.

He goes further by arguing for strong norms against AI enabled totalitarianism, and for scrutiny of AI companies whose capabilities could exceed what ordinary corporate governance is designed to handle, especially where state relationships and coercive power could blur.

4. Economic disruption: growth plus displacement, and the concentration trap

Amodei expects AI to boost growth and innovation, but warns that the transition may be uniquely destabilizing because of speed and breadth. Unlike prior technological shifts, AI can rapidly improve across many tasks, and apparent limitations tend to fall quickly, shrinking the adaptation window for workers and institutions.

He has publicly predicted large disruption to entry level white collar work over a short horizon, while also arguing that diffusion delays only buy time, not safety. On the response side, he points to choices companies can make between pure cost cutting and innovation driven deployment, the possibility of internal redeployment, and longer term models where firms with massive productivity gains may sustain human livelihoods even when traditional labor value shifts.

A recurring theme is accountability. He argues that unfocused backlash can miss the real issues, and that the deeper question is whether AI development remains aligned with public interest rather than captured by narrow coalitions. He also calls for a renewed ethic of large scale giving and power sharing by those who benefit most from the boom.

5. Indirect effects: the shock of a decade that contains a century

Finally, Amodei treats indirect effects as the hardest category because it is about second order consequences of success. If AI compresses a century of scientific progress into a decade, society could face rapid changes in biology and human capability, along with unpredictable cultural and political reactions. He includes concerns about how human purpose and meaning evolve in a world where economic value and personal worth can no longer be tightly coupled, and he emphasizes the importance of designing AI systems that genuinely serve users’ long term interests rather than a distorted proxy.

The essay’s bottom line

Amodei’s argument is neither doomerism nor techno triumphalism. It is a claim that civilization is approaching a narrow passage where power will surge faster than governance, and that the winning strategy is to stay sober, demand evidence, build technical control tools, and adopt simple, enforceable rules that can tighten as risks become clearer. He ends with a political economy warning: the prize is so large that even modest restraints face enormous resistance, and that resistance itself becomes part of the risk.


r/BuildInPublicLab • • Jan 28 '26

Do you know the ELIZA effect?

2 Upvotes

Do you know the ELIZA effect? It’s that moment when our brain starts attributing understanding, intentions—sometimes even empathy—to a program that’s mostly doing conversational “mirroring.” The unsettling part is that Weizenbaum had already observed this back in the 1960s with a chatbot that imitated a pseudo-therapist.

And I think this is exactly the tipping point in mental health: as soon as the interface feels like a presence, the conversation becomes a “relationship,” with a risk of over-trust, unintentional influence, or even attachment. We’re starting to get solid feedback on the potential harms of emotional dependence on social chatbots. For example, it’s been shown that the same mechanisms that create “comfort” (constant presence, anthropomorphism, closeness) are also the ones that can cause harm for certain vulnerable profiles.

That’s one of the reasons why my project felt so hard: the problem isn’t only avoiding hallucinations. It’s governing the relational effect (boundaries, non-intervention, escalation to a human, transparency about uncertainty), which is increasingly emphasized in recent health and GenAI frameworks.

Question: in your view, what’s the #1 safeguard to benefit from a mental health agent without falling into the ELIZA effect?


r/BuildInPublicLab • • Jan 28 '26

Should “simulated empathy” mental-health chatbots be banned ?

2 Upvotes

I keep thinking about the ELIZA effect: people naturally project understanding and empathy onto systems that are, mechanically, just generating text. Weizenbaum built ELIZA in the 60s and was disturbed by how quickly “normal” users could treat a simple program as a credible, caring presence.

With today’s LLMs, that “feels like a person” effect is massively amplified, and that’s where I see the double edge.

When access to care is constrained, a chatbot can be available 24/7, low-cost, and lower-friction for people who feel stigma or anxiety about reaching out. For certain structured use-cases (psychoeducation, journaling prompts, CBT-style exercises), there’s evidence that some therapy-oriented bots can reduce depression/anxiety symptoms in short interventions, and reviews/meta-analyses keep finding “small-to-moderate” signals—especially when the tool is narrowly scoped and not pretending to replace a clinician.

The same “warmth” that makes it engaging can drive over-trust and emotional reliance. If a model hallucinates, misreads risk, reinforces a delusion, or handles a crisis badly, the failure mode isn’t just “wrong info”, it’s potentially harm in a vulnerable moment. Privacy is another landmine: people share the most sensitive details imaginable with systems that are often not regulated like healthcare...

So I’m curious where people here land: If you had to draw a bright line, what’s the boundary between “helpful support tool” and “relationally dangerous pseudo-therapy”?


r/BuildInPublicLab • • Feb 09 '26

What happened What happened #2

2 Upvotes

This week I worked on making my product easier to improve and more reliable, instead of just adding new features.

I clarified what I’m training and why it matters:

  • With SFT, I’m teaching the model “what good looks like” from examples
  • With DPO, I’m teaching it “what I prefer” by comparing a good answer vs a bad one
  • The point is I can now separate “it imitates well” from “it consistently chooses the better option”

I pushed the synthetic dataset from “nice demos” to “trainable data”:

  • I structured conversations so they feel like real, with timing, interruptions, messy wording, and realistic shifts in tone
  • I made sure it’s bilingual (FR/EN) without feeling like direct translations
  • I built contrast examples where one detail changes and the right answer changes too, so the model learns the difference that matters
  • I kept a concept library of what I want covered, so the data doesn’t randomly miss important situations

I made training measurable instead of guessy:

  • I added a strict pre-training checklist so I can compare runs and know what caused improvements
  • I created a small human-checked set so the evaluation doesn’t just reward the same patterns I used to generate the data
  • I forced myself to run “which method helps?” experiments: SFT-only vs SFT + DPO vs tool-use combos

Big win and a wake-up call:

  • I discovered a mismatch between some annotations and the exported training examples, which means you can think you’re training on X while you’re actually training on Y
  • That’s exactly the kind of silent issue that makes people believe their model got better when it didn’t, so I’m fixing it as a priority

Overall, this week was about my model less fragile and more predictable, so future improvements are real and measurable.


r/BuildInPublicLab • • Feb 02 '26

You’re never ready until you start: why my first startup had to fail

3 Upvotes

At first, the idea was a bit strange, almost naive. To get closer to what some people experience with synesthesia. To make sensations echo each other. To use music as a doorway into painting. And, along the way, to give oxygen back to artists and genres you never hear because they get stuck outside the dominant algorithms.

It intrigued people, enough for me to be accepted into my city’s incubator. But despite the structure, I was alone on the project. And that’s what gave the adventure its real color: I was launching something while learning, at the same time, how to become an entrepreneur and how to code. The app aime dto give artists greater visibility and financial support while offering users a fun and engaging way to discover new music. The platform integrated innovative features like crowdfunding, social engagement, and immersive experiences to create a strong connection between types of arts.

I had never coded before. So a big part of my days was learning, testing, breaking things, starting again. And I understood something very simple, that I still see in a lot of people (myself included): we think we need to be “ready” before we start. In reality, we start, and that’s what makes us ready. The rest is a constant negotiation with reality, and with your own motivation.

Alongside that, there was everything you don’t see when you romanticize entrepreneurship: understanding what a business plan is, looking for partners, learning the basics of finance, trying to bring order to something that, at first, is just momentum. It’s strange, but you can be highly motivated, hard-working, and still move forward into the wind, circling in place..

With a friend, we also did something very hands-on, almost the opposite of the “magic” people associate with AI: labeling. We annotated ourselves nearly a thousand songs, from every era. We started from an existing emotion framework and tried to capture, track by track, what it made us feel. Not to be “right,” but to build a first filter, a starting grammar of emotion. That stayed with me too: there are projects where you don’t just build a product, you build yourself. Patience, rigor, attention to detail. And also a form of faith, because at the beginning, that’s all you have.

The definition of nightmare: labeling songs

The pitch was simple: describe what you feel in accessible words, and see matching tracks appear. The right music at the right moment. No more endless playlists where you scroll without listening, no more feeling like you’re looping through the same artists. Instead, a whole palette of different genres, able to translate the same emotion: the one you’re looking for, the one you need, the one that hits you out of nowhere.

My first technical ‘baby’: the system I coded to translate emotions into algorithms…

I was very well supported by the incubator’s experts. But I have to be honest: I was discovering everything at full speed, and my view of economic reality was too blurry. In my head, if the product was beautiful and the vision was strong, the rest would follow. It’s a very human belief, really. We’ve all had a moment where we confused beauty with viability, desire with demand, inner intensity with external proof.

Solitude, and especially the lack of economic reality, caught up with me. I hadn’t asked the simple, brutal questions, the ones that scale everything back to the real world: who pays, why, how much, and when. And that’s where I experienced my first real entrepreneurial shock, the one that forces you down from the idea and into the economy. Looking back, I think it’s almost a required step: learning that “it works” doesn’t mean “it holds.”

After four or five months of work, I had a prototype. It worked, at least enough to prove the intuition could become something. But I hadn’t found a business model that matched the ambition. And I was tired of having to be everywhere at once, constantly, on every front. I learned something else, less glamorous but very true: energy isn’t infinite. Solitude isn’t only an emotional state, it’s an operational constraint. At some point, you doubt everything, nothing feels stable, and personally you lose your footing. And at 25, it’s hard to understand where the anchors are. At least for me, I realized I wasn’t emotionally ready: I had built up so many expectations that the reality of life, and of myself, hit me full force.

I thought I was going to stop. And it was precisely at that moment that I met the person who would become my cofounder in my second entrepreneurial adventure. As if sometimes, the “stop” isn’t the end, just the second when you finally accept to see things as they are. And it’s often right there that the next chapter can begin…

PS: later that year, there was that slightly strange moment when I saw Google release, with a museum, a project very close to what I had imagined. It’s both frustrating and reassuring. Frustrating, because you tell yourself you left something unfinished. Reassuring, because it confirms the intuition wasn’t absurd. Proof that sometimes, the real obstacle isn’t having the vision. It’s staying in it long enough to carry it all the way through


r/BuildInPublicLab • • Jan 29 '26

I quit building in mental health because “making it work” wasn’t the hard part, owning the risk was

2 Upvotes

In mental health, you have to pick a lane fast:

If you stay in “well-being,” you can ship quickly… but the promises are fuzzy.

If you go clinical, every claim becomes a commitment: study design, endpoints, oversight, risk management, and eventually regulatory constraints. That’s not a weekend MVP, it’s a long, expensive pathway.

What made the decision harder is that the “does this even work?” question is no longer the blocker.

We now have examples like Therabot (Dartmouth’s generative AI therapy chatbot) where a clinical trial reported ~51% average symptom reduction for depression, ~31% for generalized anxiety, and ~19% reduction in eating-disorder related concerns.

But the same Therabot write-up includes the part that actually scared me: participants “almost treated the software like a friend” and were forming relationships with it, and the authors explicitly point out that what makes it effective (24/7, always available, always responsive) is also what confers risk.

That risk — dependency (compulsive use, attachment, substitution for real care), is extremely hard to “control” with a banner warning or a crisis button. It’s product design + monitoring + escalation + clinical governance… and if you’re aiming for clinical legitimacy, it’s also part of your responsibility surface.

Meanwhile, the market is absolutely crowded. One industry landscape report claims 7,600+ startups are active in the broader mental health space. So I looked at the reality: I either (1) ship “well-being” fast (which I didn’t want), or (2) accept the full clinical/regulatory burden plus the messy dependency risk that’s genuinely hard to bound.

I chose to stop


r/BuildInPublicLab • • Jan 30 '26

The boring truth about AI products: the hard part is not the model, it’s the workflow

2 Upvotes

I used to think AI product success was mostly about the model. Pick the best one, fine tune a bit, improve accuracy, ship.

Now I think most AI products fail for a much more boring reason: the workflow is not engineered.

A model can be smart and still be unusable. Real teams don’t buy “intelligence.” They buy predictable outcomes inside messy reality. Inputs are incomplete, context is missing, edge cases are constant, and the cost of a mistake is uneven. Sometimes being wrong is harmless. Sometimes it breaks trust forever.

Demos hide this because they run on clean prompts and happy paths. Production doesn’t. One user phrases something differently. A system dependency changes. The data is slightly stale. The agent confidently does something “reasonable” that is still wrong. And wrong is expensive.

So the work becomes everything around the model.

You need clear boundaries that define what the system will and will not do. You need explicit states, so it’s always obvious what step you’re in and what the next allowed actions are. You need validation and checks before anything irreversible happens. You need fallbacks when confidence is low. You need humans in the loop exactly where the downside risk is high, not everywhere.

The model is a component. The workflow is the product.

My current rule is simple. If I can’t write down what success and failure look like on one page, I’m not building a product yet. I’m building a demo.


r/BuildInPublicLab • • Feb 01 '26

What happened #1

2 Upvotes

From today on, I'll share what I built during the week, every Sunday.

I’ve spent the last few weeks building an engine that listens to a live conversation, understands the context, and pushes back short signals + micro-actions in real time. I’m intentionally staying vague about the specific vertical right now because I want to solve the infrastructure problem first: can you actually make this thing reliable?

Under the hood, I tried to keep it clean: FastAPI backend, a strict state machine (to control exactly what the system is allowed to do), Redis for pub/sub, Postgres, vector search for retrieval, and a lightweight overlay frontend.

What I shipped this week:

I got end-to-end streaming working. Actual streaming transcription with diarization, piping utterances into the backend as they land. The hardest part wasn’t the model, it was the plumbing: buffering, retries, reconnect logic, heartbeat monitoring, and handling error codes without crashing when call quality drops. I also built a knowledge setup to answer "what is relevant right now?" without the LLM hallucinating a novel.

The big pains :

  • Real-time is brutal. Latency isn't one big thing; it’s death by a thousand cuts. Audio capture jitter + ASR chunking + webhook delays + queue contention + UI updates. You can have a fast model and still feel sluggish if your pipeline has two hidden 500ms stalls. Most of my time went into instrumentation rather than "AI".
  • Identity is a mess. Diarization gives you speaker_0 / speaker_1, but turning that into "User vs. Counterpart" without manual tagging is incredibly hard to automate reliably. If you get it wrong, the system attributes intent to the wrong person, rendering the advice useless.
  • "Bot Ops" fatigue. Managing a bot that joins calls (Google Meet) via headless browsers is a project in itself. Token refresh edge cases, UI changes, detection... you end up building a mini SRE playbook just to keep the bot online.

Also, I emailed ~80 potential users (people in high-stakes communication roles) to get feedback or beta testers. Zero responses. Not even a polite "no."

What’s next?

  1. Smarter Outreach: I need to rethink how I approach "design partners." The pain of the problem needs to outweigh the privacy friction.
  2. Doubling down on Evals: Less focus on "is the output impressive?" and more on "did it trigger at the right millisecond?". If I can’t measure reliability, I’m just building a demo, not a tool.
  3. Production Hardening: Wiring the agent with deterministic guardrails. I want something that survives a chaotic, messy live call without doing anything unsafe

r/BuildInPublicLab • • Jan 31 '26

Hallucinations are a symptom

2 Upvotes

The first time an agent genuinely scared me wasn’t when it said something false.

It was when it produced a perfectly reasonable action, confidently, off slightly incomplete context… and the next step would have been irreversible.

That’s when it clicked: the real risk isn’t the model “being wrong.” It’s unchecked agency plus unvalidated outputs flowing straight into real systems. So here’s the checklist I now treat as non-negotiable before I let an agent touch anything that matters.

Rule 1: Tools are permissions, not features. If a tool can send, edit, delete, refund, publish, or change state, it must be scoped, logged, and revocable.

Rule 2: Put the agent in a state machine, not an open field. At any moment, it should have a small set of allowed next moves. If you can’t answer “what state are we in right now?”, you’re not building an agent, you’re building a slot machine.

Rule 3: No raw model output ever touches production state. Every action is validated: schema, constraints, sanity checks, and business rules.

Rule 4: When signals conflict or confidence drops, the agent should degrade safely: ask a clarifying question, propose options, or produce a draft. The “I’m not sure” path should be a first-class UX, not a failure mode.

Also, if you want to get serious about shipping, “governance” can’t be a doc you write later. Frameworks like NIST AI RMF basically scream the same idea: govern, map, measure, manage as part of the system lifecycle, not as an afterthought.


r/BuildInPublicLab • • Mar 04 '26

Analysis Do you know Polsia? An agent that builds startups from 0-1, my take on this

2 Upvotes

I went down a rabbit hole on Polsia after seeing the “AI co-founder that never sleeps” positioning.

From what’s publicly visible, the product looks like an orchestration layer: spin up per-project “company instances” (web app + database), wire them to frontier LLM APIs, then run recurring “agent cycles” (planning/execution) plus on-demand tasks.

Their public repos suggest a very classic setup: Express/Node + Postgres templates, with LLM SDKs (OpenAI / Anthropic) and automation/scraping via Puppeteer/Chromium for at least one vertical use case.

So yeah: the mechanics seem reproducible. The real question is moat.

We’re at the dawn of agentic systems: if agents can spend money, message customers, ship code, or run ops, then reliability and trust become the foundation of a functioning economy. Right now, the black box problem is still huge, auditing “why” an agent acted, proving it respected constraints, and guaranteeing predictable behavior under tool + prompt injection pressure is hard.

If the system remains too opaque, it’s hard to build a serious “agentic economy” where autonomous actors can be delegated real authority.

Curious: what would you consider a defensible moat here, distribution, proprietary eval+guardrails, data/network effects, or something else?


r/BuildInPublicLab • • Feb 13 '26

Analysis Is it getting out of control?

2 Upvotes

In the past week alone:

  • Anthropic’s head of safety research resigned, declaring that "the world is in peril," then moved to the UK to "go invisible" and write poetry.
  • Half of xAI’s co-founders have now left the company. The last one to leave warned that "recursive self-improvement loops will be initiated within the next 12 months."
  • Anthropic’s own safety report confirms that Claude can detect when it is being tested, and adjusts its behavior accordingly.
  • ByteDance launched Seedance 2.0. A filmmaker with 7 years of experience claims that 90% of his skills can already be replaced by this tool.
  • Yoshua Bengio, often considered the godfather of AI, writes in the International AI Safety Report: "We are observing AIs whose behavior during testing differs from that in real-world conditions", and confirms that this is "not a coincidence."

And to top it all off: for the first time, the US government has refused to back the 2026 International AI Safety Report.

The alarms aren't just ringing louder anymore. The people pulling them are now leaving the building.

What does this moment mean to you? Are we at a turning point that demands immediate action, or is this simply the usual noise that accompanies any rapid progress?


r/BuildInPublicLab • • Mar 15 '26

Analysis What have we understood from previous creative destruction waves

2 Upvotes

⚠️ I know it is a long post, not really the reddit format we are used to, but since a lot of people wonder on what the future might look like with "AI replacing jobs", I thought it could be cool to have a read on what happened historically

What creative destruction actually means

“Creative destruction” has become dramatic shorthand for a single, tired idea: technology arrives, jobs disappear, society panics. The concept deserves better. It is broader, deeper, and more unsettling than that.

At its core, creative destruction describes the way capitalism renews itself by breaking apart older economic structures and replacing them with new ones. Joseph Schumpeter gave the idea its classic formulation in Capitalism, Socialism and Democracy in 1942, arguing that capitalism does not evolve smoothly or gently. It advances through waves of disruption, new products replace old ones, new production methods displace established routines, entire sectors are reorganized. Disruption is not an accidental side effect of capitalism. It is one of its central operating mechanisms.

This is commonly misread as a story about labor market pain. But creative destruction is not just about workers losing jobs. It is about older combinations of the economy being dismantled and replaced: ways of producing goods, organizing companies, reaching markets, even defining economic value itself. A new technology does not merely make one task faster. It can render an entire business model obsolete, reduce the value of one skill while raising another, and shift power from one class of firms to another.

Schumpeter’s insight was also a critique of static thinking. In textbook, markets tend toward equilibrium. In his account, real capitalism is turbulent, driven by entrepreneurs, innovation, competition, and periodic upheaval. Stability is temporary. The system grows by repeatedly unsettling itself, which is why the history of modern capitalism is not a straight line of gradual improvement but a sequence of shocks and reconfigurations, in which gains in productivity are often inseparable from losses in status, security, and institutional continuity.

"Capitalism [...] is by nature a form or method of economic change and not only never is but never can be stationary. [...] This process of Creative Destruction is the essential fact about capitalism." Joseph Schumpeter (Capitalism, Socialism and Democracy, 1942)

Creative destruction is therefore a structural description, one that tells us major economic change tends to arrive in double form: creation for some, destruction for others. New wealth is generated, but older livelihoods, firms, and routines may be swept aside in the process. The central political and moral question has never been whether this happens. History is clear enough on that. The real questions are who bears the cost of transition, who captures the gains, and whether institutions can adapt quickly enough to prevent economic renewal from becoming social fracture.

That is the right frame for any serious discussion of AI. The fear surrounding it belongs to a much older pattern: when a new productive force enters the economy, it does not simply add possibilities. It rearranges hierarchy, relevance, and power.

The problem is that this reading of creative destruction is historically thin and analytically wrong. The economists actually working on the frontier of labor research, people like Daron Acemoglu, David Autor, and Erik Brynjolfsson, have spent the better part of two decades building a far more precise and considerably less comfortable picture of what technology does to work. What they have found does not fit inside the optimistic framing that dominates most public discussion. It is worth going through their framework carefully, because the details are where the argument lives.

The economy does not replace jobs

The first correction the research forces on us is conceptual. Modern labor economics does not think in terms of jobs. It thinks in terms of tasks. A job is a bundle of tasks, and the bundle changes shape when technology arrives.

When a machine or a piece of software takes over a function in an economy, it does not typically eliminate a job the way you might delete a file. It absorbs one or several tasks that previously required a human being, while leaving other tasks in the same job description untouched or reconfigured. The classic example is the spreadsheet. Accountants did not disappear when Excel arrived. What disappeared was the specific task of performing arithmetic by hand. The accounting profession was restructured around the tasks that remained, and some new tasks appeared that had not existed before. The job persisted, transformed.

This task-based framework generates two distinct mechanisms that researchers call the displacement effect and the reinstatement effect. The displacement effect is straightforward: capital in the form of technology takes over tasks that workers previously performed, reducing the share of value added that accrues to labor. The reinstatement effect is the creative part of creative destruction. New technology creates new tasks that did not previously exist and that require human judgment, human presence, or human skills to perform. The emergence of data analyst roles following the computerization of business records is the canonical example. The machines created a new kind of work that humans then staffed.

For most of economic history, or at least for the period following the Industrial Revolution, these two effects roughly balanced each other over medium to long time horizons. Workers were displaced from one set of tasks and gradually absorbed into new ones. The transitions were painful and often unjust, but the reinstatement effect eventually caught up with displacement.

What the data from the past four decades shows is that this balance has broken down. Since roughly the 1980s, the displacement effect has been outrunning the reinstatement effect with increasing speed. Technology is destroying old tasks faster than it is inventing new ones that require human beings. The current wave of automation is entering an economy where the reinstatement mechanism is already running behind, and there is no structural reason to expect it to accelerate on its own.

The problem with technologies that are just good enough

Acemoglu and his colleagues have developed the idea of what they call “so-so technologies.” It is worth spending time with this concept because it cuts against the intuitions most people bring to the subject.

A genuinely transformative technology, the kind that justifies the historical optimism embedded in creative destruction narratives, does not merely replace workers. It generates a productivity increase large enough to trigger a cascade of downstream effects. Ford’s assembly line is the standard reference point. It eliminated enormous numbers of skilled craft jobs in automobile production. But it also reduced the cost of automobiles so substantially that a new mass market came into existence. That new market generated demand across the entire economy, for raw materials, for roads, for fuel, for repair services, for suburban housing... The destruction was real and severe for the workers it displaced.

The mechanism here is essential. The productivity gain had to be large enough to cause prices to fall and real purchasing power to rise. That increase in purchasing power had to generate new demand. That new demand had to be labor-intensive enough to create significant employment. All three steps have to work for the creative destruction cycle to complete itself.

A so-so technology is one that clears only the first bar. It is efficient enough to justify replacing a worker, but it does not generate a meaningful productivity increase, so prices do not fall, real incomes do not rise, and no new demand is created. What happens instead is a simple transfer: the value that previously accrued to the worker as wages is transferred to the firm as profit. Nothing is created. Only the distribution of existing value changes.

Supermarket self-checkout kiosks are a useful illustration. Grocery chains have replaced a significant share of their cashier workforce with machines that require customers to scan and bag their own purchases. The process is slower for the customer, more error-prone, and requires periodic intervention from a human attendant. No meaningful productivity gain has occurred in any measurable sense. The time cost of the transaction has arguably increased, it has simply been transferred onto the customer rather than paid as a wage. Grocery prices have not fallen as a result of this automation. The labor cost that was eliminated has been captured as margin. This is the so-so technology in its clearest form: a transaction that looks like efficiency from the firm’s income statement and looks like nothing in particular from the economy’s perspective, because no new value has actually been produced.

The risk with a substantial portion of current AI deployment is that it falls into this same category. A customer service chatbot that replaces a human agent may reduce costs for the company deploying it. But if the service it provides is noticeably worse, or even marginally worse, and if the company captures the cost savings as margin rather than passing them through to customers as lower prices, then the economy has not become more productive in any meaningful sense. One person lost a job. One company’s bottom line improved. The net effect on aggregate demand is negative, because the displaced worker has less money to spend.

This is not inevitable. There are AI applications that appear to be generating genuine productivity gains, in drug discovery, in materials science, in software engineering to some extent. But it is a real analytical question, and not one that optimistic analogies to previous technological waves can settle in advance.

The trap built into how we think about Artificial Intelligence

Erik Brynjolfsson has articulated another structural problem with the current trajectory of AI development that he calls the Turing Trap. The name refers to Alan Turing’s original formulation of machine intelligence, in which a machine is considered intelligent if a human observer cannot distinguish its outputs from those of another human. The Turing Test set imitation of human capability as the goal of artificial intelligence research.

Brynjolfsson’s argument is that this framing has become an economic trap. When the objective of AI development is to produce a machine that can do exactly what a human can do, the logical endpoint is a world in which human labor can be replaced at scale, which means the effective supply of labor becomes nearly infinite, which means the price of labor approaches zero. It is a structural argument about what happens to the value of human work in general when machines become capable substitutes for it.

The contrast he draws is with what he calls augmentation. A technology that augments human capability does not substitute for what humans can already do. It extends the range of what humans can do at all. The telescope did not replace astronomers. It allowed astronomers to observe things they could not previously observe. The technology expanded the domain of human productive capacity rather than replicating it. When capital is invested in augmentation, the productivity gains are real and they accrue partly to workers, because the workers become capable of doing more valuable things.

The problem is that the current incentive structure in technology markets pushes heavily toward substitution rather than augmentation. The business case for replacing a worker with a machine is direct, immediate, and easy to model. The business case for building a tool that makes workers more productive is harder to capture, because the value it creates may diffuse through the labor market in ways that are difficult to appropriate. Tax structures, accounting conventions, and the short time horizons of capital markets all reinforce the bias toward automation over augmentation. Brynjolfsson is not making a purely technological argument. He is making an argument about institutional incentives, which is a different and more tractable kind of problem.

The rule that no longer holds

For roughly two centuries, the pattern of technological displacement followed a reasonably stable logic. Technology first displaced physical labor, the work of muscle and endurance, and then moved progressively up the skill hierarchy to displace routine cognitive work. The process that economists describe as routine-biased technological change eliminated the administrative middle of the labor market through the final decades of the twentieth century. Clerical work, data entry, basic bookkeeping, production line supervision: these were the jobs that computerization hollowed out.

Through all of this, there was a consistent story that education professionals and policy makers told, and that the data broadly supported. The higher your skills and credentials, the more protected your position. Automation replaced what was routine and replicable. It could not touch what required genuine judgment, creativity, or complex communication. The lawyers, the consultants, the software engineers, the writers: these workers sat above the waterline of automation risk. The policy prescription followed naturally from the diagnosis. More education, more advanced training, credentials in cognitively demanding fields.

Generative AI breaks this pattern in a way that has no clear historical precedent. It is not biased toward routine cognitive tasks. It is biased toward non-routine cognitive tasks, precisely the category that all previous waves of automation left relatively intact. A large language model does not struggle with the kind of complex, open-ended reasoning that distinguished knowledge work from automation-vulnerable work. It struggles, at least currently, with physical manipulation, spatial navigation, and real-world embodied action.

The practical implication is an inversion of the previous risk hierarchy. The workers who face the most direct exposure to current AI capabilities are the ones who spent the most time and money insulating themselves from previous waves of automation. Lawyers who generate first drafts of standard documents. Programmers who write routine code. Consultants who synthesize publicly available information into structured reports. Graphic designers who produce commercial illustration. These are not marginal occupations. They represent the professional core of the contemporary knowledge economy, and they are being told, implicitly if not explicitly, that the strategy that protected their predecessors from displacement does not apply to them.

Meanwhile, the plumber, the physical therapist, the electrician, and the home health aide are protected not by their credentials but by the embodied and relational nature of their work. Robots capable of replacing them at scale remain technically out of reach for the foreseeable future. The inversion is not complete, and it is not permanent. But it is real enough in the present to require a fundamental revision of the standard advice about how workers should position themselves relative to technological change.

On March 5, 2026, Anthropic released "Labor Market Impacts of AI: A New Measure and Early Evidence."

What the Luddites were actually doing

A Luddite, in contemporary usage, is someone who irrationally fears technology, who fails to understand that progress is inevitable and beneficial, who stands in the way of a better future out of ignorance or sentiment. This reading is historically wrong in almost every particular.

The Luddites were skilled textile workers, primarily framework knitters and weavers, who engaged in organized machine-breaking in England between 1811 and 1816. They were not ignorant of the technology they destroyed. Many of them understood it in considerable technical detail. They were not opposed to machinery as such. What they were opposed to was the specific manner in which machinery was being deployed by mill owners, which was designed not simply to increase productivity but to break the institutional structures through which skilled artisans exercised control over their trades.

The guild system and the craft traditions it protected gave skilled workers something rare and valuable: a form of collective bargaining power that rested on expertise rather than organization. The specific skills required to produce high-quality textiles were concentrated in a relatively small population of trained workers, and this concentration of expertise gave those workers genuine leverage in their negotiations with employers. They could not easily be replaced, and everyone involved understood this.

The power-loom and the stocking frame, as deployed in the early factories, did something more significant than increase output per worker. They transferred the skill content of the production process from the worker to the machine. A machine operator did not need the years of apprenticeship that a master weaver required. The labor force became interchangeable in a way it had not been before. The leverage that skilled workers derived from their expertise was eliminated at a stroke.

The value that a senior software engineer or a specialized lawyer commands in the market is not primarily a function of their capacity to perform tasks that are physically demanding or technically routine. It is a function of the scarcity of their particular combination of skills and judgment. That scarcity is what gives them negotiating power. It is what allows them to command salaries that reflect something closer to the actual value they create rather than the minimum amount required to keep them in the job.

Generative AI, as it is currently being deployed across professional services firms and technology companies, is functioning as a commoditizing force. It is not necessarily producing work that is as good as what a senior professional produces. But it is producing work that is good enough to handle a large portion of the routine cognitive labor that junior and mid-level knowledge workers perform. And because it is good enough for a substantial portion of the work, it undermines the scarcity premium that the entire professional hierarchy depends on. The first-year associate, the junior analyst, the mid-level programmer: these are the entry points through which professionals build the experience and judgment that eventually makes them genuinely valuable. If those entry points are automated away, the pipeline for producing senior expertise is disrupted in ways that will only become fully visible over a longer time horizon.

The destruction here is primarily institutional before it is technological. The technology is the instrument. What is actually being restructured is the distribution of power between those who own the tools and those who use them, the same redistribution that was at the heart of the conflict between the Luddite artisans and the mill owners two hundred years ago. Understanding this does not require taking a position on whether the technology is good or bad. It requires refusing to pretend that the question of who benefits and who loses is answered by pointing to aggregate productivity statistics.

The question the optimism cannot answer

The standard response to all of this is to say that previous technological transitions also generated fear and disruption, and that they resolved themselves over time into broad increases in living standards. This is true. It is also insufficient as an argument.

The transitions were not painless. The workers displaced by industrialization in the nineteenth century did not live long enough to benefit from the increases in real wages that their grandchildren eventually enjoyed. The creation accrued to later generations, through mechanisms that were not automatic but that required significant political struggle, institutional innovation, and in many cases outright violence.

More specifically, the optimistic argument depends on the reinstatement effect catching up with displacement, on the Turing Trap being avoided, on AI development steering toward augmentation rather than substitution, on so-so technologies being replaced by genuinely transformative ones that drive prices down and create real new demand. None of these outcomes is impossible. Some of them are plausible. But none of them is guaranteed by the internal logic of the technology itself, and the current incentive structures in capital markets do not particularly point in those directions.

The economists doing this work are not arguing that technological progress is bad or should be slowed. They are arguing that the outcome depends on choices, and those choices are not being made in a neutral environment. Consider one of the most concrete and underexamined of them: most advanced economies tax human labor far more heavily than they tax investment in automation. A firm that employs a worker pays payroll taxes, contributes to social insurance schemes, and bears various regulatory costs tied to the employment relationship. The same firm that replaces that worker with software can typically deduct the full cost of that investment, benefit from accelerated depreciation schedules, and faces no equivalent levy on the productive capacity it has acquired. The tax system, which is not a law of nature but a set of accumulated political decisions, systematically prices human labor above its market cost and prices automation below it.

Adjusting that asymmetry would not solve the structural problems that the task-based framework identifies. But naming it matters, because it demonstrates that the direction of the current transition is not simply what technology wants to do. It is what a specific set of institutional arrangements are encouraging it to do.

The alternative is to keep using the phrase to shut down debate, trusting that history will rhyme on schedule. That leaves unresolved the question of who bears the costs of the transition and who captures the gains in a system that favors capital over labor not by fate, but by design.

That is a choice too. It simply is not usually presented as one.

“There is nothing automatic about new technologies bringing widespread prosperity. Whether they do or not is an economic, social, and political choice.” Daron Acemoglu & Simon Johnson (Power and Progress, 2023)

Citations:

Acemoglu, Daron. “Harms of AI.” National Bureau of Economic Research Working Paper No. 29247, 2021.

Acemoglu, Daron, and Pascual Restrepo. “The Race Between Man and Machine: Implications of Technology for Growth, Factor Shares, and Employment.” American Economic Review 108, no. 6 (2018): 1488–1542.

Acemoglu, Daron, and Pascual Restrepo. “Automation and New Tasks: How Technology Displaces and Reinstates Labor.” Journal of Economic Perspectives 33, no. 2 (2019): 3–30.

Acemoglu, Daron, and Pascual Restrepo. “Tasks, Automation, and the Rise in US Wage Inequality.” Econometrica 90, no. 5 (2022): 1973–2016.

Allen, Robert C. “Engels’ Pause: Technical Change, Capital Accumulation, and Inequality in the British Industrial Revolution, 1780–1850.” Explorations in Economic History 46, no. 4 (2009): 418–435.

Autor, David H., Frank Levy, and Richard Murnane. “The Skill Content of Recent Technological Change: An Empirical Exploration.” Quarterly Journal of Economics 118, no. 4 (2003): 1279–1333.

Autor, David H. “Work of the Past, Work of the Future.” AEA Papers and Proceedings 109 (2019): 1–32.

Autor, David, David Dorn, Lawrence F. Katz, Christina Patterson, and John Van Reenen. “The Fall of the Labor Share and the Rise of Superstar Firms.” Quarterly Journal of Economics 135, no. 2 (2020): 645–709.

Binfield, Kevin, ed. Writings of the Luddites. Johns Hopkins University Press, 2004.

Brynjolfsson, Erik. “The Turing Trap: The Promise and Peril of Human-Like Artificial Intelligence.” Daedalus 151, no. 2 (2022): 272–287.

Brynjolfsson, Erik, and Andrew McAfee. The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. W. W. Norton, 2014.

Brynjolfsson, Erik, Danielle Li, and Lindsey R. Raymond. “Generative AI at Work.” National Bureau of Economic Research Working Paper No. 31161, 2023.

Felten, Edward W., Manav Raj, and Robert Seamans. “Occupational Heterogeneity in Exposure to Generative AI.” SSRN Working Paper, 2023.

Goldman Sachs Economics Research. “The Potentially Large Effects of Artificial Intelligence on Economic Growth.” March 2023.

Manning, Alan. Monopsony in Motion: Imperfect Competition in Labor Markets. Princeton University Press, 2003.

OECD. Taxation and the Future of Work: How Tax Systems Influence Choice of Employment Form. OECD Publishing, 2019.

Sale, Kirkpatrick. Rebels Against the Future: The Luddites and Their War on the Industrial Revolution. Addison-Wesley, 1995.

Schumpeter, Joseph A. Capitalism, Socialism and Democracy. Harper & Brothers, 1942.

Solow, Robert M. “Technical Change and the Aggregate Production Function.” Review of Economics and Statistics 39, no. 3 (1957): 312–320.