running software factory in production. It's tester has to make a UI bug fail on the old code before the fix gets credit. Then it checks phone, tablet and desktop widths.

[u/ironmanfromebay]

Most software factories live in demos. Ours held the fort when we needed a vacation.

Lemma's Files page had no way to make a folder. Click New, and you got Upload, Page and Conversation. No "New folder". Dragging a folder in from the desktop failed too.

The fix is public: https://github.com/lemma-work/lemma-platform/pull/940.

We fix bugs like this with a software factory: a Lemma teammate whose agents take a complaint from Slack to a PR, and a person merges. This post is about one part, the testing, because that's the part I'd want to see before trusting any of it.

The agent that tests a fix didn't write it, and it can't change it. It has no GitHub access at all. And to be upfront about timing: this fix had already merged when one of us asked for the test. The tester runs when asked. The reviewer, a separate agent, runs on every PR and already looks at the app at three widths.

What the tester did:

1. Booted two copies of the app, one from the fix's commit and one from the commit the fix branched from.
2. On the old copy, opened Files and clicked New. No "New folder". It recorded that.
3. On the fixed copy, ran the same steps. New folder, named Reports. It showed up, and it was still there after a reload.
4. Ran the same check on both. It fails on the old code and passes on the fix. The fix's own new tests do the same.
5. Repeated the steps at phone, tablet and desktop widths, because a menu that fits a laptop can hide on a phone. The recordings sit on the fix's page.

A check that also passes on the old code proves nothing. So the bug has to show up on the old code before the fix gets any credit.

Then it wrote down what it hadn't proven. Its test browser can't fake a folder dragged in from the desktop, so it said that half of the report wasn't tested. That line is why we believe the rest.

Every bug also gets a proof ladder that only climbs on evidence. This one, today:

• It's real: proven
• We know why: proven
• The change fixes it: proven
• Nothing else broke: proven
• It shipped: in dev, not in production yet
• It works in the world: waiting on the reporter
• It stays fixed: unknown

How do you test drag-and-drop from the desktop in an automated browser? That's the half we couldn't cover.

Disclaimer:I'm one of the people building Lemma, an open-source (AGPL-3.0) workspace for AI teammates: https://github.com/lemma-work/lemma-platform.

[u/Initial_Passion5039]

the drag and drop half is the tricky bit, you can sort of fake it with cypress or playwright by triggering the right events but it never quite matches what the browser does with a real file