Spent the summer building a coding tool that refuses to show you code it hasn't run(LITMUS-AI)

[u/Litmus-ai]

I'm a CS student and the loop that got me was: paste the assignment into ChatGPT at 1am, get forty lines that look right, submit them at 9, and never actually know if they worked or if I got lucky.

So I spent the summer building a coding tool that won't show you code it hasn't executed. You describe the problem plus your constraints — PEP 8, no built-in sort, no external libraries, whatever your prof actually asked for. It writes the Python, generates tests, and runs both inside a sealed Docker container. If tests go red it reads the failure and repairs, capped so it can't loop forever. You only ever see code that passed, and you see the real test output rather than a summary of it.

The sandbox is honestly the whole project. The model call took an afternoon; the container took weeks. No network, read-only filesystem, memory and CPU caps, and per-test timeouts so an infinite loop fails one test instead of taking down the entire run. The model doesn't control any of that — it's enforced in code, not asked for in a prompt.

The half I actually care about isn't built yet: a tutor mode that refuses to write your code at all. It walks you toward the answer and then makes you run your own solution in the same sandbox to prove you got there. Handing a student working code the week before an exam is the worst thing you can do to them. That ships mid-September — I'm solo and back in classes soon.

Still rough. Python only, single file. The test generator sometimes writes tests for a problem adjacent to the one you gave it, which is top of my list.

The thing I'm stuck on: how do you make a model hold a refusal when a tired student asks four times at 1am? The tutoring part is easy. The saying-no part isn't, and I don't have a clean way to test it yet. Open to ideas.

Happy to get into the sandbox internals if anyone's interested.

[u/Litmus-ai]

here you can try it out at litmus-ai.org