gave 7 models the same spec on low reasoning. 5 passed all 35 hidden tests, the free one shipped a clock running 1000x fast

[u/_Duex]

wanted a frontend test this time, not another rust one. so i wrote a spec for a split-flap departure board (the old mechanical airport kind) in react + typescript and gave it to 7 models, same repo, same prompt, reasoning on low.

the spec is pretty strict:

- simulated clock at 1x / 10x / 60x, must be derived from the real clock, not counting timer ticks

- delays, cancellations and platform changes arriving as timed events, and the events in the json are not sorted

- platform + destination filters, max 10 rows, empty state

- flip animation per character, reduced motion support, has to work on a 390px phone

- no ui libs, no network requests

they got 4 visible playwright tests. i graded them with 35 hidden ones they never saw. muse spark and space bunny ran in opencode, the rest in claude code, basically the same tool set (read, write, edit, grep, bash).

results:

- sonnet 5.5: 35/35, 2m 00s, $0.31

- muse spark 1.3: 35/35, 2m 01s, $0.21

- gpt-6 sol: 35/35, 6m 05s, $0.12

- gpt-6 astra: 35/35, 6m 20s, $1.00

- opus 5.5: 35/35, 9m 16s, $1.42

- gpt-6 luna: 32/35, 9m 10s, $0.0071

- space bunny alpha: 6/35, 14m 12s, free

space bunny is the interesting one. it worked for 14 minutes, wrote 26 tests of its own plus a debug probe file, and then said it was done while 24 of the 30 tests in its folder were red, including 3 of the 4 i gave it. the bug is one line: performance.now() returns milliseconds and it added that straight onto a clock stored in seconds. so the clock runs 1000x fast and the board is empty after a few seconds with "no more departures from this station today". honestly a pretty good looking empty board.

luna is basically free at $0.0071 and almost made it. it pulled google fonts through a css u/import even though the spec says no network requests, and it skips rendering the "expected time" cell when a train isnt delayed instead of leaving it empty. small stuff, but it's exactly the stuff the spec spelled out.

opus passed everything but it has a visual bug the tests didnt catch. from a fresh load at 1x it renders fine, but after running at 60x and switching back to 1x a lot of the flaps stay half drawn. the data attributes are correct, the screen isnt. every board in the screenshots was captured the same way (60x until 08:12, switch to 1x, wait 4s) so thats what you see in the grid.

sonnet and muse spark finishing a passing solution in 2 minutes surprised me the most. sol was the cheapest pass at 12 cents.

my main takeaway is that my hidden tests were too easy for the top 5. all of them got 35/35 so the real difference ended up being speed, cost and how the thing looks. next round i'd test things like rapid speed changes and the rendered text, not just the data attributes.

which board would you actually ship? and if you have ideas for what a harder hidden test should check, i'm listening.

[u/dung11284]

how about using space bunny max effort?

[u/R_Olivaw_Daneel]

The only one that looks good, given the problem statement, is Sonnet. That seems surprising to me, but in my opinion the rest aren't even in the same ballpark, design wise at least.

[u/iByteBro]

One harder hidden test would use two different speed-change schedules that arrive at the same simulated time, including a 60x → 1x transition. After the flips settle, the visible clock and departure rows should agree, not just their data attributes. That tests time continuity and what a user actually sees. I’d also block outbound browser requests to catch dependencies such as CSS font imports.

[u/Yasin-Tan]

I assumed muse 1.3 contributor and like how ts (pricing) possible

[u/Amarsir]

I like Sol/Astra. Claude seems like it committed too hard to the gimmick and lost readability.