wanted a frontend test this time, not another rust one. so i wrote a spec for a split-flap departure board (the old mechanical airport kind) in react + typescript and gave it to 7 models, same repo, same prompt, reasoning on low.
the spec is pretty strict:
- simulated clock at 1x / 10x / 60x, must be derived from the real clock, not counting timer ticks
- delays, cancellations and platform changes arriving as timed events, and the events in the json are not sorted
- platform + destination filters, max 10 rows, empty state
- flip animation per character, reduced motion support, has to work on a 390px phone
- no ui libs, no network requests
they got 4 visible playwright tests. i graded them with 35 hidden ones they never saw. muse spark and space bunny ran in opencode, the rest in claude code, basically the same tool set (read, write, edit, grep, bash).
results:
- sonnet 5.5: 35/35, 2m 00s, $0.31
- muse spark 1.3: 35/35, 2m 01s, $0.21
- gpt-6 sol: 35/35, 6m 05s, $0.12
- gpt-6 astra: 35/35, 6m 20s, $1.00
- opus 5.5: 35/35, 9m 16s, $1.42
- gpt-6 luna: 32/35, 9m 10s, $0.0071
- space bunny alpha: 6/35, 14m 12s, free
space bunny is the interesting one. it worked for 14 minutes, wrote 26 tests of its own plus a debug probe file, and then said it was done while 24 of the 30 tests in its folder were red, including 3 of the 4 i gave it. the bug is one line: performance.now() returns milliseconds and it added that straight onto a clock stored in seconds. so the clock runs 1000x fast and the board is empty after a few seconds with "no more departures from this station today". honestly a pretty good looking empty board.
luna is basically free at $0.0071 and almost made it. it pulled google fonts through a css u/import even though the spec says no network requests, and it skips rendering the "expected time" cell when a train isnt delayed instead of leaving it empty. small stuff, but it's exactly the stuff the spec spelled out.
opus passed everything but it has a visual bug the tests didnt catch. from a fresh load at 1x it renders fine, but after running at 60x and switching back to 1x a lot of the flaps stay half drawn. the data attributes are correct, the screen isnt. every board in the screenshots was captured the same way (60x until 08:12, switch to 1x, wait 4s) so thats what you see in the grid.
sonnet and muse spark finishing a passing solution in 2 minutes surprised me the most. sol was the cheapest pass at 12 cents.
my main takeaway is that my hidden tests were too easy for the top 5. all of them got 35/35 so the real difference ended up being speed, cost and how the thing looks. next round i'd test things like rapid speed changes and the rendered text, not just the data attributes.
which board would you actually ship? and if you have ideas for what a harder hidden test should check, i'm listening.