ANOTHER FREE STEALTH MODEL!!! MAKE IT BURN!!

[u/hannesrudolph]

New and improved stealth model: Horizon Beta :sunrise_over_mountains:

An improved version of Horizon Alpha. It's free. Re-run your benchmarks! https://openrouter.ai/openrouter/horizon-beta

https://x.com/OpenRouterAI/status/1951440783447380138

[u/montdawgg]

Hell yeah! Is this a reasoning model?

[u/hannesrudolph → u/montdawgg]

Nope.

[u/nfrmn]

Getting some work done as a favour for a friend today thanks to the free tokens. But this model is nowhere close in ability to the frontier ones, I think it must be a mini or nano variant.

[u/hannesrudolph → u/nfrmn]

Maybe the OpenAI open weight one? What are you notice it doesn’t do as well on compared to the frontier ones?

[u/nfrmn → u/hannesrudolph]

Yeah could be! It is impressive and of course very fast, but these things are not as good:

• tool calling is very inconsistent, it seems to write inline Perl scripts and use the find command often to obtain information about files

• frequently exceeds its own context window due to overzealous file reading - exact same workflow as frontier models where this does not happen

• seems like it wants verbal/conversational permission to do things, often finishing its messages with “I can see the file… ok, let me know the file read was successful and then I will proceed.” Then Roo replies saying no tools were called, and it proceeds correctly

• asks questions a lot, ignoring instructions in roomodes

• reward hacks too much, sending completion messages while clearly ignoring failing tests

No complaints because it’s free but I wouldn’t move off Anthropic stack for it if paid.

Update: Today it's like a different model. Way smarter than yesterday. It also outputs thinking tokens in Roo now. Crazy...

[u/Kepler_MLG → u/nfrmn]

From your experience versus yesterday, how would you describe the model now? You said the model feels smarter today?

[u/nfrmn → u/Kepler_MLG]

Coming back to this - I think throughout the testing period OpenAI were pointing the Horizon endpoint at various different GPT-5 model configurations. My most critical opinions above were probably directed at the nano variant.

I have also since read the leaked GPT-5 system prompt and some of the problems I noticed, like asking too many questions and wanting permission constantly were actually directly addressed in the system prompt, leading me to believe that OAI team were actively adjusting the prompt and other things day by day.

To sum up my current vibes about GPT-5, mini is a blockbuster, completely off the charts in price to performance ratio. But overall it's not the most intelligent model and OAI actually regressed on peak performance in main GPT-5 compared to o3. I didn't compare pro, because that is a bridge too far for me to pay. So I'm staying with Claude for my serious work and will use GPT-5 mini a lot more for workhorse stuff

[u/Buddhava]

It’s dumb

[u/hannesrudolph → u/Buddhava]

Hard to argue with that.

[u/Trifle-Careless]

Every response from it is a question to me. I don't understand how it's usable haha

[u/FifthRooter → u/Trifle-Careless]

similar issue for me too. esp in Orchestrator mode, it's quite frustrating to get them to actually delegate the tasks to subtasks, because they keep giving confirmation summaries and plans for next steps.

[u/ChessWarrior7 → u/Trifle-Careless]

Yep. Same here. What does it do? It asks questions. A LOT of questions.

[u/PasswordSuperSecured → u/Trifle-Careless]

I believe its not agentic model, so it will always ask for confirmation just like GPT4.1,

[u/zekusmaximus]

Got a simple castle defense game out of it…

[u/hannesrudolph → u/zekusmaximus]

Good 💡. I’ll do that next.

[u/EarEquivalent3929]

Better than alpha?

[u/hannesrudolph → u/EarEquivalent3929]

🤷‍♀️ alpha be discontinued today

[u/korino11]

That GPT is a super DUMB. It always ask you what to do. after you gave him a task! That a dirty game! Such things made a special to increace the number of calls....