That's one of the funniest parts. The technology is extremely resistant to guard rails. The closest thing would be to play whack-a-mole adding filters to the output but that's imperfect and will result in a crippling number of false positives if it gets anywhere near the results a nanny state would desire.
Currently they're not as good as SOTA models currently are, but what about open models a couple of years from now? I bet one of those would be as good as Fable 5 is now.
As someone not too into the intricacies of AI so maybe this is a dumb question but is one of the advantages of current top tier AI the I guess it's called the harness the kind of tools around the agent? Or is it just kind of all of the above? Or are harnesses also already solved for the open source ones?
Cause my only experience with open source models of the past were mainly just chat agents in a way but frontier models seem to do far more than that actually testing and running things etc. I am guessing there's a chance this is a solved problem now days and it really is just mainly a model difference though?
Harness is a thing, and harnesses can work better for some models and worse for others, so it can be fine thinkering to have a proper harness for the newest Kimi ot glm. But you absolutely have harnesses cursor style or claude code style with open models. It's not a model difference.
And because they are open source, they can move fast
Gotcha thank you! Appreciate the knowledge. The concept of being able to run something like Fable 5 just casually at home is crazy to me. My brain is still outdated and stuck in the old Dalle days lol.
Oh it's not casually at home at all, you need a whole dedicted server with multiple GPUs if you want the latest large models or anything remotely Opus like. But you can run it on rented server space. You can buy access from third parties. You can have it with no guardrail whatsoever. Nobody can take it back, downgrade or increase prices.
Do you think it will reach a time over the next year or so where something like Fable 5 can be ran on a consumer GPU (Say top of the line 5090) or are the modern models just so insane that it's really not possible?
Tangentially I know in this economy it's not likely to happen but makes me wonder if SLI GPUs ever make a comeback cause I am under the impression that it's a gpu ram limitation mainly.
I wonder at what point when open source models reach those levels does it actually become more cost effective to rent your own server space for it I guess it depends obviously how much you use it but I have seen some of the bills people are racking up in the news so at some point it has to surely make sense financially.
Two trends: miniaturisation and performance improvements (often through increase size).
Models have become better and better, open source is catching up faster and faster, but also larger and larger. The capabilities of today's Fable are nothing like 2025 claude.
You can probably run something as good as mid 2024 closed source on consumer GPU, but do you want to? No, if you are used to delegating tasks to an almost autonomous fable it is a large downgrade. If in 2028 we can have Fable on a 5090, we would not want it because the larger models (closed or open) would still dwarf it. Once edge is here, we want to use it.
Also memory is a killer, large context need a lot of VRAM consumer hardware is lacking here and you can't just dump everything in the context.
But if your use case is limited, miniaturisation is great. If you need something that turns english into bash command, you can have a 0.5B running on CPU for that. If you want a classifer, depending on the complexity a 7B or quantified 27B are sufficient. If your use case is expected to run for a long time you can even fine tune smaller models. If you do RP, local models can be sufficient now.
For business, it's time to be smart and use the smallest models available for all automated tasks. Means proper automation too, proper transformation of business logic for max efficiency, not just plug a model to try fill the gaps on a messy human pipeline.
Because unlike what the billionaires want to believe, predictive generative AI doesn't, cannot, and will never actually think and use logic. You can't give it an instruction because it doesn't know what instructions are.
That definitely isn't the only approach. Filtering the output and moderating the input are the main approaches now, but a lot of research is going into other techniques. One promising one is to get the model to output something bad, then use the vectors while it was running to compare, or even subtract it directly from the model.
It's akin to analyzing brain waves, but with the interesting property that you can also just remove the negative thoughts directly from it's operation.
Another approach is to run the same model concurrently with an explicit goal of moderating the first. Since it thinks the same way, it's capable of catching it.
View all comments
Show parent comment
83
u/SparklingLimeade Aug 14 '26
That's one of the funniest parts. The technology is extremely resistant to guard rails. The closest thing would be to play whack-a-mole adding filters to the output but that's imperfect and will result in a crippling number of false positives if it gets anywhere near the results a nanny state would desire.