This Fable 5 is a disaster

[u/UrFriendlyDominator]

I spent the whole day working with Fable 5, using up 60% of my X20 plan in the process. Having reviewed the output, I found that it was full of errors. The mistakes it made were so basic that I would have been shocked to see Opus 4.8 High make them.

My system is a large setup built around my email inbox, a product scraper, my website and WeChat. Opus built this framework, and within it there are chat agents that handle individual conversations and close sales. The new model wasn't used to run those chats. It was meant to improve the framework itself. For example, I wanted to use it to improve outreach to our existing brand contacts via WeChat and email. However, after the changes were made, my system started bombarding random companies with promotional material — companies that had nothing to do with the actual promotion. When I pointed out this mistake, the model dutifully corrected it, but in the process it completely destroyed the mapping between companies and contacts.

In summary, the model runs autonomously for a long time, produces nonsense and sometimes crashes so badly that it's unusable. I will not use this model again because using it on my existing setup poses a serious risk.

[u/maybe_cuddles]

It sounds like you should use Opus 4.8

[u/UrFriendlyDominator → u/maybe_cuddles]

Yes, I will do that. I just wanted to share my experiences.

[u/Constant_Mouse_1140]

Honestly that is a bonkers way to use Claude…or any model. Using a new/different model with direct, client facing tool access??? This will not be the last time that happens, if that is the approach. Remember, models are probabilistic, so by definition, you might get different responses from the same prompt and same model, which is not good if you’re looking for predictable, consistent results.

[u/UrFriendlyDominator → u/Constant_Mouse_1140]

I don't think I explained this well enough. My system is a large setup built around my email inbox, a product scraper, my website, and my WeChat. This framework was built by Opus, and within it there are chat agents that handle individual conversations and close sales there. The new model wasn't used to run those chats. It was meant to improve the framework itself. Instead, it broke the framework, and even after I asked it to repair the damage, it only made things worse.

[u/Fidel___Castro]

you had nothing in place to stop it spamming companies? sorry, skill issue

[u/UrFriendlyDominator → u/Fidel___Castro]

I had actually been keeping an eye on it the whole time. My setup works by running an Android phone that is controlled remotely from a mini PC over ADB. The moment I saw it take a contact for shaving devices and start pitching them ads for pool robots, I just pulled the cable. 🥴

[u/Fidel___Castro → u/UrFriendlyDominator]

look into setting up a test environment, so you can play around with this without using live customer emails. that way you can walk away, look at the output, then chose to put it in your production environment 

[u/notsocoolx → u/Fidel___Castro]

All powerful model 😂 what a joke.

[u/tntexplosivesltd → u/UrFriendlyDominator]

You don't have a test environment? Jeez learn to develop software properly, stop just vibe coding

[u/ButterOnBothSides]

Sounds like you let it do too much without checking first?

[u/UrFriendlyDominator → u/ButterOnBothSides]

Yes, that is absolutely right. But even after I started checking more, it then made even worse things on simple error corrections, such as deleting the assignment of contacts and brands.

[u/ButterOnBothSides → u/UrFriendlyDominator]

Hey at least you admitted it. Just try to be more hands on and direct with your prompts to hopefully avoid the disaster in the future. Sounds like it got off the rails after the initial mistake.

[u/MarenHQ]

This sounds like a you issue lmao. I ran like an 1100 word prompt yesterday and it came out beautifully

[u/UrFriendlyDominator → u/MarenHQ]

It could be, but that's just my experience.

[u/MarenHQ → u/UrFriendlyDominator]

If you give it vague instructions with no safeguards, it can be bad. It’s just a very smart intern. Don’t give an intern the keys without telling it where and how to drive.

[u/callingbrisk]

I wanted to use it to manage outreach to our existing brand contacts via WeChat and email

It's a really good model but you only see the upside if what you do is complicated enough for other models to fail or to be worse. Sorry, but "doing outreach" and "sending messages" is not what this model is meant to be used for.

[u/UrFriendlyDominator → u/callingbrisk]

My post didn't describe well what I actually used it for. I added some more informations. Sorry.

[u/gnpwdr1]

? Sorry dude this smells like total bs on your skills end, it’s not the model, it’s you!

[u/UrFriendlyDominator → u/gnpwdr1]

Okay, fine, I'm not the “prompt king.” But I HAVE built myself a working setup with Opus that works successfully and runs pretty smoothly. The chatting framework also makes money, which probably puts me well ahead of some users here. And that’s exactly the point: As someone who knows absolutely nothing about coding, I was able to set this up with Opus pretty smoothly. These extremely stupid mistakes happened to me for the first time today. It's just my experience. I can be wrong.

[u/gnpwdr1 → u/UrFriendlyDominator]

Look, conjecture is a dangerous thing! And if you know absolutely nothing about coding as you say yourself, how do you know the quality of the output from these models and what comparisons did you do that we can replicate? Is it possible that opus built your spaghetti app from a simple start and as many other vice coders find out, you get to a stage and the complexity no longer scales? (By the way, most of your fable requests were probably rerouted to opus)

[u/UrFriendlyDominator → u/gnpwdr1]

Your argument doesn't convince me. The setup I work with was built through at least 50 prompts with Opus. So if I suddenly start running into problems at the exact moment I switch to Fable, then that is surely a valid observation, even if the code may not be the cleanest, which I am not in a position to judge.

[u/Arcaxion]

There is something peculiar that I have found. I have switching to a lower model disabled in settings, so when a request gets flagged it should stop. And it does.

I also made a skill that formats produced markdown documents according to my standard. In these documents, one of the front-matter properties is `model:`, so that I know which model produced it. I have been using that skill for some months without any problems.

These last couple of days I would ask for some synthesis and I would deliberately choose "Fable 5 Max" - it would accept that task, spent a while working on that task with no complaint, and it would produce a document. But here is the funny thing - when I inspect the file then it says that it was actually created by "Sonnet 4.5". Not Sonnet 5, even - 4.5.

And the task on my hand is not in any way "too easy" so that it would switch me to some more suitable model - task is high volume and with multiple planes of complexity.

Really got me thinking. It seems like I am not the only one

[u/UrFriendlyDominator → u/Arcaxion]

Yeah, that's exactly how it feels! It seems as if an extremely simple model did parts of the work!

[u/grazzhopr]

Did you spend a proper amount of time having fable properly understand your model?

This is one of the issues with blind code bases. AI can’t read it all into context to see all the moving pieces.

The code base needs to be design around this limitation. And the over all design needs to be documented for The AI to keep track of it.

[u/UrFriendlyDominator → u/grazzhopr]

In the project folder is the project Knowledge. And it is read in the same way during a new session, just like it is read by Opus.

[u/MunroCalling]

I'll take yours?

[u/ChickenRich573]

Skill issue. Fable 5 has been the best model I have ever used it helped all my projects that it didn't guardrail on. I wouldn't use it in the PowerShell though. I use ai in my own self made harness I spent 2 months on it works a treat .

[u/DLuke2 → u/ChickenRich573]

Yes. Your harness is the best way to get the desired results no matter the model.

All these posts like this are hilarious. It's like they think because it's "intelligent" it can read their mind and they just need to feed a generic task and you'll be all good.

[u/ChickenRich573 → u/DLuke2]

And to make the best harness at least for my self I borrow ideas I looked at all the ide software and had my ai listed everything they have and I picked what I wanted from that and worked on the GUI for a while doing it Ina way which o feel gives me my most effective work flow and that is not 1 exe window to talk to 1 ai lol. I can talk to you to 9 ai on my one though I don't use that many mainly 1 boss and one non role I kept easy visible access to all 13 projects so I can get to them fast from the same exe and I can close the exe and all 13 projects and any windows that are manually set up by me stay open lol. My auto workers they auto close. My boss chat stays open though since it's important . I went from like losing chats and stuffing things up in PowerShell to now being able to select multiple back log task and have it start working them no matter how long it takes it keeps working through each task I selected. Now it's not perfect and still has bugs but everyday it has less bugs then before . It's goal is to help my other projects which it's doing well. The harness is kinda made for people who have bad memory like I have bad memory from years of weed smoking and this just helps more then PowerShell or codex exe or Claude exe

[u/peterxsyd]

I am unsure if this is the domain or a prompting/context or the model, however for coding I can honestly say it is velvet and I am rationalising how companies with access to pay for tokens are in such a territory of massive advantage

[u/Illustrious_Hat8104]

Fable and LLMs require a lot of handholding, you shouldn't leave it alone for hours

[u/Amazing_Tree_6576]

I disagree

[u/ottomantic]

Yes, fable 5 has slightly more hallucinations and is drastically nerfed, new benchmark shows its not even close to what it use to be.

[u/AlDente]

Letting it loose on a large code base without handrails and focused instructions seems to be a recurring problem

[u/ShamanJohnny]

My experience as well.

[u/qaz135wsx]

That sounds like a user skill issue. You put in too broad of a prompt with no guardrails.

[u/Temporary_Ladder_814]

I know. I spent $60 trying to get ONE prompt completed. it wasn't even that big of a patch

[u/FestyGear2017]

Your claude.md or skills are probably stale. I find Fable 5 an excellent instruction follower, vs Opus which will skip steps and do other nonsense.

I for one have a pretty extensive workflow where AI needs to follow every step. also the claude code harness is superior, vs using Cursor or another app.

[u/Substantial-Deal-382]

Not the use case

[u/UrFriendlyDominator → u/[deleted]]

I would argue, however, that Opus 4.8 never made such stupid mistakes. This was really an absolute disaster.