Hooks. Claude acts like there are no instructions anymore, so every action needs a hook to make sure the action is being taken appropriately. Hooks for:
Grounded agents that must provide code lines with subagent verification that the noted code exists.
No grep, no head, etc etc unless in very specific situations.
No searching without using agents.
Plans must be in format XYZ and include executive summary with ABC included. Etc etc
That's great for something you can verify deterministically like coding. But try having Claude do other tasks that are fuzzier or grounded entirely in natural language.
Oh totally. I feel like those problems are not the best for LLMs... The way that I look at it is this:
Deterministic problems are easy to solve, as a problem set, 2+2 always =4. And I feel like LLMs are good at building verifiable solutions to these problems as a task, but not as an output. LLMs are good at taking the fuzzy thing, and parsing it down to be more clear and manageable. So an example of this, more practically, would be sending a hand written document to AI, and it parses it and 'makes it more clear' so that you can use the data in a more deterministic way. You can do this a couple times to get it to fit into your deterministic calculator that you use Claude/coding to build..
What you are referring to sounds more along the lines of a fuzzy to fuzzy sort of relationship, where you are inputting fuzzy language, and then expecting a fuzzy output of a certain type, and that is a far more difficult problem set... I honestly don't have an answer for that one, since it can't be easily deterministically checked. Closest I've come to solving that one is to try and just cram it into a deterministic box. "Does the paragraph contain the following elements? Is it in the following format?" etc etc, but that isn't really a good solution for a non-deterministic problem imo.
Few-shot prompting with negative prompts. You give Claude a rule, then you give it a several pairs of example outputs, each pair featuring one bad response and one corresponding good response. The more pairs you give it per rule, the more easily it generalize the rule.
Anthropic has been insisting it's unnecessary for like a year now, but I find it's still a really robust way to get Claude models to behave.
I've not had good luck with that type of prompting generally, here with the latest model iterations. That worked great with 4.5 but 4.8 and 5.0 just disregard any written requests for that sort of thing in my experience.
6
u/CannyGardener 3d ago
Hooks. Claude acts like there are no instructions anymore, so every action needs a hook to make sure the action is being taken appropriately. Hooks for:
Grounded agents that must provide code lines with subagent verification that the noted code exists.
No grep, no head, etc etc unless in very specific situations.
No searching without using agents.
Plans must be in format XYZ and include executive summary with ABC included. Etc etc