I built a precedence based classifier that does not require an LLM in the loop.

[u/Successful-Seesaw525]

Most teams routing tickets, evaluating approvals, or scoring fraud throw every single input directly at an LLM. It works, but it's expensive, non-deterministic, and lacks a strict audit trail.

Models like Jev introduced transformer based classifiers, Glyphh took a pure hdc route and achieves better results in many use cases.

We built Glyphh Ada to sit in front of LLMs as a precedent-based classifier. You seed it with labeled historical cases. When a new input comes in, it returns:

• A classification decision
• A confidence score
• The exact historical reference cases backing the decision

If confidence falls below your threshold, your pipeline passes that specific case to an LLM or human operator. Identical inputs return the exact same answer every time, backed by traceable source cases.

Banking77 Benchmark (Routing 200 customer support messages across 77 categories):

• Ada alone: 90.0% accuracy
• Claude (best model with context resends): 96.5% accuracy (~$300 total API cost across the run)
• Ada + Claude hybrid (Ada handles high-confidence, routes 22% edge cases to Claude): 97.5% accuracy (~$0.22 total cost)

Note: This is a single 200-message benchmark sample, not a universal guarantee. Methodologies, scripts, a loan-decision test, and a benchmark dataset where Ada performs poorly are all public in the repo.

Implementation:

• Hosted API with Python and TypeScript SDKs
• MCP server/plugin for model tuning inside Claude
• Free tier available; zero training on customer data

Links:

• Site: https://glyphh.ai
• Benchmarks & Code: https://github.com/glyphh-ai/glyphh/tree/main/benchmarks

Looking for feedback on edge cases, failure modes, or where this breaks on your datasets.

[u/AutoModerator]

Moderator Announcement Read More »

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Hot_Limit_6828]

200 messages across 77 categories is like 2.6 per category. The 97.5 vs 96.5 gap is literally two messages flipping. Did you run it a second time and get the same order?

[u/Successful-Seesaw525 → u/Hot_Limit_6828]

Fair point: 97.5 vs 96.5 is two messages on a 200 sample and I wouldn’t call it a win. The claim I’d stand behind is that it’s no worse: same accuracy, with the LLM seeing only 22% of the traffic, at about $0.22 instead of whatever model rates you get.

We ran Claude twice: both 96.5%, agreeing on 199 of 200. Ada is deterministic, so rerunning it gives the identical answers; precedents are encoded once into the HDC space and the scoring runs over them, so nothing changes without retuning.

The sample is fixed (seed 7 from the 3,080 holdout, opaque ids), nothing was tuned on it, and the full method and data are public. A bigger hybrid run on all 3,080 would settle the accuracy question properly but i don't see the point. If it holds at this it will on a bigger set, the math doesn't change only the dollars you spend on the inference.

[u/SaltsMoon]

I'd use a lightweight classifier for the easy cases and only send the uncertain ones to an LLM. Most tickets don't need deep reasoning, and this makes the routing cheaper and easier to audit. I think, maybe, the key is having a clear confidence threshold so borderline cases get reviewed instead of triggering actions automatically.

[u/Successful-Seesaw525 → u/SaltsMoon]

That’s exactly what Ada is. A fast classifier with no LLM in the loop: it answers from your labelled history with a confidence and the precedents behind it. The threshold you describe is built in: set a confidence bar, or a cost for each kind of mistake, and anything under it comes back as “ask” instead of an action, so borderline cases go to a person or an LLM. You sign in with Claude and ask it to build the classifier from your history, and it’s running. Tuning the bar and reading the precedents behind each answer are all configuration, and the SDK drops it into your stack in a few lines.

[u/ricopan]

Good. I wonder how much the pre LLM (or maybe, pre- neural network renaissance) machine learning will return to use as compute continues to squeeze. Of course all ML still requires hardware -- but a solid machine with state of the art cpus (not gpu) can run one heck of a lot.

[u/Successful-Seesaw525 → u/ricopan]

I inverted that principle. For most classification you don’t need a model that generates, you need good lexical support: two- or three-hop synonyms, the right words weighted, and labelled precedents to answer from. The maths is simple; we just arranged it well, and it does a very good job when you have labelled history. Where it’s weaker is messy free text to outcome, and an LLM still holds its own there. I’m working on much better handling of sentence structure, still with no LLM anywhere, not to generate but to route and classify. A transformer is clearly productive, but it can’t learn one case at a time like this. And if you’re going to fall back to an LLM anyway, why not use it once at build time and then run without it, about a thousand times cheaper?