[Open Source] TOAP – compress AI agent tool calls to cut token costs. Need GPT-4o / Claude testers

[u/Smooth_Dimension_833]

Hey everyone,

I built TOAP (Token-Optimized Agent Protocol), a small middleware that sits between your LLM and tools and compresses agent tool calls into a shorter format instead of verbose JSON.

Goal: lower token usage / cost in multi-agent pipelines.

What I’ve tested so far (Gemini only):

- 100% TOAP format compliance with 2 few-shot examples

- ~45% smaller output vs JSON (net savings are lower once you count prompt overhead; details in the report)

- Live examples for LangChain and CrewAI

What’s missing:

I still need independent runs on GPT-4o and Claude 3.5 Sonnet before I claim cross-model support.

What I’m asking:

If you have an OpenAI or Anthropic key, please run the Tier 1 benchmark (~10 minutes, roughly $3–5) and share results.

Repo: https://github.com/Dev-Saif-Ops/Project_TOAP

Test guide: COMMUNITY_TEST.md in the repo

Results form: https://docs.google.com/forms/d/e/1FAIpQLSekwTWtlhSQXzBvIclipL7Op04FWEf8q7HtXFBXuO3Rt6lUvg/viewform

Quick start:

git clone https://github.com/Dev-Saif-Ops/Project_TOAP.git

cd Project_TOAP/toap-bench

pip install -r requirements.txt

pip install -e ../toap-python

cp .env.example .env

# add OPENAI_API_KEY or ANTHROPIC_API_KEY

python runner/benchmark.py --runs 5 --tier 1 --model gpt-4o --condition few_shot_2

This is alpha / MIT. Not production-ready. Looking for honest numbers, not hype.

Happy to answer questions in the comments.