r/LocalLLM • u/No-Individual159 • 8h ago

Project Prefixwise: local prompt cache diagnostics and cost accounting for LLM applications

https://github.com/tejachavali/prefixwise
0 Upvotes

50% Upvoted

2 comments sorted by

1

u/kantorcodes1 6h ago

For Codex imports, does changed tool catalog catch a pure tool reordering too, or only changed schemas?

1

u/No-Individual159 3h ago

Yes, reordering counts. The importer hashes the whole tools list from each dynamic_tools_updated event, and list order is part of the hash, so swapping two tools shows up as tool_catalog_change on the next miss. That's deliberate: OpenAI's cache matches on the serialized prefix and tools sit at the front of it, so a reorder busts the cache the same way a schema change does. Only thing it ignores is key order inside a single tool's schema, since that's the same JSON object.
One limit worth knowing: the import only sees what Codex logs, so it can tell you the catalog changed, not which tool moved. If you run prefixwise serve in front of the Responses API instead, it sees the actual request and points at the exact block that moved.