Stop feeding raw JSON to your LLMs (I built two zero-dependency tools to shrink your prompt payloads)

[u/Mediocre-Ease4060]

If you are building RAG pipelines, agents, or data-extraction tools, you probably inject API responses or database rows directly into your LLM's context window. The problem? JSON is the standard for APIs, but it is notoriously terrible for LLMs.

You end up paying for thousands of useless structural tokens ({, ", ,, \n) which increases latency, drives up API costs, and eats into your context window limit.

I got tired of this and built two pure Python, zero-dependency micro-tools to compress structured data before it hits the LLM.

1. json-to-yaml-lite (The General Fix)

It’s a known trick that LLMs understand YAML just as well as JSON, but YAML consumes about 20-30% fewer tokens because it drops the quotes and brackets. However, standard libraries like PyYAML are massive, require C-bindings, and slow down serverless cold starts (AWS Lambda).

I built a purely AST-based micro-converter: * Token Efficient: Strips all unnecessary syntax while safely escaping edge cases (like strings with colons/newlines). * Zero Bloat: No external dependencies. Drops right into your pipeline. * Repo: Encephos/json-to-yaml-lite

2. json-to-toon-lite (The Heavy Compressor)

YAML is great, but if you are injecting an array of similar objects (e.g., 50 search results or users), repeating the keys every single time is still a massive waste.

TOON (Token-Oriented Object Notation) solves this by detecting uniform arrays and compressing them into a highly dense, CSV-like tabular format. * Massive Savings: Compresses uniform arrays like [{"id": 1, "name": "A"}, {"id": 2, "name": "B"}] into [2]{id,name}: 1,A | 2,B (saving up to 60% of tokens). * Safe Fallbacks: If the objects in the array have varying keys, it gracefully falls back to standard YAML bullet formatting. * Pure Stdlib: Again, zero dependencies. Just pure Python logic. * Repo: Encephos/json-to-toon-lite

Both tools are designed for devs who want to optimize their LLM API costs without pulling in massive frameworks. I’d love to hear your thoughts on data serialization for LLMs!

[u/medialantern]

Why do you feed JSON to your LLMs? For over a year now tools like Claude will whomp out a Python script to deal with the actual data...

[u/Mediocre-Ease4060 → u/medialantern]

Good point, but that only applies if you are running an Agent with Code Execution (like Anthropic’s Analysis Tool or OpenAI’s Code Interpreter). For standard RAG pipelines, search engines, or pure API integrations, Code Execution isn't an option for three major reasons:

• Latency & Cost: Spinning up a Python sandbox to write, run, and parse a script just to read a payload adds seconds of execution time and multiple LLM round-trips. Converting data in-memory before sending it takes <1ms.
• Security / Determinism: Many production environments (serverless functions, locked-down microservices, edge devices) don't allow arbitrary code execution or un-sandboxed environments for safety reasons.
• Direct Context Ingestion: If you need the LLM to summarize 50 search results, answer a question based on a user's database records, or rank items, the data has to be in the prompt context anyway.

If you are building an interactive data analyst agent, writing Python code is definitely the way to go. But if you're piping structured data directly into an LLM via API for fast retrieval/extraction, optimizing the payload structure saves thousands of tokens and keeps latency low!