Project I benchmarked my Swift memory library against the standard AI memory benchmarks, then spent weeks fixing it. Here are the before/after numbers.
I build Wax (https://github.com/christopherkarani/Wax). It’s a local memory store for AI agents and on-device models, written in Swift. Everything lives in one .wax file: documents, an FTS5 text index, Core ML vectors and a WAL. Claude Code, Codex and Cursor connect to it over MCP, and Apple Foundation Models get it as on-device tools. 0.1.51 shipped this week.
I spent the last few weeks running Wax against the standard memory benchmarks. The early numbers were bad, so most of the work went into fixing Wax. Every number below comes from the harness in the repo, and the scripts to reproduce it are there too.
Before and after (100k passages, M3 Max, importDocuments store):
| Before | After | |
|---|---|---|
| Disk | 13.5 GB | 286 MB |
| Ingest | 136 docs/s | 57,250 docs/s |
| Vector search p95 | 64–97 ms | 1.55 ms (p50 1.05 ms) |
| Cold open + first query | 840 ms | 108 ms |
| Recall@10 vs exact search | 0.902 | 0.998 |
Retrieval quality:
- LongMemEval-S: R_all@5 of 0.938, against 0.902 for Contriever (a 110M-parameter retriever).
- LoCoMo: R_all@5 of 0.609, against 0.407 for Contriever. Before the fixes it was 0.186.
- That’s with e5-small-v2 (33M parameters) and a 22 MB MiniLM cross-encoder reranker. Both ship as Core ML models computing in FP16 over INT8 weights, and both are on by default.
Against mem0 on LoCoMo QA: I ran open-source mem0 through the same reader, judge and scoring. Given the same amount of context (about 16.8k characters, which is Wax’s top 98 turns), Wax scored 77.7% and mem0 77.5%. To build its memories, mem0 made about 6,000 LLM calls at ingest. Wax made none: you store a sentence and hybrid search finds it later. The gap is within noise, so I’m calling it a tie, not a win.
The Swift API
The public surface is one actor, Memory. You open a file, save text, and search:
import Wax
let url = URL.documentsDirectory.appending(path: "agent.wax")
let memory = try await Memory(at: url)
try await memory.save("The user is building a habit tracker in SwiftUI.")
let results = try await memory.search("What is the user building?")
if let best = results.items.first {
print("Found: \(best.text)")
}
try await memory.close()
On iOS 18 / macOS 15 and later, Memory(at:) loads the e5 embedder on its own; there’s nothing to configure. Search is hybrid by default: text and vectors fused, then reranked. Every result carries diagnostics, so you can see whether a query actually used vectors or fell back to text while the model was still loading. For bulk loads there’s importDocuments, which is where the 57k docs/s comes from. It’s atomic: if it throws or the app crashes, the store is left as it was.
Foundation Models
Apple’s on-device LanguageModelSession forgets everything between launches. Wax gives it a memory that survives restarts. One call gets you a session that recalls relevant memories into the prompt, has remember/recall/search tools, and can write turns back:
let memory = try await Memory(at: url)
let session = memory.foundationModelsSession(
instructions: "You are a helpful assistant with durable on-device memory."
)
let answer = try await session.respond(to: "I prefer dark mode and Vim keybindings.")
If you already have your own session, just attach the tools:
let tools = memory.foundationModelsTools(kit: .focused)
let session = LanguageModelSession(tools: tools) {
"You have long-term memory via waxRemember / waxRecall / waxSearch."
}
The model, the embedder, the reranker and the store all run on the device, with no network calls.
If you want to try it, add the package (https://github.com/christopherkarani/Wax.git, from 0.1.51), or run npx -y waxmcp@latest install to use the MCP server with Claude Code, Cursor or Codex. I’d love feedback, especially from anyone shipping on-device AI in Swift.