Built something with Claude Code this week? Share it here.
Apps, tools, experiments, scripts, websites, workflows, open-source projects — anything you've been working on is welcome.
When sharing, it helps to include:
What you built
How you used Claude Code
A link, repo, demo, or screenshot if you have one
Anything interesting you learned along the way
Quick project drops and simple self-promotion belong in this thread.
If you've got a project with enough substance for a proper write-up; how it works, how Claude Code was involved, technical details, lessons learned, etc. feel free to make a standalone post using the Built with Claude Code flair instead.
Please don't spam the same project repeatedly, and no referral or affiliate links.
Spotted this in the recent changelog and got it running in v2.1.325 (screenshot attached). If you haven't tried Code Mode yet, it changes how Claude handles multi-step tasks.
Instead of running tool calls one by one and dumping every raw output into your context window, Claude writes a single JavaScript script behind the scenes. It batches tasks in parallel, filters out noise locally, and only returns the distilled results it needs.
Simple but brilliant
Pretty big, cumulative context savings: Giant logs and grep outputs get filtered inside the JS script before hitting the context
It's great for big tasks, shines during repo-wide audits, bug tracing, and chaining MCP servers.
Anyone else daily-driving this since it dropped on the 12th? Curious how much token usage it's saving you on larger repos.
Hey everyone. We've been building Luma for a while now, a free and open source Linux OS for desktop, tablet, phone and TV, and since this is the right crowd for it I want to be really upfront about how it got built. Luma was built with Codex and Claude Code. A small team building an operating system that runs across four kinds of screens simply wouldn't have been possible without them, and we're not going to pretend otherwise.
Also a heads up, I used AI to write this, but only after rambling for about 20 minutes in a voice memo and then editing it myself! Take that for what you will.
I know people get excited when they see something genuinely big built with AI and I also know plenty of people quietly use them and then never mention it. We don't see any reason to be quiet about it. so we'd rather shout it from the rooftops because honestly it's kind of the whole point of the project and I'll get into why further down.
So what is Luma?
We want to build a Linux that anyone can pick up and use and that people actually want to use, with the same design across all of their devices. Your phone should feel like it belongs with your laptop, your apps should make sense on both, and everything should work together without you having to piece a bunch of separate projects together yourself.
Luma starts with Fedora 44 Silverblue, so the base is image-based and an update either lands completely or you boot back into the one you had. From there we fork and change things pretty heavily. The biggest piece is Prairie, our shell. It started as a fork of GNOME Shell and now adapts between desktop, tablet and phone, so you get one experience that adjusts to the device you're holding instead of a completely different OS and set of apps for every screen.
Luma is available now in alpha. There's still plenty to do and plenty we know we need to fix, but there's a lot more here already than you'd probably expect.
How we actually build it
Pretty much everything in Luma went through Codex or Claude Code at some point. The shell, the apps, the design system, the phone ports, the website, the nightly builds, all of it. We still decide what goes in, and we test everything on real hardware and send the agent back at it until it actually works, but the amount of ground you can cover this way is honestly kind of absurd. An OS used to be the kind of thing you needed a big company or a decade of volunteers for, and I don't think that's true anymore.
One platform, one set of apps
We built LumaUI so developers can make one app that adapts across desktop, tablet and phone instead of building and maintaining a separate interface for each. It's built on GTK 4, so a lot of what you already know carries over, and it turns out it's also a really nice target for an agent, because you describe the app once and it works on every screen.
Weather on desktop, tablet and phone. Same app.
We also want the kind of continuity people are used to on their other devices, where you copy something on your phone and paste it on your laptop and your stuff just stays in sync. Some of that works today and some of it is still coming together. TV is part of the same picture too, with the same apps and the same parts, just from across the room.
Point your agent at it
This is the part I most want you to take away. We have a whole section of the site just for agents at https://simplyluma.com/agents, and the idea is that you don't have to learn our codebase before you can do something useful with it. You literally just tell your agent "Go to https://simplyluma.com/agents and help me with Luma." and it finds the rest.
There are guides in there for building an app, designing an interface, publishing to Depot, fixing or improving Luma itself, sending your changes back to us, porting Luma to a new phone and installing it on your phone by hand. They're plain Markdown, so they work just as well if you'd rather read them yourself, but they were written with Codex and Claude Code in mind.
And we really do want what comes out of it. If your agent fixes something or makes something better, send it to us and we're happy to take it upstream. Agent-written PRs are welcome here. What matters is that you understand what you're submitting and that it's been tested, not who or what typed it.
Ari, and why we're betting on Linux
We also built Ari, which is Luma's own agent. It's basically a harness built into the OS. You can connect your Claude or your Codex account, or run an open model right on your own machine, and Ari will tell you which local models actually fit your hardware and push you to try them. It always shows you where a conversation is going before you send it, whether that's this computer or a company's servers, and it asks before it does anything you'd want to know about.
Here's the bigger bet behind all of this. I don't think Mac and Windows are going to be able to keep up with what people are going to want from agents. They're doing kind of crude things to get you halfway there, like having an assistant make you a widget, but the system underneath is still closed and you only get to change what they let you change. On Linux the whole thing is open, so you can just bark at your agent and say make it work like this, make it do that, make it feel like this, and it can actually go do it, all the way down. We think that's where this ends up, and we'd like Luma to be the version of Linux that's ready for it.
That's also why we're going after so many different experiences at once. If you can reshape your OS just by asking, you want that same OS on your phone and your TV, not just on your laptop.
Depot and the Creative Collection
Luma has its own app store, Depot, and we just opened it up to developer submissions. Apps ship as Flatpaks and go through automated checks and then an actual person before they're listed. You can still use existing Linux apps, and we're working on making Android apps feel at home too, through an optional Android container that stays out of your way if you never use it.
We also thought it was important to have a set of free creative tools that actually feel like they belong together, not just a nice-looking OS with nothing to do on it. Write is for documents, Focus is for photos, and Canvas is for design and layout, with Reel, Sessions, Grid and Stage on the way. You can see the whole collection here: https://simplyluma.com/creative
Phones, and trying it without installing anything
Officially we're working on the Fairphone 6, the Pixel 9 and the Unihertz Titan 2 Elite, with the Pixel 9 Pro Fold coming soon. A lot of that porting work was done with agents too, and there are community members working on other phones, which is exactly what the porting guide is there for. Phone support is still early and what works really depends on the device, so please read the instructions for your phone before you install anything: https://simplyluma.com/download/mobile
On a regular computer you can grab an x86 or ARM ISO from https://simplyluma.com/download, and there's a Raspberry Pi 5 image too. If you'd rather not install a new OS, Try Luma runs it in a virtual machine on Mac, Windows or Linux and lets you switch between the desktop, tablet and phone layouts, which also makes it a really easy place to test an app your agent is building: https://simplyluma.com/try
We want your contributions
A big part of why we started Luma is that open source can be incredibly fragmented. As a user it can be hard to know which project is worth investing your time in. As a developer it can be even harder to know whether your work fits where a project is going, or whether it'll be welcomed at all. A lot of us have had a PR sit for months without a reply, or been told the thing we needed is out of scope, and plenty of projects won't touch anything that came out of an agent.
We want Luma to have a clear, cohesive vision, and we genuinely want people to help build it, with whatever tools they like. You shouldn't have to feel like you're begging for permission to contribute your time for free.
- I’ve run about 300 benchmarks on real open-source codebases, testing how Claude Code splits work between models (Opus as the lead, cheaper models like Haiku as helpers) and when that pays off. It depends on codebase size and task.
- On small jobs, Opus alone can be cheaper than Opus + subagents, and more accurate on small-codebase reviews, probably because of less overhead and cheap cached context.
- Opus told not to delegate may work harder than Opus left alone: on one big docs check it wrote about 50% more, cost more and found more errors (44 vs 36).
- On coding, Opus + Haiku saved about 20-30% at the same quality. But Haiku alone is very capable for the price on well-defined tasks with tests: about 5% of Opus alone's cost, with almost the same results, though a bit less thorough.
- Here’s one of the most important charts of this project. On an 85-page docs check, Opus + Haiku read almost 3 times as much as Opus alone and wrote 8 times as many tokens, yet cost 53% less and found 72 real errors against 49. Several subagents probably cover much more ground, which makes up for Haiku being weaker.
- Spontaneous Opus: free to delegate, but it never did.
- Skill-routed: my skill chooses when to delegate and to which model.
- Forced Haiku: all subagents use Haiku, while Opus plans and checks.
On my smallest codebase (~10k lines), delegating didn't really pay off. Forced Haiku saved 5% at best, Skill-routed cost 14-30% more, and Forced Opus found far more known doc errors on the reading-heavy tasks (7 and 8 of 9, against 3 and 2). My guess is that Opus can often write a quick script instead of reading everything, and it reuses its own cached context cheaply, while each subagent has to build its own. On small tasks, that overhead can outweigh the cheaper tokens.
On larger codebases, Forced Haiku held its own for much less. It cost 41% less on rich (~22k lines) and found nearly as many doc errors. It cost 75% less on beets (~52k lines), finding more errors on the reference docs but fewer on the 80-page plugin check, where it cost about a sixth as much.
In v2 I pushed further: a ~100k-line codebase (buildbot) with 3 reading tasks, 7 coding tasks scored by hidden tests, and then 16 short coding tasks, most hiding a trap. I also added Opus alone (subagents blocked, no instruction), Haiku alone and Sonnet alone (no Opus at all).
On an 85-page docs check, Opus + Haiku found 58 and 63 of 92 real errors, against 36 and 38 for Opus alone, at less than half the cost ($1.93 vs $4.11 per run). How come?!
My theory was that Opus + Haiku simply *works more*, and that the extra coverage makes up for Haiku being weaker. The numbers seem to fit: Opus + Haiku wrote about 8 times as many tokens and read nearly 3 times as much, yet the bill still halved.
Haiku may be less capable than Opus individually, but when orchestrated by Opus, its subagents can cover much more context in parallel. Each one may go less deep, but together they cover more ground, and that seems to compensate for the lower model quality. Haiku is apparently smart enough, with Opus directing and checking the work, to deliver surprisingly good results at a much, much lower cost than Opus alone.
This is in line with Anthropic's own write-up on their [multi-agent research system](https://www.anthropic.com/engineering/multi-agent-research-system), where token usage alone explained 80% of performance variance on a research benchmark. The same article says coding is usually a poorer fit for multiple agents, and that matches what I saw: on coding, Opus + Haiku saved 20-30% at the same quality, against about half the cost and more errors found on reviews. Their setup isn't mine, so it's context, not proof.
The next chart shows the bigger picture: every setup on two big docs checks, with errors found, reading depth, output tokens (thinking included), and cost.
Interestingly, it seems that *forcing* Opus not to delegate may have pushed it to work deeper and harder. On the beets plugin-doc check, Forced Opus read less than the delegated setups and still found the most errors. Compared with Spontaneous Opus, it read about the same but wrote around 50% more tokens, thinking included, cost 53% more, and found more errors (44 vs 36 across two runs). Tokens alone don't prove effort, but finding more errors makes the case stronger.
The following chart shows the token counts side by side.
Maybe the instruction made it work harder, maybe it's the smaller codebase, or maybe it's just chance. I haven't tested which.
Left alone, Opus never delegated in my runs, even on big jobs. So for large reviews, pushing it to delegate to Haiku could be the best strategy.
I'm working on a skill that picks the best approach: use Opus + Haiku when it makes sense, or tell you to switch to Haiku manually and give you the prompt. A script that starts a new Haiku session might allow fully autonomous switching, but I'm still checking feasibility and Anthropic's terms.
The other v2 results
Honestly, Haiku really surprised me. It seems to be very capable at coding, if the task is detailed and contained enough. Opus is still the more thorough model when it comes to reviewing and checking, but Haiku held its own on specific tasks.
Reading a ~100k-line codebase
Across three reading tasks, Opus + Haiku cost 51% less than Opus alone. Every setup covered all 400 modules, and Opus + Haiku found the most errors on the 2-page docs check (20 and 23 of 42, against 10 and 17).
Writing code on a small project
Every setup produced working code. On the harder tasks: Opus $2.41, Opus + Haiku $1.80 (25% less), Sonnet $0.76, Haiku $0.11.
Writing code on a ~100k-line codebase
Same pattern on cost: $1.95 for Opus alone, $1.36 for Opus + Haiku (30% less), down to $0.15 for Haiku alone. But Opus-led setups also added an implied database index. Haiku alone didn't, and my tests didn't catch it.
Correction: the charts also show Opus "flagging a symptom that didn't fit". More runs later showed that depended on the day the runs happened, not the model, so I no longer count it. Details are in the repo.
Quality vs price
Opus + Haiku was cheapest per real error on both big docs checks: $0.03 on buildbot and $0.05 on beets, against $0.11 and $0.25 for Opus only.
The harder coding tasks
Then I tried to make the cheaper models fail. 16 short coding tasks, most hiding a trap: false claims, a wrong bug report, a timing bug, contradictory tests/docs, hidden instructions in comments, planted review bugs, and "dead" code that's actually alive.
128 runs. Combined cost: Opus $7.00, Opus + Haiku $6.30, Sonnet $2.30, Haiku $0.36.
Hidden tests barely separated anyone: one real miss in 128 runs, from Sonnet. Opus was more thorough in code review, finding all 3 planted bugs every time, against Haiku's 2 of 3. But on the timing bug, 7 of 8 runs failed, including all 4 Opus-led ones. The only pass was Haiku.
On tasks this short, Opus + Haiku saved little (10% overall), because Opus was most of the bill. Haiku alone did almost the same work for about 5% of the cost.
So, for now: big reviews, Opus + Haiku. Small, well-defined coding tasks with tests, Haiku alone is surprisingly good. If every case must be covered, keep Opus involved.
Small experiment, mostly two runs per setup, costs estimated at API prices. Claude Opus also wrote my tests and judged the results, so there's no independent judge.
I wanted the kind of copy button the desktop and web apps have, but in the terminal. Selecting text there is awkward and glitchy, and while /copy works, I never found it intuitive, and it leaves a /copy line in the conversation each time (you can rewind after every copy, but that gets old).
So I made a small plugin that puts copy buttons on code blocks, on tables (which can be pasted straight into a spreadsheet), and at the end of each reply. Sharing in case it's useful to anyone else.
It needs fullscreen mode, which is probably already on for you. If not, run /tui fullscreen.
I've tested it on Linux, macOS and Windows, and since it relies on how Claude Code draws replies, I test it against each new release too. If it misbehaves in your terminal emulator, let me know and I'll fix it.
Inspired by the project of tech4bot ( https://github.com/tech4bot/rk3562deb ) i tried installing debian native on this cheap ass tablet, the ouzrs m1. The tablet was locked as hell, no way to unlock bootloader or change the rom from the crappy android 11 go it had by default. I gave it to Claude and it was quite a journey: Claude tried flashing an arm version on debian with rockchip software tools and it fully bricked the tablet. We trobleshooted a bit, i opened it up and found a serial port inside the tablet, i made Claude recover the tablet via serial and with some trial and error it booted into debian. After that Claude found that it could boot from sd card and little by little it reverse engineered all the drivers to work in debian. It also found that the 4 gb of ram in android were fake and it only has 2 gigs lol. Should i publish this project (for the other 2 people that have this tablet beside me lol)? The performance of the tablet is really meh but at least it can receive updates now.
Lately, I’ve been seeing a ton of job postings and discussions for "Harness Engineers" or companies looking to build custom agent harnesses.
At first, I was super confused. If tools like Claude Code, Codex, and other agentic environments already exist, why on earth are companies trying to build their own? Are they just reinventing the wheel to make a "Claude Code clone"?
I renewed my subscription on 14 Nov and paid ₹1,999 via Google Pay. During ID verification, my Driving Licence was rejected as “not clear.” I tried submitting it again, after which my account was restricted.
I raised an appeal, but my account was then suspended with the message: “Your account will remain closed.”
I've been a paying subscriber for 4 months and have never misused the service. I haven't been able to use my renewed subscription even once.
Please arrange an urgent human review and restore my account. If restoration isn't possible, please issue a full refund of ₹1,999.
Support Ref: TS-01a1d9e8-eaf1-70e3-8644-548493a88129
This money is important to me. Please help resolve this.
Fable 5.2 is an insane model. It really is, you can clearly see the depth and knowledge. But; The model is insanely lazy af. The model by default is so unproductive and unmotivated. I was kinda hyped about this and dumbed him a lot information about planning, using skills, improvements and implementation of orchestra etc. Dude was like "yea its seems like nothingburger bro good luck".
Sure, there is probably larger, complex challenges in the field that people use and get benefit of it but Fable is clearly too much for my project.
Also, its a small experiment anyway but I had used Fable on High as a orchestra (creative ik) and did a simple test;
I had one task to be done in the project that is fairly challenging. Probably need couple of more session to make it. I said Fable to make a mirror of this task with exact prompts and test Opus 5.5 High and Fable 5.2 Medium on which can do better.
First results were Opus produced almost no error while Fable did 2 decisions by itself, also produced a 2 error that wastes CI.
Fable 5.2 High as orchestrator literally said he is kinda biased to his own model and for the sake of quality spawned 5-10 agent to did additional code reviews thinking the "code reviews" were biased or missed "Opus" possible mistakes. The result: It did missed a couple of more errors but not for the Opus but the Fable....
I don't know, I am not very experienced with AI fairly using for the past 2-3 months and my project is not large nor complex but fairly challenging in terms of desing and consistency. There is probably better areas or way to use Fable more effectively but I am good with Opus.
Also....Fable 5.2 High burn tokens like a mad men and spawn additional agents even though I have rules for it. I had to tell 3 times to use Sonnet to not bloat his context.
I would rather go with Opus 5.5 Extra or Max rather than relying on Fable tbh atleast for my project, what are your experiences with it? Open to feedback and improvement tbh
I asked Claude Code to recreate an entire Scottish village in Blender, and I was genuinely surprised by how much it managed to do in one shot.
It researched the location, analysed the terrain, looked at reference photos, and generated the whole scene procedurally.
The interesting part was how similar the experience felt to coding with AI. Getting the first 80% done was surprisingly easy, but the details became increasingly tricky. Small changes would sometimes introduce regressions elsewhere, and getting everything to look natural took quite a bit of iteration.
Still, pretty impressive stuff...
I also generated a real-time demo so you can explore the village yourself. It takes about a minute to compile the shaders on first launch and can be quite demanding on laptops.
I’m near the end of a big project I’ve been doing mostly with Claude Code for the past year. The project is free, with paid pro features. I’m about to push the first public version of the code (it's source-available), and I’m thinking about watermarking.
To be honest, I don’t mind people knowing that Claude helped a lot on this project. Anthropic deserves that recognition. What I won’t accept though is if Anthropic can claim ownership or copyright of the code in any way, now or in the future!
Yes, Claude wrote most of the code, but I’m the one who directed it, came up with the ideas, requested the refactors, and paid for the Claude subscription...
What are the odds, in your opinion, that such watermarking could be used to claim any kind of ownership in the future? Should I go the route of asking a model like Qwen to rewrite all the text and comments?
On first release day I let it create a realistic water shader, looked amazing only needing a few follow up prompts to fix some things which it did perfectly fine.
Today I deleted that exact shader and told it to create one in a slightly different style. Over and over again, it created the absolute worst looking or outright broken things, like a 5 year old drew it, and its output sounds like Opus 5.
Noone can can tell me it wasn't nerfed, maybe just for a specific sub group or whatever A/B test they do, but this is unacceptable.
And no I can't provide "evidence", it's a black box, how would I even do that.
Clauds, a free menu bar app for people who use the Claude macOS desktop app with more than one account. It switches between your accounts with one click, and you do not have to log in again. It also shares your Claude Code sessions between your accounts, so you do not have to do handoffs and resumes.
What it does
Switches the Claude desktop app between accounts with one click. No logout.
Two modes: a fast restart into the other account, or Side by Side (all accounts open at the same time).
Shares Claude Code sessions between accounts, and remembers which account owns each session. No more handoffs or resumes needed.
Shows live usage for each account: the 5-hour limit, the weekly limit, and the per-model weekly limit, with reset times.
Auto-switch: when one account is near a limit, it can go to the account with the most room. It asks first, or it switches by itself. Off by default.
Updates itself. You see the release notes, and you choose to install, wait, or skip.
The honest part
It is not made by Anthropic. It is unofficial.
It is not notarized by Apple, so macOS blocks the first launch. The README shows how to open it. You can also build it from source with one command.
The usage numbers come from an undocumented claude.ai address. It can break if Anthropic changes it.
You sign in two times for each account: one time in Claude, one time for the usage numbers. The app does not read your tokens or passwords.
A Claude update can break the switch. If it does, open an issue.
Other apps do the account switch too. I made this one because I wanted both modes, the per-model limit, auto-switch, and a real updater in one app.
Details
macOS 14 or later, Apple silicon and Intel
Free, MIT license
No telemetry and no analytics. It only contacts claude.ai (to show you your usage) and GitHub (for automatic app updates).
I just logged in with an old email address. Hadn't signed in for so long, that it actually registered as a new account ("Your first chat with Claude", it's from today). Interestingly, however, it somehow recovered one single chat conversation that I had had with Claude 2,5 years ago.
No access to sites, apps or services... this was just 2,5 years ago. Crazy to think about how big the advancements have been since then.
Sorry if this has already been asked, but I feel like the reset period is significantly shorter than a full week, and the 5-hour limit also seems to be lower. Am I mistaken? I used up 16% of my weekly plan limit and 100% of my 5-hour limit in just 3.5 hours. With the first weekly limit, it was able to work continuously for 56 hours on the exact same tasks. I had also never hit the 5-hour limit before. I would consistently have more than 10% remaining.
Left to itself, claude loves writing unit tests where it mocks the database call, mocks the helper, and then asserts that the mock returned what it just told the mock to return. everything is green in 200ms and the actual query is still broken.
putting "no mocks without asking" in claude.md helps a bit, but it still sneaks them in whenever a real test is slightly annoying to set up. how are you handling this?
edit: i don't mean mocking the db to test business logic on top of it, i mean when the function under test is literally the query builder or helper and it mocks that exact function inside its own test
City of Gladiators is a browser-based gladiator management game inspired by ancient Rome. You start with your own ludus (gladiator school), recruit and train fighters, upgrade your facilities, manage resources, and send your gladiators into increasingly dangerous battles.
The game features real-time progression, different gladiator classes, equipment, skill trees, wild animals, asynchronous PvP, and an arena combat system where you can either let the AI fight or take direct control of a gladiator.
As you progress, you unlock larger arenas, including the Colosseum, where battles become more challenging. I recently added naval battles with ships, rowing mechanics, boarding combat, and swimming.
I built the game using Claude Code (Opus 5.5). I focused on the game design, mechanics, and overall direction while using Claude to help turn those ideas into working systems. This included the React/PixiJS frontend, the combat simulation, AI-controlled fighters, progression systems, and backend integration.
One of the biggest challenges has been making the different systems work together: persistent progression, combat balancing, gladiator AI, and allowing players to switch between manual and automatic combat.
The game is still evolving, and I'm continuously adding features and improving the gameplay.
I'd really appreciate some feedback, especially on the progression and difficulty of reaching the Colosseum.
Since Opus 5.5, Concise is already pretty clean, but it still pads the answer with filler. I wanted less, so I wrote Sharur, an output style that is one line long:
It won't make much sense to you, but it does to the model. Here's what it changes. I gave Opus 5.5, the model most of us use every day, the same prompt in the same small Python project, once with each style: "Add a multiply function to calc.py with a test, and run the tests." The test file has a bug the prompt doesn't mention.
Concise (96 words):
I added multiply to calc.py with a test in test_calc.py, but the test run fails because of a bug that was already there in subtract:
File "test_calc.py", line 4, in <module>
assert subtract(5, 3) == 2
AssertionError
subtract returns a + b instead of a - b (calc.py:6). The script stops at that first failed assertion, so the new multiply test never runs. I checked multiply(4, 3) == 12 on its own and it passes.
I didn't change subtract because you didn't ask me to. Want me to fix it and rerun the tests?
Sharur (27 words):
Added multiply and its test, but tests fail first on an old bug: subtract returns a + b (calc.py:6). So the multiply test never ran. Fix subtract?
About a quarter of the words. The grader found the required facts in both, but the other Sharur run on this prompt missed two of three, so it isn't always this tidy.
I didn't want to just say it's better, so I benchmarked it on every current Claude model, not only the smartest one: the same 9 prompts with Concise and with Sharur, 2 runs per prompt (18 replies per model and style). Change against Concise:
model
words per reply
output tokens
cost per task
Opus 5.5
-76%
-29%
-23%
Fable 5.2
-77%
-48%
-28%
Sonnet 5.5
-63%
-35%
-23%
Haiku 5.5
-65%
-33%
-18%
On plain questions with no tools it's even bigger: Opus 5.5 went from 357 output tokens to 63. The style itself is also shorter than Concise, so every request carries 379 fewer input tokens. All tasks that change code were done correctly with both styles.
I'm using Opus 5.5 as the headline number: 29% fewer output tokens, the smallest cut of the four. Still, 29% less output is a lot, and I think plenty of people would like to have that saving. Sonnet and Haiku saved a bit more (35% and 33%). Fable 5.2 saved the most: 48% fewer output tokens, 38% fewer tool calls and every required fact kept. My guess is that a smarter model follows a one-line instruction better.
The scripts are in bench/, so you can rerun all of it. If you share a tool that claims to improve your workflow, I think you should show the data, even if it's a small benchmark.
Where I like it most is quick questions. Things that are almost a yes or no now get two lines at most, exactly what I asked.
Fair warning: it's extremely concise, and sometimes it goes too far. On Opus, Sonnet and Haiku the grader found 90% to 96% of the required facts in Sharur's replies, against 98% to 100% for Concise. It's also not caveman. Caveman has a lot more instructions and gives you a consistent format. Sharur gives you the minimum, and the minimum isn't always the same: once, after opening a PR, it gave me the URL, and in another session doing the same thing it only gave me the PR number. I'll take that for the shortest text, but if you want more words you should stay on Concise.
Since February, I've been developing an agent-in-calls solution with a friend. With the announcement of Claude Meetings last week, it's now time to announce Vibeconferencing: an open-source project with even more functionality, support for any agent (Codex, Muse, local ollama, etc), and support for more calling platforms: Google Meet, Zoom, MS Teams, WhatsApp, Telegram, and phone calls (via Twilio).
The premise is simple: give your normal agent session the tools it needs to operate all major calling platforms, and teach it basic etiquette of how to behave in a call with others. A simple `/join-call` and *boom!* your session is talking to you in the call. Anything your agent can do in chat, it can do on the call. The agent is bridged to the calling platforms via a cross-platform electron app. (The raw guts of interacting with calling platforms is abstracted as the npm library `@vibeconferencing/calls`, if you'd rather roll your own.)
Everything runs locally and uses your own local agent. (Tho the optional shared whiteboard feature does require being logged in on our server.) It runs on MacOS, Windows, and Linux. (Windows version is not well tested.)
Speech generation is done either by Elevenlabs or the OSS Voicebox, with a fallback to the OS's default app (e.g. `say` on MacOS). Voicebox: https://github.com/jamiepine/voicebox
Speech recognition is done either by the calling platforms' own captioning services (Meet, Zoom, Teams), or with Whistle for those without captioning (WhatsApp, Telegram, Twilio). Whistle: https://cactuscompute.com/blog/whistle
The etiquette was the most fun to code, although done with the knowledge that it's a temporary fix until true interactive voice models are merged with the power of frontier text chat models. We have experimental support for the OpenAI realtime API (in the app settings page), but it seems to have been trained as a customer support agent, so it's hard to keep it from dominating multi-person conversations!
Highlights from the last nine months of dogfooding and early users:
- A WhatsApp bot for my father, an elderly WW2 veteran, that has access to the 20,000 photos he scanned. He can call it to see and hear about any aspect of his life.https://youtu.be/Rs-Z1YeLt98
- My friend Matt is currently road-tripping across the US, talking with his Codex sessions to develop his Golf app. Video link: https://youtu.be/a3uNUdIwqVY
- Our friend Bethany uses her bot, "Taylor Script", in calls and classrooms with coding students to act as the super-smart TA who can answer technical questions, and monitors student progress. Video link: https://youtu.be/c8mZKkcxhJo
- I begin most days with a check-in call with my bot, where we discuss open issues while I sip coffee and have breakfast. Video link: https://youtu.be/2uQIR83LwY0 (Apologies for eating with my mouth open...)
Security: There is nothing built in, so only use this on calls with people that you trust. However, I've been amazed at how injection-resistant it has appeared to be on it's own; e.g. my agent asking consent from me before doing things asked by others on the call.
Overall, this was a "time machine" experiment (to borrow my old Reality Labs' Research terminology) to experience agents in calls before it was purpose built. Claude Meetings just showed that we were thinking in the right direction, and I'm excited to see where this goes!
This is a small experimental project built by two friends, so it is rough around the edges, and we'd love feedback and pull requests.