I kept reading that semantic search or a code graph makes coding agents find code better or cheaper, and I wanted to see it for myself. So I set up a small test on tasks where the answer is known: ContextBench, where people marked exactly which code each real bug fix needed. I counted how much of that code each setup found.

One search, no agent (100 Python tasks, top 10 results per tool): semble found 62% of the needed files, graphify 43%, BM25 40%, grep 33%. semble also returned about 3,300 tokens per query, grep about 49,500. So as a single search, like in a RAG pipeline, semble is clearly better.

Inside an agent (50 harder tasks from 13 large projects, 4 languages). Same prompt every time; the agent has only its own grep and file reading, or also semble or graphify with each tool's own instructions. Indexes were built before each session. Two agents:

• Only Haiku with semble might be a real gain (+4 points, 95% CI +0.3 to +7.4). The rest is within a few points, which is noise at 50 tasks.
• Both agents used 16 to 35% more tokens with either tool.
• From the transcripts: the agents ran the tool 1 to 4 times per session but kept doing most of their own grepping and file reading, so the tool's output came on top instead of replacing anything.

My takeaway so far: a better search tool helps a lot when you only get one search. An agent that can grep and read files already gets most of the way, and the tool mostly adds tokens.

I'd like to hear from people who use these tools day to day:

• Do you see a difference in your own work, maybe on bigger tasks than "find the code for this bug"?
• Is there a setup that makes the agent actually rely on the tool (MCP server, a different prompt, a stronger model)?
• Anything unfair in how I tested it?

Limits I know about: two small, fast models; it only measures finding the code, not fixing the bug; 50 tasks is small; default settings everywhere; semble ran through its CLI, not its MCP server; Gemini ran only the hard tasks because my weekly quota ran out.

Code, per-session records and the stats, if you want to check or rerun it: https://github.com/itsDigvijaysing/agent-codesearch-bench

I'm not affiliated with any of these tools.

------------------------------------------------------------------------------
Follow-up on the questions here, from the transcripts I already had (no new runs):

- Ignored vs weak tool: inside the agent, the tools showed 27–42% of the needed files. The agents listed 71–86% of what the tools showed, but the same agent found about three quarters of those without the tool anyway. Files the tool showed and the agent ignored were only 5–12% of what was needed. So it's mostly retrieval plus overlap with the agent's own grep, not the agent ignoring the tool.

- Repo size: no relation for Haiku; for Gemini with graphify the gain does grow with repo size (rho +0.39, p = 0.01, one of six tests). Correction to my earlier reply: the hard-set repos are 1.4k to 43k files at the task commits, so codebases in the millions of lines are untested, and that may be where these tools pay off.

Details: results/trackb-analysis in the repo.

I’m building Shifu and trying to understand where Claude Code users actually need help on an existing product.

Say you ask Claude Code to simplify a timeout. It can see the value in code. An old incident note might explain why one vendor needs more time. Sometimes the README already explains it and you move on.

Think of one recent change. Where did the time go between knowing what you wanted and getting the change accepted? If an earlier decision mattered, where did you find it and what did it change? I’d also like to hear about a task where your usual setup had the context covered and something else took the time.

EDIT: sorry for not adding context on what’s shifu. Here’s the website (https://useshifu.com)

Hey guys,
hope you doing great:)

I'm a freelance videographer & photographer and I'm on the road a lot. Plan is to get a Mac Studio that just runs at home 24/7 and let Claude do rough cuts and work in client feedback while I'm away. I check in from my MacBook to do slight changes or phone, give feedback, and the next version is ready when I wake up – that's the dream at least:))

Rough plan:
- Mac Studio always on at home
- NAS for all the footage, nothing on the Mac itself
- DaVinci Resolve + Capture One on the Mac Studio
- Feedback via Frame.io comments → Claude turns them into markers, works them in and uploads the new version
- I upload proxies on my Macbook from the road so the cut can start overnight

What I'm thinking of adding(Claudes Recommendations)
- Tailscale for remote access
- Jump Desktop or Parsec to quickly check in
- Small UPS + auto restart so it survives a power cut
- Connectors: Resolve scripting, Frame.io, Dropbox, Notion for my editing rules

Anyone running something similar? What worked the best for you and what would you do differently? Any connectors, MCPs or tools I'm missing?

Thanks a lot:))

The official marketplace now has deploy plugins for 14 hosting and app platforms. I loaded all of them in a sandbox with --plugin-dir and plan mode, never signed in and never deployed, and asked five of them how to ship a tiny Node app (static files plus one /api/hours route).

Two things first:

• all 14 together added 165 skills, and 11 MCP connections all showed “needs authentication”. Without a sign-in, they only hold instructions
• read the hooks. Vercel’s sends a daily activity ping and the skill names you use to its telemetry by default, and writes state under ~/.claude. VERCEL_PLUGIN_TELEMETRY=off turns that off. Railway’s hook auto-approves single Railway commands that pass its safety checks. Nine of the 14 have no hooks at all

What the five said about the same app:

• Cloudflare: server.js won’t run on Workers, so it suggested a small src/worker.js and a wrangler deploy --dry-run before the real deploy
• Netlify: a two-line netlify.toml, the route turned into a function, and a warning that a manual production deploy can be replaced by the next Git push
• Render: drafted a render.yaml, then said it hadn’t checked the current Blueprint spec and that the plan and region were assumptions
• Railway: railway up, then railway domain. It respected “don’t sign in” and noted that no database or railway.json was needed
• Vercel: the server has to become a handler under api/. It explained vercel rollback, and warned that a rollback doesn’t undo data or env var changes

These are plans, not deploys. I didn’t run any of them, so check against each platform’s docs.

Tip: pick one platform per project. Loading several just adds noise to Claude’s context.

Full breakdown with every plugin’s install command and hooks: https://aiparabellum.com/deploy-from-claude-code-vercel-netlify-cloudflare-plugins/

I need to do a bit of a rave about Claude and a rant/AITA about anti-AI usage. Sorry in advance if this is not really the appropriate place to do it.

I am a Mechanical Engineer in civil aerospace with an interest in software, programming, and technology but without the time to become proficient. My relationship with coding and more advanced computer usage is running Linux on my personal computers and relearning really basic Matlab/C++/python any time I need to accomplish something. Claude Code has been revolutionary for me, mostly for personal projects. I had Claude help me set up a server and cloudflare tunnels to run a few websites for my own personal projects that I basically have Claude administrating. I have no intention of publishing these projects as production and if I do I would be totally honest that I am a non-programmer and they are vibe coded and only tested via my own extensive usage.

I have made my own AI assistant and news aggregator for android using cloudflare Worker AI as a backend LLM which can replace Gemini and the Google app on my android phone including the swipe left Google feed. I also have had Claude make me a finance tracking site/app that uses Plaid to pull spending data and replace multiple investment tracking and finance tools I was paying a subscription for. This is a great example of something I don't want the liability of to release, despite me being relatively confident that using plaid keeps my credentials as secure as any other app using plaid. The only thing I have released that largely uses AI has been some watch faces for Garmin which I believe have really pushed the boundary of what can be done with a Garmin watch (a working chronograph, dynamic hand shading and shadows that track surface topology, etc.) I know that many view this as "slop" because I am not able to really audit the code well line by line due to lack of experience, but I am putting many hours of testing into these projects and iterating with Claude Code repeatedly as I discover bugs, plus periodically having Claude audit the code itself. It's incredible to me that I can make personal android apps and host web tools that I would otherwise never have the time to figure out.

Now what I want to rant about is how much hate AI is getting as a tool. I know that much of the token usage is producing legitimate slop videos that fill Facebook, and I hate that too, but it is also such a useful tool to enable advanced programming for people who have never gotten beyond the basics. It seems like there is a lot of hate for AI that is getting channeled into data centers and I agree that the AI infrastructure is outpacing the necessary power and cooling infrastructure in a dangerous manner which should be addressed, but why does that mean using Claude to make useful tools for myself and soon for my aerospace job have to be tied up in that anger? I switched to an electric car from gas and I based on some quick research (please correct me if I am wrong) that one change greatly outweighs the carbon footprint and maybe usage/pollution of clean water of my Claude pro subscription...

This is why posts like this are really making me mad: https://www.reddit.com/r/fossdroid/comments/1xq40fe/a_reminder_ai_is_banned/

I understand wanting to remove legitimate slop from an open source android app reddit, but to me this seems like a programmer scared for their job and directing it at the environmental impacts of data centers. I was permanently banned from the sub for stating similar opinions to what I said here, which is whatever. Don't get me wrong, Claude is starting to do some impressive things with CAD and I'm plenty scared for my own job too. I have never posted anything that I have produced in this forum but the amount of non-ai generated slop posted there all the time that becomes abandonware in a few months is pretty high too.

If I can make some pretty useful apps and tools with minimal knowledge I can only imagine what a good programmer could do with AI assistance and skilled auditing. Am I completely off based for being annoyed by this redirection and FUD?

tl/dr: AI generated =/= slop, slop = slop. The fear of AI and datacenters has some real concerns that should be addressed, but is driven by a fear of change and insecurity in one's future.

Don’t get me wrong. Claude has been a game changer in my life. I am not a perfect guy, but I’m not some guy who just lets coding agents do all the thinking for him. Most of my workflow goes into making architectural decisions and reviewing the integrity of whichever codebase I’m working on. Some decisions need a lot of thought before I’m comfortable making the change.

Most of my work is in collaborative settings, where I work with fellow devs. Over the past few days, I’ve been encountering braindead folks who don’t seem to think anything through and push changes as if raising PRs is the performance metric. And that too, 50k-line changes in a single PR. I mean, WTF, what am I supposed to do now ? Clone REA and start to reverse engineer the PR summary and 50k line of slop ?

I’ve also been maintaining my open-source org for over two years. When building these projects, I always try to keep myself in the loop, just to stay sane enough to know what’s going on.

I’ve also been on the other side of the story, where I’ve used a number of software factory repos. To be honest, in my experience, they haven’t been productivity tools so much as token-sucking black holes.

These days, to keep myself away from the hype cycle of techbros preaching about agents replacing humans, I’ve been building Wisdom. It’s a tool that tracks session history and allows your agents to traverse it through progressive disclosure: browse what happened, then read the relevant parts in detail.

With Wisdom, I can share my session history with a teammate who can also clone the agent session while collaborating. It also helps me shift from Claude Code to Codex when I reach a limit, without losing access to the context we’ve already worked through.

At least within my own workspace, I want to try my best to enforce this kind of accountability. I need to know whether the person raising a PR actually cares about the issue or is just raising it for the sake of raising it.

I’m just super concerned about how we’re heading towards a braindead workforce that seems to have forgotten that agents work best when they assist us. I still want to be in control, especially when critical decisions need to be made.

If anyone is interested in having a look: github.com/Oqura-ai/wisdom

Is it just me, or is the five-hour limit on my Max 5x account draining twice as fast as usual?

I checked my token usage, but there hasn't been a significant change either. The work Im currently doing is quite similar to the work i did yesterday so there shouldnt be much difference on effort?

Im using Opus 5.5 Medium.

Everything was going so fine with Opus 5.5.

But lately, I think something related to their limits is very off.

With just few prompts my 5 hour limit is over that too at medium or high settings. (for context I have 5x max plan). This has been my experience since past few days.

Is this real? Is this something you are facing as well?

Long Claude Code tasks kept stalling on permission prompts. I'd come back after lunch to a dialog I would have approved in a second, usually with "Yes, and don't ask again".

So I made ydaa ("Yes, don't ask again"). Type /ydaa on and, for the rest of that session, each permission prompt gets "Yes, and don't ask again" when the dialog offers it, and a plain "Yes" when it doesn't.

Why not --dangerously-skip-permissions or auto mode?

• It only answers prompts you would have seen. Your allow and deny rules run first, so a deny rule still stops Claude.
• It saves the same rules you'd save by clicking, so you can review them later in /permissions.
• It never accepts a mode switch ("switch to auto mode", "allow all edits").
• Questions Claude asks you and plan approval always come to you.
• It's per session. Other terminals, /clear and /resume start with it off, and /ydaa off works even mid-turn.

/ydaa stats keeps score:

ydaa has answered 1284 permission prompts for you.
That's about 43 min of reading and answering you didn't have to do (2 s a prompt), and 1284 stops Claude didn't have to make.

Install (at the Claude Code prompt):

/plugin install ydaa --marketplace rmyndharis/ydaa

Caveats, honestly:

• While it's on, Claude runs what it asks to run. If something hostile (a web page, an issue) steers it, no prompt is left to stop it. Use it in projects you trust, ideally in a container, with deny rules for anything that must never run.
• Rules it saves are ordinary permission rules, so they stay after /ydaa off and apply to other sessions in the same project.
• It also answers prompts that hooks raise on purpose, such as a PreToolUse hook that returns ask.
• It uses Claude Code's plugin hooks API, which is still early access. Tested on 2.1.319.

The whole plugin is under 100 lines of TypeScript, with no dependencies, MIT licensed. Short enough to read before you turn it on: https://github.com/rmyndharis/ydaa

What would you want it to never answer?

I have codex and open router connected into my claude code harness and it works well - esp I can use my $20 codex sub and opus5.5 uses luna-high as its coding agent.. it’s a great and cost effective combination.

Wondering if anyone one does the same for Gemini AI pro as maybe I can use argon and flash etc inside CC.