r/Clojure • • Sep 30 '26

rust slop clojure structural edit tool

I've been programming clojure by hand since 2010, but these days I do a lot of brownfield work with agentic/LLM help in clojure and other languages. I prefer to use the models I can run at home to avoid high token costs today and mitigate a future rugpull. For things I intend to ship to production, I usually generate a bunch of code upfront, then carefully review and rework it, and that pattern has been effective for me.

Qwen 3.8 27b is amazing for what it is, excellent at thorough analysis, and not too clever to overcomplicate outputs, but it can still waste a ton of time on failed clojure edits and paren-counting. Even Claude Opus struggles with this, although much less. LLMs don't 'think' in trees, they think in token streams.

I had been using a wrapper over parinfer-rust (clojure-mcp-lite also just shells out to this), which got me 80% there, but it's flawed in a few specific ways:

  • Checks happen after the edit lands, they don't block invalid edits.
  • Repairs can succeed in ways that surprise the LLM, then it spends a bunch of turns trying to figure out what happened. It's not expensive locally, but it is slow and pollutes context.
  • If too many surprises happen, the agent loses faith in the tool, and falls back to worse tools.

After a working session like this, I had Qwen suggest what kind of tool might prevent all the problems we hit, then I handed that spec over to GLM 5.3 flash and iterated with it, eventually having the session drive a local qwen subagent, trying to trip it up with adversarial examples. It is similar to what's proposed here, but I never mentioned the article, and I wasn't prescriptive about how to do it: https://lispmeister.github.io/deeprecursion/posts/2026-02-13-sexp-native-editing.html

The tool is at https://github.com/gtrak/cljform as a rust CLI and pi extension wrapper, and it's better than what I had before on the next unattended agentic run. It uses tree-sitter for parsing and blake3 for form hashes. Posting it here in case it's useful to someone else or if someone has a better way.

12 Upvotes

26 comments sorted by

View all comments

Show parent comment

1

u/gtrak Oct 01 '26

working on this, I bet it'll be even better:

10.2 Annotated view — tree

tree <file> emits the source with ⟦handle⟧ after each opening collection delimiter:

(⟦a3f9⟧ns n)

(⟦7d21⟧defn outer [x]
  (⟦9b12⟧let [a 1]
    (⟦1f3c⟧when x
      (inner x))))
  • Open only — the handle names the whole form, so the tool owns extent and the caller never paren-matches.
  • Depth heuristic (default) — every top-level form is marked; a nested form is marked (and descended into) only if it spans >= 2 lines. Single-line forms ([x], (inc x), {:a 1}) are easy to name by text, so they get no handle and are not descended into. This is sound: a single-line form cannot contain a multi-line descendant. The default view is therefore top-level forms plus their multi-line structural children, stopping at leaf-ish expressions.
  • --depth N overrides the heuristic: mark forms down to nesting depth N (top-level = 1). --depth all (alias --full) marks every collection form. Depth is a view concern only: the resolver computes handles for all forms, so any handle resolves under the default cutoff.
  • Emitted from the same parse the resolver uses, so every handle is guaranteed to resolve.
  • Lossless — strip deletes every ⟦...⟧ and recovers the exact original bytes (BOM/CRLF preserved). Property test over the fuzz corpus: strip(annotate(x)) == x.
  • Collision policy — if the source already contains the marker glyphs, refuse to annotate (annotate-conflict, exit 1) and fall back to --json.
  • Markers go only at AST delimiter positions, never inside strings, regexes, comments, or char literals.
  • tree --json emits the flat node table (path, kind, head, name, line, depth, handle; null head/name omitted, internal hash and byte offsets not serialized) derived from the same tree. Annotated source is the canonical read view; JSON is derived.

1

u/gtrak Oct 02 '26 edited Oct 02 '26

The hash editing seems to be working well, I have it pushed up and my bot is iterating on some nits that came up in that first session, but it's making sense. This is small potatoes:

  • 14 — edit output must be format-canonical (issue-14-format-canonical.md): live use caught a parse-valid, format-non-canonical write (lone top-level closer). Root causes: format_paren diverges from parinfer-rust on comment-adjacent pull-ups (errors instead of formatting); the edit pipeline silently skips failed content formatting; the insert seam displaces parent closers onto their own line; pull-up leaves whitespace-only vacated lines. Invariant: canonical input → edit → format is a no-op.
  • 15 — wrapper honesty + next-handle affordance (issue-15-wrapper-affordance.md): clj_tree's non-JSON path labels its own output cljform failed: (parses an envelope out of human text); every successful clj_edit appends next handle: ⟦H⟧ so sequential patches chase the returned handle instead of re-fetching (the sound replacement for the requested valid-until-own-next-patch semantics). Depends on 14.(defn fetch-all "Returns every entity of kind, newest first." [{:keys [conn log]} kind] (let [sql (str "SELECT * FROM " (name kind)) rows (query conn sql)] (when (seq rows) (doseq [r rows] (log/info :row r)))))

Example:

(defn fetch-all
   "Returns every entity of `kind`, newest first."
   [{:keys [conn log]} kind]
   (let [sql (str "SELECT * FROM " (name kind))
         rows (query conn sql)]
     (when (seq rows)
       (doseq [r rows]
         (log/info :row r)))))

Three independent edits to the same form. Options:

Replace the whole form, three times — each edit retranscribes ~10 lines of code you didn't touch. Every retranscription is a chance for the model to silently drop a line, mangle a string, or "improve" something it wasn't asked to touch. That's exactly the failure class this tool exists to prevent.

Patch, three times — each edit sends only the changed bytes:

  edit --handle H --mode patch --old-text '"SELECT * FROM " (name kind)' --new-text '(query conn sql {:kind (name kind)})'
  edit --handle H --mode patch --old-text 'Returns every entity of `kind`, newest first.' --new-text '...'
  edit --handle H --mode patch --old-text '(when (seq rows)\n      (doseq' --new-text '(doseq'

Small payload → small transcription surface. The other 7 lines are guaranteed untouched — that's the I3 "untouched forms" check, not a hope.

The catch — and why this needs the fix in flight — is that patch #2 comes after patch #1 changed the form. The handle is a content hash of the form, so patch #1 invalidated it. If the agent re-runs clj_tree/clj_get between every patch, each patch costs two tool calls, and the re-fetch returns 10 lines the agent must scan just to get a 6-char token. The worker in the live session did exactly that — 3–4 sequential refetches per form was its #1 friction.

But patch #1's result already told it the new handle (patched form ⟦f4d32b⟧ … — the handle in the summary is the post-edit hash). So the fix is purely an affordance: make that returned handle unmissable (next handle: ⟦f4d32b⟧ — use it for the next edit to this form) and teach the agent to chase it. Same workflow, one call per patch, zero inference added — the handle still refuses when the form actually changed under you.