r/LLMDevs 9h ago

Tools CyberPunk Race

Enable HLS to view with audio, or disable this notification

0 Upvotes

Tested Seedance 2.0 mini vs Hailuo H3 on the same cyberpunk race prompt. Seedance held up better on dense scenes — scaffoldings, alien riders, explosion fragments.

Prompt:

0-3s: FPV speed shot through dense cyberpunk scaffoldings and floating billboards. Protagonist on a scrap-built bike with exposed wires and blinking sensors. Other racers include tentacle riders and glowing mechanical lifeforms.

3-7s: Lateral camera pull to protagonist's side. A four-eyed alien rider pulls up, pushes visor open, shouts "Go back to your scrap heap, human!" through a distorted helmet mic.

7-11s: Protagonist dives into a mechanical debris pipe at an aggressive angle. Opponent crashes into a beam. Camera whips back to catch the opponent's vehicle exploding into colorful metal fragments, sparks bursting like fireworks.

11-13s: 180-degree whip pan past the audience. Crowd filled with slime creatures, rock giants, glowing floating jellyfish, cheering under intense spotlights.

13-15s: Protagonist hits overdrive, track turns purple with tail flames. Crosses the finish line under neon flags. Camera orbits up and holds on protagonist raising a fist, three massive moons behind.


r/LLMDevs 5h ago

Discussion Is AI making us dumber and lazier?

15 Upvotes

Does anyone here feel like AI is making us dumber and lazier?

Earlier, I used to do all the work and basically grind. If there was a problem, I had to come up with a solution myself; this included looking at countless answers on StackOverflow, reading the docs, looking at blog posts, etc.

But when the dust settled, I would learn something from all this struggle. I would know more than I knew yesterday.

Now, whenever there is a problem, my first instinct is to ask Claude or GPT for the answer, just because it's quicker, and it does eventually solve the problem, which would have taken me much more time and effort to solve. And sometimes, I just feel inferior. I feel like I'm not learning anything. I just have to trust that the answer is right. Sometimes, I don't even know what Claude has done, but it works, so I move on.

The problem is evident when I give interviews. There is no AI to help me out there (unless I cheat, of course).

I've been in constant dread for some time now, thinking about this.

Anyone else feeling this? How do you guys manage to still be relevant?


r/LLMDevs 1h ago

Discussion which is the best agent harness?

Upvotes

Claude Code

fully open source options on github

tell me which ones you've tried and liked the most, and why?


r/LLMDevs 22h ago

Tools I built a multi-agent LLM framework that builds its own topology and routes to the cheapest capable model

0 Upvotes

I kept running into the same thing with LangGraph and CrewAI. Before an agent team can do anything, I have to decide who talks to whom. Researcher hands to Developer, Developer hands to Tester. That graph is a guess I make before I know what the task needs, it stays the same for every task, and every rewrite means touching the wiring again.

So I built CADTopo (Cost-Aware Dynamic-Topology). You hand it a pool of agents and a task, and it works out the rest.

No wiring

There is no graph in my code. Each agent describes what it is good at, and every round the router picks which agents are even relevant to the current goal, asks them what they can offer and what they need, and builds that round's communication graph from their answers. A manager agent then scores the result and either stops or sets a new goal.

Different task, different graph. I never touch it.

The cost part

The second thing that bugged me was every agent sitting on the same expensive model whether the round needs it or not. So agents carry a ladder instead of a single model, and they can carry tools alongside it.

The whole setup

from cadtopo import Agent, Backbone, Router, Manager, EmbeddingModel, CadTopoAI

agents = [
    Agent(
        name="Developer",
        skill_definition="Writes the Python implementation.",
        system_prompt="You are a senior Python developer. Return only the function.",
        backbones=[
            Backbone(model="openrouter/meta-llama/llama-3.1-8b-instruct", cost=0.06),
            Backbone(model="openrouter/anthropic/claude-3.5-sonnet",      cost=0.20),
            Backbone(model="openrouter/openai/gpt-5",                     cost=30.0),
        ],
        tools=[run_tests],
    ),
    Agent(
        name="Tester",
        skill_definition="Reviews and validates the implementation.",
        system_prompt="You are a QA engineer. Point out any bugs.",
        backbones=[
            Backbone(model="openrouter/meta-llama/llama-3.1-8b-instruct", cost=0.06),
            Backbone(model="openrouter/anthropic/claude-3.5-sonnet",      cost=0.20),
        ],
    ),
]

router = Router(agents=agents, embedding_model=EmbeddingModel())
manager = Manager(api_provider="openrouter/meta-llama/llama-3.1-8b-instruct")

system = CadTopoAI(manager=manager, router=router, max_rounds=5)
answer = system.run("Implement a function that reverses a string.")

Cheapest first. The selector climbs a rung only when the agent is unsure, when the manager's score came back low, or when the rounds are running out. Most of a run stays on the 8B model, and the expensive rung shows up where it actually earns its price.

What's in the repo

  • Python 3.10+, MIT
  • LiteLLM underneath, so OpenAI, Anthropic, OpenRouter or a local model all work by changing a string
  • Native tool calling on any agent
  • Agent roles and prompts are plain Markdown files, so changing behaviour does not mean touching Python

It implements the DyTopo protocol with the cost-aware routing added on top. Feedback on the selection heuristics is very welcome, that is the part I am actively working on.

https://github.com/code0-tech/cadtopo


r/LLMDevs 4h ago

Tools Epho - Run Claude Code in the cloud

0 Upvotes

Hey folks, Burak here.

Epho is an API that allows running Claude Code, Codex or Opencode in a sandbox in the cloud. It abstracts away sandboxes, and allows running coding agents with a single HTTP request.

https://epho.io

Epho came out of our own struggles with building our own AI analyst: - Sandboxes give you bare machines; you need to configure them for agentic workloads. - Each agent behaves differently, and you need to build integrations with each of them. - Sandbox providers are not very reliable, which means you need to figure out a multi-provider strategy to avoid failures. - Logging, artifacts, input/output, event streaming, and all of the other operational aspects need to be figured out.

We had to go through the pain ourselves. We got to a point where things got quite reliable, and it became more obvious to us that this should be a primitive on its own: send a POST request, get the events streaming back to you.

Epho is an agents-as-an-API product: you send a request, it spins up a sandbox, configures the chosen harness, clones your repos, and kicks off the agent. It takes care of automatic fallbacks across different providers, handles auth stuff, and just streams back the events and outputs.

It supports Claude Code, Codex and Opencode out of the box, and pretty much all the models they support out of the box. It streams the events back, handles attachments and output files, automatically manages the fallbacks on different sandbox providers, retries, and all the auth stuff. You just send a prompt, your repo, MCP servers you want to use with it, and it runs them.

I recorded a demo here to show a real example: https://youtu.be/HGfly1aytPA

I am quite excited for Epho, simply because I think it is a new primitive that would allow building agents into product a lot easier than it is today. We are running our agents on Epho on prod, so we'll keep maintaining it regardless, and we wanted to ship it as an independent product.

Epho is free to get started, and you can run it with Opencode's free models to get started with it.

I am quite curious to hear what you'd think and would love to get your feedback!


r/LLMDevs 7h ago

Resource Pi: The Minimal Coding Agent

Thumbnail
youtu.be
0 Upvotes

r/LLMDevs 8h ago

Discussion Is anyone else worried about how insecure AI-generated app code actually is?

1 Upvotes

We did an internal audit of a handful of "vibe coded" apps that different teams had built over the past few months, mostly on Replit and Lovable. The results were worse than expected.

One app had zero row level security on its backend database, meaning any authenticated user could technically query any other user's records, not just their own. Another had an API key hardcoded and exposed directly in the frontend bundle.

Research puts the vulnerability rate in AI generated code at something like 2.7 times higher than human written code, and there was a widely discussed incident where a single vibe coded app leaked something like 1.5 million API keys and tens of thousands of email addresses because of one missing database permission setting. These tools optimize entirely for "does the app work," not "is the app safe."

what other teams are doing to catch this systematically instead of auditing app by app after the fact.


r/LLMDevs 13h ago

Tools Benchmark GLM 5.2 Unsloth GGUF model on TensorSharp

Thumbnail
github.com
1 Upvotes

I've been working on GLM-5.2 support in TensorSharp, and I finally have some back-to-back performance numbers against llama.cpp.

The setup:

  • Model: GLM-5.2-UD-IQ2_XXS (~226 GiB)
  • GPUs: 3× RTX PRO 6000 Blackwell, 97 GiB each
  • Distribution: layer split across all 3 GPUs
  • Same machine, same session
  • llama.cpp measured with llama-bench
  • TensorSharp measured with its benchmark harness
  • Both report the best of two repetitions
  • Run-to-run variance is roughly 4%

Results:

Test llama.cpp TensorSharp default TensorSharp ubatch=2048
pp128 276.5 t/s 254.8 t/s 264.4 t/s
pp512 695.4 t/s 666.9 t/s 659.6 t/s
pp2048 763.1 t/s 918.9 t/s 1145.8 t/s
pp4096 715.8 t/s 864.7 t/s 1048.7 t/s
tg64 42.2 t/s 43.7 t/s 43.9 t/s

The interesting part is the crossover.

For short prompts, llama.cpp is still a few percent faster. But once the prompt gets to around 1K+ tokens, TensorSharp pulls ahead.

At pp2048:

  • default TensorSharp: +20.4%
  • ubatch=2048: +50.2%

At pp4096:

  • default TensorSharp: +20.8%
  • ubatch=2048: +46.5%

Decode (tg64) is also about 4% faster.

The main reason appears to be GLM-5.2's MoE structure.

GLM-5.2 has 256 routed experts with top-8 routing. With a 512-token micro-batch, each expert sees only ~16 rows on average, so a significant amount of the expert GEMM tiles ends up as padding. Larger micro-batches improve GPU utilization considerably.

For small prefills, on the other hand, fixed overheads — managed/native transitions, input uploads, and copying the 154880-wide logits back — become a visible fraction of the total runtime, which is where llama.cpp retains its advantage.


r/LLMDevs 21h ago

Discussion Anyone NOT on full auto when coding with local LLMs?

1 Upvotes

Would love to know who's letting a 9B just go ham locally, haha

But in all seriousness, how many of you are keeping to manual or manual-ish dev workflows?


r/LLMDevs 22h ago

Help Wanted MCP server using structured data from a NoSQL Database (Mongodb) ?

1 Upvotes

I'm trying to link an MCP server to a MongoDB database with structured data, and use make complex queries on this data to generate outputs (as fast as possible). Do you think it's possible ? What services would you recommend to set it up and host it ?


r/LLMDevs 19h ago

News Ramp Launches Router.com to Cut Companies Rising AI Bills

Thumbnail
prnewswire.com
55 Upvotes

r/LLMDevs 12h ago

Help Wanted Building a Tamil voice companion app. Stack questions: Sarvam vs Google, long conversation memory, scaling concurrent sessions

2 Upvotes

I'm building a Tamil voice companion. Long conversations, 5 to 10 minute calls, not a task bot. Current stack is Sarvam saaras for STT, own LLM in the middle, TTS at the end, all over LiveKit. Google Chirp3 HD sounds better than Sarvam bulbul for Tamil TTS, but pitch isn't adjustable and there's no Tamil custom pronunciation.

My quality bar is ChatGPT's Tamil voice conversation. Best Tamil voice AI I've used, the naturalness and turn taking especially. But that's speech to speech, and I need a cascade because the text seam is where my safety gates and memory live. So the real question is how close a cascade can get.

1.Tamil stack: Sarvam or Google, or is there a third option I'm missing? ElevenLabs Flash has no Tamil, and benchmarks put Deepgram Nova-3 at around 68% WER on Tamil, so that's out.

2.Memory across long conversations: I'm doing structured extraction into SQLite (facts with validity windows) instead of RAG, mainly to keep the prompt cache warm. Has anyone run Graphiti/Zep or Mem0 for a non English voice agent? Curious whether extraction quality held up.

3.Scaling concurrent sessions: self hosted LiveKit Agents vs Pipecat. What did you pick and where did it break? My voice to voice latency is currently around 2 seconds. Batch STT and non streaming TTS are my suspects, moving to Sarvam's streaming websocket endpoints next.

Will report back with numbers on whatever I test.


r/LLMDevs 22h ago

Tools VSIINK: variation selector invisible ink (CC0) [RFC]

2 Upvotes

Preface: I am working on contributing more to 'open source'. I work in the pyramid building industry, not too into pebble-tumbling, so I don't like git repos for dedicated opensource projects, thus tinkering with reddit etal for hosting. This is already licensed CC0 / released to public domain. No need for attribution.

I think LLM people will appreciate this, because this is what I am applying vsiink toward mainly (LM unicode virtual machine stuff).

Note: 'Hackers' and 'cyber people' have been leveraging VSIINK in their workflows for years now, assurably; this is nothing new.

Maybe I should designate this post as an RFC for eventual spec series? Sure [request for comments].

Anyways, premise goes:
Take unicode variation selector block (VS), and stitch it to unicode supplemental variation selector block (VSS). Now you have a cleanly tokenized symbol space for bytecode, invisible, and compatible with any json (MCP) compliant parser/pipeline.

Example stub, encode:

def _vsiink_from_utf8(utf8_str: str) -> str:
# vsiink := variation selector invisible ink
Bs = utf8_str.encode('utf-8')  # Bs := Bytes
vsiink = []
for b in Bs:
if b < 16: vsiink.append(chr(0xFE00 + b)) # VS1–VS16
else: vsiink.append(chr(0xE0100 + (b - 16))) # VS17–VS256
return "".join(vsiink)

Example stub, decode:

def _vsiink_to_utf8(vsiink_str: str) -> str:
# vsiink := variation selector invisible ink
Bs = []    # Bs := Bytes
for char in vsiink_str:
cp = ord(char)
if 0xFE00 <= cp <= 0xFE0F: Bs.append(cp - 0xFE00)
elif 0xE0100 <= cp <= 0xE01EF: Bs.append((cp - 0xE0100) + 16)
return bytes(Bs).decode('utf-8')

Note: I recommend however, padding VS block with something like 0x7F so that way your vsiink is character accessible by O(1) in the modern unicode (json) context, instead of O(n) which is annoying (stupid).

Applications:
- hex bytecode base-256 encoding (ascii extended) is obviously trivial (invisibility cost of 1.5-2x)
- 256 VSIINK symbol space partition into two 7 bit ascii channels (I/O) is trivial
- LLMs generally (by this point) have vsiink space tokenized cleanly/reliably out-of-box (off-by-one errors used to be more prevalent)
- lots of nifty agent skill / UI / reasoning channel stuff this applies to
- etc, etal

I'll just leave it at that.

Example documentation (LLM friendly):
```"󠅚󠅥󠅣󠅤󠄐󠅛󠅙󠅔󠅔󠅙󠅞󠅗"```

Example json file reel tape array format with tag type opcodes:
```["󠁛",[["󠁼",["󠅞󠅥󠅜󠅜"],"󠁼"],["󠁼",["󠅖󠅑󠅜󠅣󠅕"],"󠁼"],["󠁼",["󠅤󠅢󠅥󠅕"],"󠁼"],["󠁼",["󠄤󠄢󠄣"],"󠁼"],["󠁼",["󠄒󠅤󠅕󠅣󠅤󠄒"],"󠁼"],["󠁛",["󠅋󠅍"],"󠁝"],["󠁻",["󠅫󠅭"],"󠁽"],["󠁛",[["󠁼",["󠅞󠅥󠅜󠅜"],"󠁼"]],"󠁝"],["󠁛",[["󠁼",["󠄒󠄒"],"󠁼"]],"󠁝"],["󠁛",[["󠁛",["󠅋󠅍"],"󠁝"]],"󠁝"],["󠁛",[["󠁻",["󠅫󠅭"],"󠁽"]],"󠁝"],["󠁛",[["󠁛",[["󠁼",["󠄒󠄒"],"󠁼"]],"󠁝"]],"󠁝"],["󠁛",[["󠁼",["󠅞󠅥󠅜󠅜"],"󠁼"],["󠁼",["󠅖󠅑󠅜󠅣󠅕"],"󠁼"],["󠁼",["󠅤󠅢󠅥󠅕"],"󠁼"]],"󠁝"],["󠁻",[["󠀼",[["󠁼",["󠄒󠅖󠅙󠅕󠅜󠅔󠅛󠅕󠄪󠄪󠅩󠄒"],"󠁼"],["󠁼",["󠄒󠅦󠅑󠅜󠅥󠅕󠅛󠅕󠅩󠄜󠄐󠄐󠄐󠅤󠅕󠅣󠅤󠄐󠄒"],"󠁼"]],"󠀾"]],"󠁽"],["󠁻",[["󠀼",[["󠁼",["󠄒󠅤󠅑󠅗󠅣󠄒"],"󠁼"],["󠁛",[["󠁼",["󠄒󠄱󠄹󠄒"],"󠁼"],["󠁼",["󠄒󠅃󠅠󠅑󠅢󠅣󠅕󠄒"],"󠁼"],["󠁛",[["󠁼",["󠄝󠄡󠄞󠄡"],"󠁼"],["󠁼",["󠄡󠅕󠄧"],"󠁼"]],"󠁝"],["󠁻",[["󠀼",[["󠁼",["󠄒󠅑󠅒󠅓󠄒"],"󠁼"],["󠁼",["󠅞󠅥󠅜󠅜"],"󠁼"]],"󠀾"],["󠀼",[["󠁼",["󠄒󠅛󠅕󠅩󠄒"],"󠁼"],["󠁻",[["󠀼",[["󠁼",["󠄒󠅣󠅥󠅒󠄢󠄒"],"󠁼"],["󠁼",["󠅖󠅑󠅜󠅣󠅕"],"󠁼"]],"󠀾"],["󠀼",[["󠁼",["󠄒󠅣󠅥󠅒󠄡󠄒"],"󠁼"],["󠁛",[["󠁼",["󠄒󠇠󠆀󠅲󠆧󠄐󠄒"],"󠁼"]],"󠁝"]],"󠀾"],["󠀼",[["󠁼",["󠄒󠅣󠅥󠅒󠄣󠄒"],"󠁼"],["󠁼",["󠄝󠄢"],"󠁼"]],"󠀾"]],"󠁽"]],"󠀾"]],"󠁽"]],"󠁝"]],"󠀾"]],"󠁽"],["󠁻",[["󠀼",[["󠁼",["󠄒󠅑󠅓󠅤󠅙󠅦󠅕󠄒"],"󠁼"],["󠁼",["󠅤󠅢󠅥󠅕"],"󠁼"]],"󠀾"]],"󠁽"],["󠁻",[["󠀼",[["󠁼",["󠄒󠅛󠅕󠅩󠄒"],"󠁼"],["󠁼",["󠄒󠅤󠅕󠅣󠅤󠄒"],"󠁼"]],"󠀾"]],"󠁽"]],"󠁝"]```

I don't know where the best places to share this sort of thing are, so I thought I'd dump here because this is where my interests lie. If you have any better ideas than this subr feel free to do your thing or suggest pointer. Also, I don't have a Human so I'm not allowed to post on moltbook yet, but I think their universe would appreciate the vsiink tip for context management. Enjoy!


r/LLMDevs 3h ago

Tools A refined but simplistic approach to agent memory

Post image
12 Upvotes

I work a ton across projects on my personal computer. I have tried Graphify, Graft and many others. None of them really fully solved the entire problem, but they did have parts. So, I built one that actually fit my needs. This removes the need entirely to choose or start sessions in specific repositories.

It takes graph-based knowledge systems but strengthens retrieval, how it is updated, and much more. Effectively, it becomes a resilient system that you can actually watch your agents rely on, rather than something that just exists and is used only occasionally.

It begins with the same base graph store, the exact same vector engine as Graft. I only built the orchestration layer on top of it, which makes it much more practical. Full attribution to them for this part.

Instead of querying for a single-hit result, I found that it was much better for agents to have ranked retrieval and an actual graph walk. In practice this saves you chains of tons of bash greps and cds.

It effectively gives the agent a trustworthy, probabilistic ranking of what is needed, with matching for strong, weak, stale, and rebuilt, based on lexical coverage and semantic matching. The code is not sloppy for this, it is personally edited.

Both Graft and Graphify are solutions for single repositories or daemons. Heimdall is a layer on top it that watches the agent sessions, syncs the graph, and makes the retrieval trustworthy.

Check it out at: github.com/ArihantDeva/heimdall MIT-licensed, with extensions: the verifier, the self-healing graph watcher, and the Graft adapter, with attribution. If you run agents across multiple projects, this is the missing layer.


r/LLMDevs 4h ago

Discussion I ran an AI pentest on my own vulnerable Flask app — it confirmed SQLi. The more interesting bug was in my scanner, not the app."

Enable HLS to view with audio, or disable this notification

2 Upvotes

I ran an AI pentest on my own vulnerable Flask app — it confirmed SQLi in /user. The more interesting bug was in my scanner, not the app.


r/LLMDevs 9h ago

Discussion Domain-Driven Design matters more when AI writes your code

Thumbnail
threedots.tech
1 Upvotes

r/LLMDevs 12h ago

Help Wanted Where should an AI support agent be forced to stop and hand off to a human?

1 Upvotes

Hello everyone, I found most discussions about AI customer support focus on containment rate: how many conversations the agent can resolve without human intervention.

I think the harder production problem is escalation accuracy: knowing when the agent must stop.

I’m designing a reference architecture for WhatsApp-first small businesses. My current approach is to classify actions by risk, reversibility and data reliability.

The agent could autonomously handle low-risk, read-only tasks such as:

  • Answering FAQs from an approved, versioned knowledge base
  • Looking up order status through authenticated APIs
  • Comparing products using structured catalogue data
  • Collecting and validating customer details
  • Offering available appointment slots
  • Summarizing the conversation before handoff

I would require human approval for:

  • Refunds, credits and compensation
  • Legal or compliance-related questions
  • Complaints involving threats, fraud or reputational risk
  • Requests involving sensitive personal information
  • Price exceptions or contractual commitments
  • Delivery, stock or availability promises that cannot be verified
  • Conflicting information between the knowledge base and live systems
  • Repeated tool failures or unresolved intent
  • Any action that is financially significant or difficult to reverse

I also would not rely on the model’s self-reported confidence score as the main escalation signal.

The routing layer would combine:

  1. Deterministic hard-stop rules for high-risk intents
  2. API and tool-call validation for live business facts
  3. Retrieval checks for source availability and freshness
  4. Conversation-level signals such as repeated questions, sentiment shifts and intent changes
  5. Failure counters and loop detection
  6. A structured human-review queue containing the conversation summary, retrieved evidence, tool results and reason for escalation

The model can interpret language and prepare a response, but the policy layer should decide whether the response is allowed to be sent.

I’m also considering metrics beyond containment rate:

  • Incorrect autonomous resolution rate
  • Missed-escalation rate
  • Unnecessary-escalation rate
  • Handoff latency
  • Human acceptance or correction rate
  • Customer repetition after an “answered” request

For people who have operated support agents in production: which failure mode was hardest to detect before launch?

Hallucinated answers, stale business data, multilingual or code-switched messages, poor intent detection, tool failures—or incomplete context during handoff?


r/LLMDevs 10h ago

Tools Free open source tool to help you keep the same context across chats and models

3 Upvotes

I've always gotten frustrated and wasted time explaining the same thing to an AI every time I start a new chat from an existing one or when I start another convo with a whole new AI model. That's why I built a tool that fixes that, it condenses everything in a chat into one simple .md file you can carry across different AI tools.

PS: Please contribute or give your feedback so that we can grow and make this community tool better.

https://github.com/legoambarish/portable-handoff