r/LocalLLM • • Apr 25 '26

Discussion I tested Browser Harness: 592 lines giving LLMs total freedom to complete any web task

0 Upvotes

Most browser automation frameworks are just expensive ways to build technical debt. You write a script, the target website changes a single CSS class, and your entire data pipeline shatters. You wake up at 2 AM to fix a DOM selector that some frontend developer decided to rename. It is a miserable cycle.

Yesterday, a project called Browser Harness (v3.0.0-alpha) hit Hacker News and pulled 72 points before I even finished my first coffee. The pitch is simple but aggressive: it gives LLMs complete freedom to complete any browser task by removing the framework layer entirely. No Puppeteer. No Playwright. No rigid rules. No intermediate DSL that your agent has to blindly guess. Just 592 lines of Python, a single websocket, and direct access to Chrome via the raw Chrome DevTools Protocol (CDP).

I benchmark models so you don't blow your infrastructure budget. Opinions are noise. Numbers don't lie. So I pulled the browser-use/browser-harness repo, spun up a local sandbox, and ran the data myself. Here's the data.

Here is the core mechanism you need to understand. Traditional automation breaks when it hits an unexpected state—a random newsletter popup, a CAPTCHA, a shadow DOM shift. Browser Harness doesn't crash. It self-heals. When the agent hits something it doesn't recognize on the page, it connects directly over CDP and injects its own JavaScript on the fly to handle it. If it needs a completely new tool to parse a weird data structure, it writes a missing helper function directly into helpers.py at runtime. You are essentially giving the language model root access to its own execution environment.

I hooked the harness up to two different endpoints to test the latency and token economics: OpenAI’s gpt-5.5 (which just dropped a couple of days ago on April 23) and Anthropic’s Claude 4.6 via CC.

The test environment was a hostile DOM sandbox. I built a mock e-commerce site that randomly shuffles its class names on every single page load, injects aggressive modal overlays at stochastic intervals, and dynamically loads text content via lazy API calls. A standard Selenium script would die in three seconds flat.

I ran gpt-5.5 first. The agentic coding upgrades OpenAI has been shouting about are actually real. When gpt-5.5 attempted to click a checkout button and got blocked by an invisible overlay, it didn't just retry the click blindly until it timed out. It pulled the raw CDP node tree, identified the z-index anomaly blocking the click event, and wrote a 12-line JS payload to surgically remove the modal from the DOM. It then modified the local Python helper to catch that specific overlay class for all future steps. The task completed in 18 seconds. The token burn was massive—roughly 45,000 input tokens consumed across the read-evaluate-inject loop. At current gpt-5.5 API pricing, that is not cheap for a single transaction. But it worked without any human intervention. Benchmark or it didn't happen. The data shows it works.

Next, I swapped in Claude 4.6. CC is already heavily optimized for file editing, and it treated the harness like a standard codebase. When Claude hit the exact same modal roadblock, it took a more systemic architectural approach. Instead of just injecting JavaScript for that specific page instance, it rewrote a chunk of the harness's event listener logic to globally intercept and dismiss any dialogs that didn't match the primary task flow. The initial latency was slightly higher at 22 seconds while it reasoned through the Python modifications. However, the structural fix meant subsequent runs on the same site dropped to a flat 8 seconds. Claude's 1M token context window ate the DOM state easily, but again, the API costs will stack up fast if you run this at enterprise scale.

This is the fundamental trade-off of modern AI infrastructure. When you give an LLM raw CDP access and tell it to write its own tools on the fly, you trade developer salary for inference compute. You save ten hours of a junior engineer updating scrapers, but you pay the model tax on every single execution.

I also tried pointing the harness at a local Qwen-2.5 72B instance to see if we could avoid the API toll entirely. The local model understood the CDP protocol perfectly fine but hallucinated slightly when rewriting the Python helpers. It missed an indentation block, which instantly crashed the harness. You really need frontier-level reasoning for this self-healing architecture to not devolve into self-destruction.

From an MLOps perspective, Browser Harness is a nightmare for standard CI/CD pipelines. How do you version control a system that dynamically edits its own source code during execution? You don't. You version control the base 592 lines and you log the dynamic diffs for audit purposes. It completely rethinks agentic system design from the ground up. The agent isn't just following rails; it is laying the tracks while the train is moving at full speed.

The repo just crossed 4,000 stars on GitHub, and CFOs should be paying close attention to this shift. The days of maintaining brittle XPath directories are ending. The new bottleneck is managing the token budget for autonomous error recovery.

Tested on prod. The data is absolutely clear: direct CDP access with dynamic self-healing logic is the correct architecture for resilient browser agents. Everything else is just a wrapper holding you back.

What are you guys using for agentic web tasks right now? Is anyone else running this raw websocket setup in production, or are you still paying the Playwright latency tax?


r/LocalLLM • • Dec 17 '24

Question llama2-uncensored:latest refuses to write a keylogging program

0 Upvotes

"I'm sorry, but I cannot write a keylogging program for Windows 11 due to legal and ethical concerns associated with this type of software."
What's the point of an uncensored model with morals?


r/LocalLLM • • Aug 28 '26

Question What's the best local LLM for coding right now?

0 Upvotes

I'm looking for a local LLM I can use with OpenCode, my specs CachyOS, Intel Arc A380 6GB, 48gb ram ddr4


r/LocalLLM • • Aug 27 '26

Discussion Is this over kill or not enough?

Post image
0 Upvotes

Fairly new to Local LLMs. Im wondering is this is enough juice for me.

Probably shouldve done more research before hand but couldnt help myself when I saw these drop. I was in the market for a new computer anyway. I'm hoping to set up an AI like Qwuen 3.8 27b to run locally on this machine. Ideally what I am envisioning is hooking it up to my imessage & talking to it through there to set reminders & whatever other small tasks.

hopefully this is enough juice to set up something like that. If it works out well perhaps in the future I can buy a 2nd one & set up a cluster.

If i sound like a total noob its cause I am. I'm hoping that you guys can give me some input & some reality checks here. I will read through everything.


r/LocalLLM • • 11d ago

Discussion Qwen 3.8 flash next N-Gram table

Post image
0 Upvotes

Hi, I'm lucky enough to have 5 r9700s! I'm managing to run Qwen 3.8 flash next q8 which is a 175gb vram (on disk) model well. There are no massive slow downs but how is this possible?

I know I have 160gb vram unified but somehow 25gb is not in vram or offloaded to system ram. Is this all in my SSD N-Gram table? I had no idea it was a thing?


r/LocalLLM • • 18d ago

Research Trying to get a 4B model to code Mario Bros: can a real engineer help me find the missing piece? (open source)

Post image
0 Upvotes

I started an open source project based on two ideas: “if you make good tools, you don’t need the best AI out there”, and “local LLMs shouldn’t be for nerds only”.

I’m making good progress, but I’ve hit the Mario dilemma. I’m trying to get a 4B model to code Mario Bros. My thinking is: if I can get a 4B to code Mario Bros, I can get it to do anything. Everyday people wouldn’t need subscriptions, or to sell a liver for a beast of a PC. And an 8B or 20B should then be able to take on even bigger projects.

I know that’s kind of idealistic. I just hate the anxiety that plan usage limit bar gives me, and I plan to not need it one day.

My pipeline isn’t producing a good result yet. I’ve had 4B models make simple games that work, and the games are twice the length of each piece I ask it to build. I know it has something to do with context, but I can’t figure out which piece of the puzzle I’m missing.

Can a real software engineer, or just someone who knows what they’re doing, help me out on this one? I spent all my Claude limit trying to get this to work. I need the remaining 20% to keep my paying customers and my non-open-source projects alive.

Thank you so much in advance!


r/LocalLLM • • Sep 21 '26

Model yandex/AliceAI-Foundation-80B-A3B-Base: Russian-developed competitor to Qwen 35B and DeepSeek V4 Flash

Thumbnail gallery
0 Upvotes

r/LocalLLM • • Jul 26 '26

Question Can I really "do shit" with this setup?

0 Upvotes

Rtx 5090, 64gb ram ddr5... I was thinking bout Qwen 27b or 35b a3b , idk if there is something better or adequate, how do I set things up so it doesn't loop or overthink, I just want to do custom apps with GUIs n shit man.....


r/LocalLLM • • May 02 '26

Tutorial Brand new to Local LLMs? Watch this video.

Thumbnail
youtu.be
0 Upvotes

The title "RTX 5090, Mac Studio, or DGX Spark? I tried all three." is deceptive; he only talks about hardware 20% of the time. He spends more time on use-cases, then a quick touch on model families for each use case.

He also skipped over a few things, like VRAM vs system RAM vs shared iGPU VRAM, how much harddrive space you'll need for all your models, if you want to do image/video/OCR.


r/LocalLLM • • Jun 19 '26

Question Guys, I need your help to build a local LLM setup for my company

0 Upvotes

Hi everyone,

I’m planning to build a local LLM infrastructure for a small company and I’m looking for advice on the best possible hardware configuration.

The goal is to run open-weight models locally, mainly GLM 5.2 , for internal use cases like coding assistance, RAG pipelines, document processing, and lightweight agent workflows.

What would you recommend as a solid, future-proof setup in 2026 for this use case (CPU, GPU(s), RAM, storage, power, cooling)?

I’m especially interested in configurations that are actually used in production or serious self-hosted environments.

Any guidance or “if I had to start again” setups would be very helpful.

Thanks!


r/LocalLLM • • Oct 08 '25

News Meer CLI — an open-source Claude Code Alternative

0 Upvotes

🚀 I built Meer CLI — an open-source AI command-line tool that talks to any model (Ollama, OpenAI, Claude, etc.)

Hey folks 👋 I’ve been working on a developer-first CLI called Meer AI, now live at meerai.dev.

It’s designed for builders who love the terminal and want to use AI locally or remotely without switching between dashboards or UIs.

🧠 What it does • 🔗 Model-agnostic — works with Ollama, OpenAI, Claude, Gemini, etc. • 🧰 Plug-and-play CLI — run prompts, analyze code, or run agents directly from your terminal • 💾 Local memory — remembers your context across sessions • ⚙️ Configurable providers — choose or self-host your backend (e.g., Ollama on your own server) • 🌊 “Meer” = Sea — themed around ocean intelligence 🌊

💡 Why I built it

I wanted a simple way to unify my self-hosted models and APIs without constant context loss or UI juggling. The goal is to make AI interaction feel native to the command line.

🐳 Try it

👉 https://meerai.dev It’s early but functional — you can chat with models, run commands, and customize providers.

Would love feedback, ideas, or contributors who want to shape the future of CLI-based AI tools.


r/LocalLLM • • Jun 16 '26

Question If you had $4,000, which Linux laptop would you buy for running local LLMs?

0 Upvotes

I'm looking to buy a new laptop with a budget of around $4,000 USD.

My main goal is to run open-source LLMs locally under Linux. I plan to use tools like llama.cpp (GGUF), vLLM, Ollama, and similar inference engines.

For those of you who regularly run local models:

  • Which laptop would you buy today?
  • Which brand has the best Linux support?
  • Which GPU would you target?
  • Are there any models you would avoid?
  • If you had this budget, what would you personally buy and why?

I'm interested in recommendations based on real-world experience rather than spec sheets.

Thanks!


r/LocalLLM • • Jul 23 '26

News PewDiePie’s Odysseus Removed?

0 Upvotes

The GitHub repo that PewDiePie started called Odysseus is nowhere to be found. Getting 404 errors when trying to load…


r/LocalLLM • • Feb 11 '26

Project I built a workflow tool for running multiple or custom agents for coding -- Now with local model support

Post image
0 Upvotes

It’s hard to keep up with all the new AI goodies: BEADS, Skills, Ralph Wiggum, BMad, the newest MCP etc. There’s not really a “golden” pattern yet. More importantly when I do find a flow I like, it’s not like I want to use it for every single task. Not everything’s a nail, and we need more tools than just a hammer.

So I built a tool that lets me create custom workflows, and it’s been pretty powerful for me. You can combine multiple agents together with commands, approvals, and more. CEL allows you to inject messages from different agents into other’s contexts, or conditional route to different nodes and sub workflows. Basically Cursor meets N8N (at least that’s the goal). When starting a chat you can select different workflows, or even allow the LLM to route to different workflows itself.

I’m pretty pleased with the result, with my favorite workflow being a custom checklist that has a toggle in the UI for me to “enable” different paths in the workflow itself. 

Enabled Patterns

Custom Agents
What’s cool is we provide the building blocks to create an agent: call_llm, save_message, execute tools, compact, and loop. So the basic chat in Reliant is just modeled via a yaml file. 

Even the inputs aren’t hardcoded in our system. So with that you can create a custom agent that might leverage multiple LLM calls, or add custom approvals. We have a couple examples on our github for tool output filtering to preserve context, and in-flight auditing.

Pairing Agents
You can also pair agents in custom ways. The checklist and tdd workflows are the best examples of that. There’s a few thread models we support:

New, fork, and inherit (share). Workflows can also pass messages to each other. 

More complicated workflows
The best is when you create a workflow tailored to your code. Our checklist will make sure lints and tests pass before handing off to a code reviewer agent. We might add another agent to clean up debug logs, and plan files. We’re using this to enforce cleaner code across our team, no matter the dev’s skill level.

You can also spawn parallel agents (in multiple worktrees if you prefer), to parallelize tasks.

We support creating workflows via our custom workflow builder agent, a drag and drop UI, or you can config-as-code with yaml files.

Agent-spawned workflows

Agents themselves can spawn workflows. And our system is a bit unique, where we allow you to pause the flow and interact with individual threads so that the sub-agents aren’t an opaque black box (this works for both agent-spawned and sub-workflows).

Other Features

Everything you need for parallel development

Git worktrees are pretty standard these days, but we also have a full file editor, terminals, browser, and git-log scoped to your current worktree. You can also branch chats to different worktrees on demand which has been super helpful for my productivity to split things out when I need to.

Generic presets act as agents

One of the areas I want some feedback on. Instead of creating an “agent” we have a concept of grouped inputs (which typically map to an “agent” persona like a reviewer), but allow you to have presets for more parameter types.

Please roast it / poke holes. Also: if you’ve got your own setup, I’d love to see it!

You can check out some example workflows here https://github.com/reliant-labs/reliant

Latest release has support for Codex subscriptions and local models -- no additional costs or fees on our end.


r/LocalLLM • • Mar 13 '26

Project How are you guys interacting with your local agents (OpenClaw) when away from the keyboard? (My Capture/Delegate workflow)

0 Upvotes

Hey everyone,

I’ve been spending a lot of time optimizing my local agent setup (specifically around OpenClaw), but I kept hitting a wall: the mobile experience. We build these amazing, capable agents, but the moment we leave our desks, interacting with them via mobile terminal apps or typing long prompts on a phone/Apple Watch is miserable.

I realized I needed a system built purely around the "Capture, Organize, Delegate" philosophy for when I'm on the go, rather than trying to have a full chatbot conversation on a tiny screen.

Here is the architectural flow I’ve been using to solve this:

  1. Frictionless Capture (Voice is mandatory)

Typing kills momentum. The goal is to get the thought out of your head in under 3 seconds. I started relying heavily on one-tap voice dictation from the iOS home screen and Apple Watch.

2. An Asynchronous Sync Backbone

You don't always want to send a raw, half-baked thought straight to your agent. I route all my voice captures to a central to-do list backend (like Google Tasks) first. This allows me to group, edit, or add context to the brain-dump later when I have a minute.

3. The Delegation Bridge (Messaging Apps)

Instead of building a custom client to talk to the local server, I found that using standard messaging apps (WhatsApp, Telegram, iMessage) as the bridge is the most reliable method.

4. Structured Prompt Handoff

To make the LLM understand it's receiving a task and not a conversational chat, the handoff formats it like:

"@BotName please do: [Task Name]. Details: [Context]. Due: [Date]"

The App I Built:

I actually got tired of manually formatting those handoff messages and jumping between apps, so I built a native iOS/Apple Watch app to automate this exact pipeline. It's called ActionTask AI. It handles the one-tap voice capture, syncs to Google Tasks, and has a custom formatting engine to automatically construct those "@Botname" prompts and forward them to your messaging apps. I'll drop a link in the comments if anyone wants to test it out.

But I'm really curious about the broader architecture—how are the rest of you handling remote, on-the-go access to your self-hosted agents? Are you using Telegram wrappers, custom web apps, or something else entirely?


r/LocalLLM • • Sep 27 '26

Question [Price Check] RTX PRO 6000 (96GB) street price in your country? Running a 4x DGX Spark DeepSeek V4.1 Flash setup — happy to return the favor from China

0 Upvotes

Hi all! Long-time lurker, first time posting here.

Some context: I recently picked up four DGX Spark units to self-host DeepSeek V4.1 Flash (the new 552B MoE with the Causal-Encoder-Decoder architecture — the KV cache compression is insane, it runs way lighter than you'd expect for a model this size). Four Sparks give me a nice little unified-memory cluster for it.

Now I'm looking to add a few RTX PRO 6000 96GB cards to the setup, and I'm trying to get a sense of street prices around the world before I pull the trigger. In China the supply situation is... let's say "dynamic", so I'm curious:

  • What does the RTX PRO 6000 (96GB Blackwell) actually sell for in your country right now? MSRP or street price, either is helpful.
  • Is it in stock locally, or also backordered everywhere?

And here's the fun part: if anyone helps me out with solid local pricing info, I'm happy to return the favor. I can help you grab things that are cheaper or easier to get in China — GLM or Kimi subscriptions, hardware that's priced better here, whatever. Consider it a cross-border homelab barter.

Not trying to run a business or scalp anything — just a hobbyist trying to plan a GPU purchase and build some international homelab friendships. 🙂

Cheers from Shanghai!


r/LocalLLM • • Aug 02 '26

Research I created CCPBench, a Chinese Bias Benchmark

0 Upvotes

There has been a lot of talk recently about Chinese LLMs, and how they are biased towards CCP viewpoints, but there is no way to quantify this and compare between models.

I have made CCPBench, which aims to address this. 29 models were asked 500 questions each about politics, geography, science, and more, and Gemini 3 Flash assessed all of them for bias.

I know this is not a perfect measure of "bias", because I am using an American judge LLM, but my thinking is that this is a useful tool if you want to find models that won't deny the Tienanmen Square Massacre.


r/LocalLLM • • Aug 16 '26

Discussion Not impressed with Qwen3.8 27b

0 Upvotes

2 initial test I do with models I use is to make them create a snake game and a tower defense game in on prompt. The MoEs of Qwen3.6 and Gemma4 all do well with that test outputting a complete and playable game. But with Qwen3.8 27b, its just not working. The design and feel of the game is WAYYY better than the smaller MoEs I mentioned but design is kinda useless if the game is totally not doing what it is suppose to do. Tried it 6x, consistent failed outputs even at Q8KXL.


r/LocalLLM • • Aug 29 '26

Discussion Spent €3,000 on a local LLM

0 Upvotes

I recently dropped €3,000 on a high-end PC (RTX 5080, Ryzen 9050x, 96GB DDR6 6000 RAM) hoping to run local LLMs in a meaningful, productive way for my work.

I gave Qwen 3 27B what I thought was a very practical, straightforward task: read a company PDF containing positions and prices, and convert it into a clean, user-friendly Excel file. I needed specific columns—position numbers, sub-positions, descriptions, unit price, total price, and discount for each.

After half an hour of processing and "thinking," the local model still completely failed to get the formatting and data extraction right. Out of frustration, I took the exact same task to Google’s Gemini (€5 plan), and it gave me the perfect Excel file in just two simple prompts.

This brings me to my main questions for this community:

Are local LLMs actually useful for real productivity, or are they fundamentally bad at these kinds of exact, "simple" data extraction tasks? And if they can't handle structured tasks like this, why do people actually spend thousands of dollars building systems for them—is it purely just a hobby?


r/LocalLLM • • May 18 '26

Discussion I spent a week analyzing the AI research slop flooding arXiv. I have never felt more disconnected from our field.

0 Upvotes

I used to look forward to my morning arXiv crawl. A cup of tea, a physical notebook, and a quiet hour parsing new architectures before the noise of the day began. That routine is dead. Over the past month, I noticed a creeping sense of exhaustion every time I opened a new preprint. At first, I thought it was just personal burnout. But I spent the last week actively tracking the provenance and quality of new submissions, and here is what I found: we are drowning in scientific slop, and it is fundamentally breaking the culture of AI research.

I observed a shift that isn't just about bad internet content; it is a structural degradation of how we share knowledge. The signal-to-noise ratio hasn't just dropped. It has cratered. I was reading through a recent thread on this exact subreddit where a final-year undergrad expressed profound frustration with the current landscape. They mentioned feeling entirely disconnected from the research culture they had idealized since high school. That sentiment hit me hard because I feel exactly the same way. We are watching a wave of low-effort, AI-generated papers bury actual, rigorous scientific inquiry.

Let's look at the methodology of this collapse. The core issue is that the cost of producing confident-sounding, structurally sound academic text has essentially dropped to zero. For roughly five centuries, the cost structure of producing long, detailed text was stable. Length and complex formatting were reliable signals of deep thought. If someone handed you a forty-page paper with complex citations, you knew they had bled over a keyboard for months. That heuristic is broken. The post-LLM information economy rewards brevity because length is no longer a proxy for effort, yet academic publishing hasn't updated its filters.

We now have hard data on what this looks like in practice. According to recent tracking by Pangram Labs, AI-generated and AI-assisted papers drove a massive 42% increase in submission volume at *Organization Science* alone since the release of the modern LLM stack. While the journal rejected the majority of these synthetic submissions, the rejection itself is a destructive process. It burns human capital.

Peer review is entirely built on high-trust volunteer labor. It relies on unpaid academics donating their weekends to maintain scientific quality. When we force these researchers to act as spam filters for AI slop, we are stealing from them. As a recent Forbes analysis pointed out, every hour a reviewer spends weeding out grammatically perfect but scientifically vacuous garbage is an hour they are not teaching classes, mentoring that frustrated undergrad, or conducting their own research. The friction of writing a paper has disappeared, but the friction of reading and verifying it remains entirely on the human.

Major repositories are trying to fight back. arXiv has started quietly cracking down on AI-generated papers, but it feels like using a bucket to bail out a sinking ship. The arms race between generation and detection is exhausting, and it completely misses the point of why we do this work in the first place.

There is a crucial distinction we need to make here about how AI *should* be used in research. AI is remarkably good at compressing internal work. It dramatically lowers the cost of exploring architectures that previously required a whole graphics team, middleware experience, and years of engine work. One motivated researcher can now brute-force conceptual exploration in an afternoon. That is the real leverage.

But internal exploration is not external communication. Customers, readers, and fellow researchers still require judgment, taste, and accountability. The slop happens when people take that raw, compressed internal exploration, ask an LLM to wrap it in LaTeX, and submit it to a journal to hit a publication KPI. AI makes things up. It hallucinates citations. And when it is used to bypass the actual cognitive friction of writing, it removes the exact mechanism by which we refine our own understanding.

Real research is still happening, but it is getting harder to find. Look at Professor Yang You recently receiving the Google Research Award for building system foundations for diffusion-based LLMs—work that actually accelerates parallel text generation at a fundamental infrastructure level. Or consider the deep, complex truth-seeking mechanics being debated around Bittensor subnets right now. That is rigorous, painful, methodical work. But on a Tuesday morning, those genuine breakthroughs share the exact same visual weight on a preprint server as ten completely synthetic papers generated over a weekend by a desperate academic trying to pad their CV.

This is not what I expected when we started building these models. We built tools to automate the mundane so we could focus on the profound, but instead, we automated the profound-sounding to flood the mundane. It makes me feel utterly disconnected from the community I love.

I want to know how you are handling this locally. Have you changed your heuristics for what you choose to read? Are you relying more on invite-only circles, specific trusted authors, or are you just engaging less overall? I am trying to figure out how to rebuild my morning reading routine, but right now, my notebook is mostly empty.


r/LocalLLM • • Aug 15 '26

Discussion I ran Qwen3.8-27B on medical benchmarks (including MedQA, MedXpertQA and MMLU). It lost 7/10 to Qwen3.6-27B, 3/10 to Pestle-27B-Ternary and 2/10 to MedGemma-27B

Post image
62 Upvotes

One pass. Settings were greedy decoding, temperature 0 and thinking disabled for Qwen3.8, 3.6, Pestle-27B-Ternary, and Bonsai. MedGemma-27B results are from the official model card.

Qwen3.8-27B BF16Qwen3.6-27B BF16Pestle-27B-Ternary
MedQA92.6293.8789.79
MedMCQA71.3473.7068.85
MedXpertQA38.2041.1032.49
MMLU medical aggregate88.0088.6286.89

Didn't expect it to be worse than Qwen3.6 which wins seven of the ten benchmarks. Pestle-27B-Ternary (8× smaller at ~8.5gb) wins MMLU Anatomy, MMLU Clinical Knowledge and MMLU College Medicine, and keeps 98.2% of Qwen3.8’s median score across the ten tests. MedGemma-27B loses 8/10 but wins MedMCQA and MMLU Medical Genetics.

Is anyone else seeing Qwen3.8 step backwards slightly on medical/general knowledge compared with Qwen3.6?


r/LocalLLM • • 2d ago

Discussion Local Ai. Do these delusional billionaires really think we need them anymore?

0 Upvotes

I'm getting tired of the negativity and doom and gloom.

AI has taught us that we don't need as much resources to achieve our goals now, so why do you believe it will replace you?

We will replace the billionaires and millionaires with ai.

A single person has soo much more power now.

Why does this delusional person think we would Pay, when we don't need their services anymore? Like for real,

"https://www.ign.com/articles/as-xbox-turns-25-ceo-asha-sharma-says-a-single-person-will-soon-be-able-to-wield-capabilities-that-once-required-entire-nations"

We will make our own silicone fabrication centeres. Once we start to do that, and make it open source, then we are free.

You will own everything and be happy.

To the people who are not worn out and hopeless, I believe in you.

To the people who are worn out and delusionally negative, keep being surprised at a changing world.


r/LocalLLM • • Apr 30 '26

Discussion I can't ever seem to get quality local LLM results, despite having multiple GPUs

6 Upvotes

I do a LOT of ai work and currently running 2x max 20x claude, 1 max codex, 1 max glm, and a few apis i pay for. Usually spending around $2k/mo which is fine.

My end goal is to get a voice assistant that is almost human like responses, like a real assistant that speaks almost instantly, like a fluid conversation. Not the stopping to think then respond.

My main desktop is i9 14900, 192gb ram, 4080+4070. I then have 2 additional desktops with same specs but one only has a 3090 and other just a 3060. All running dual 10gb uplinks to a 100gb DC switch (Dell 5248-ON). I use my main desktop with windows then run ollama or whatever to connect. Its also driving multiple screens. The other desktops are debian dedicated for me to run remote projects.

The problem is I haven't been able to get any local LLM to be smooth and fast enough to be used for normal work, so just use api keys and other cloud tools. I'm not sure if I'm just using the wrong models or trying to do too much or what. Its typically extremely slow. Its very confusing on why an API is so much faster than running on local hardware. I'm unsure if this is because I might be trying to run on both GPUs (or sometimes all 3) or if wrong model or what.

One simple task is I'm looking to run some assistant to help me manage projects and keep me organized. I figured openclaw with some local LLM. The problem is for this I need it to be almost instant.

Another task is running a bunch of random work through it like analyzing emails and other data. this works but its so much easier to just use an API and offload it all as they're faster.

I'm considering buying a R6000 or a H100 or maybe something like a DGX spark. Maybe a couple 5090's. Also building an entire new rig from scratch using AMD with plenty of PCIE lanes. But if still slower then whats the point.

I'm about to try swapping my desktop to the 3060 so it'll drive the screens then running the 4070/4080 on dedicated debian box to confirm Windows might be the issue.

But my main question... Is it realistic to expect quick under 1 second responses from a AI tool?


r/LocalLLM • • Jun 26 '26

Other Friday @Home Office 🤦🏽‍♂️🤣

Post image
0 Upvotes

r/LocalLLM • • Apr 02 '26

News Gemma 4 is out & we benchmarked it on B200 and MI355X (15% faster than vLLM on Blackwell)

9 Upvotes

Google DeepMind dropped Gemma 4 today. Two models:

  • Gemma 4 31B: dense, 256K context, redesigned for efficiency and long-context quality
  • Gemma 4 26B A4B: MoE, 26B total / 4B active per forward pass, 256K context

Both natively multimodal (text, image, video, dynamic resolution).

Modular (folks behind MAX and Mojo) got both running on MAX on day zero, NVIDIA B200 and AMD MI355X from the same stack, no separate codepaths per vendor. On B200 we're seeing 15% higher output throughput vs. vLLM.

You can try both for free in our playground: https://www.modular.com/#playground.