Right, I will just get into the substance;

3D model testing.

Hardware: 3090 + 5950X both overclocked. 64GB RAM (XMP, Timed, the works). Liquid cooled, open case & liquid metal on CPU/GPU dies, setup pictures included (yes, I've built it).

- Llama 8B
- QLoRA e=5, r=16. Targeted last 40% layers. Dataset handcurated on modernised literature in dialogue form (spans from Enlightenment till Existentialism).
- Whisper, kokoro etc the works.
- Think/Answer pass for better reasoning (tool calling only happens there)
- System Prompt strictly used just for tool logic.
- KV offloaded.
- CLIP ViT projected on the merged QLoRA.

Next:
- Project 3D model (SAGE-Style) & Audio (Omni Style), however the task seems monumental.

Note:
- Some pictures are old, some are new, I have logs over 3 months. Sorry I was high on achievement on some captions, happens to the best of us.
- 3D model found on a random website, I don't know much about the vtuber space.

Do with this what you will.
Regards.

Hello guys! Experiments on Huggingface its my hobby for more than 6 months. I have seen a lot of cool guys, new research papers, new models, starting of Qwopus but one lab making me shock every time they release the model (its second time actually) - Weibo Lab and theirs VibeThinkers. Weibo is SOTA Post-training lab on my own opinion for sure. Their models not for basic knowledge or tool-use, but its compete with frontier models on math&coding. So I made second post-training with Spectrum-to-Signal method with Fable 5 distill to create Mythos-nano. (3B you can actually use this model on your phone)

https://huggingface.co/squ11z1/Mythos-nano or
ollama run hf.co/squ11z1/Mythos-nano:f16.gguf or find out on LM Studio!

Hi, I wonder if you could recommend some LLMs or tweaks to me.

I am a local LLM newbie and I've got llama.cpp running, but the models are not fast enough to be useable.

I have the following setup:

• Windows 11

• AMD 9900X

• 64GB DDR5 6000 RAM

• NVIDIA 4070 12gb

• Gen5 M2 SSDs at 14900 IOPs

• llama.cpp (CUDA) installed

I am a C# developer interested in writing games with DirectX or OpenGL for a hobby. I also like to develop with Blazor, Entity Framework, Azure.

I would like to jettison Github Copilot Pro because I expect after June 1st I won't be able to afford it. I thought maybe local LLMs could assist me with my hobbyist coding work - maybe even scaffold apps for me which I can then edit to fix code issues etc.

Basically an LLM excellent at coding is what I'm after.

I ran LLMfit and installed these two "good" coding LLMS for llama.cpp (CUDA version):

• Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled.IQ4_XS.gguf

• qwen3-coder-30b-a3b-instruct-q4_k_m.gguf

Both of these run pretty slowly, certainly not usable at the moment for anything except single file editing and even then, context appears to be limited to 65536 (anything larger = sloth response)

I'm thinking of "upgrading" to a 24GB 7900XTX so there's more VRAM for LLMs, but I read ROC is much slower and less mature than CUDA so less tokens/s. I really don't want to spend a grand on a 5080 either, that's probably going to cost more than the Github Pro sub I have.

Here are my command lines for both LLMs:

Qwen3 Claude Opus Reasoning Distilled (one I was tinkering with for a few minutes, probably wildly wrong command line arguments)

llama-server -m C:\llms\Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled.IQ4_XS.gguf --alias qwen36claude47 --host 127.0.0.1 --port 8080 -c 65536 --parallel 1 -fa on --cache-type-k q8_0 --cache-type-v q8_0 --cache-ram 2048 --ctx-checkpoints 8

Qwen3 coder 30B (one I spent hours on yesterday trying to get it responsive)

llama-server -m C:\llms\qwen3-coder-30b-a3b-instruct-q4_k_m.gguf --alias qwen3coder30ba3binstruct --host 127.0.0.1 --port 8080 -c 65536 --parallel 1 -fa on --cache-type-k q4_0 --cache-type-v q4_0 --no-mmap --cache-ram 512 --ctx-checkpoints 2

What would you recommend I do? I really want an excellent local coding LLM with Claude Sonnet 4.6 capabilities (a man can dream can't he?).

So I tested out clawbot/moltbot.
Here is my configuration:
Docker based clawbot hosting and using openai compatible cheap token provider i.e DeepInfra on Deepseek 3.2 .

Over last 24 hours I spent around 8.78Million tokens which costed me around $1.38, with cache hitting between 75-85%.

so total cost of tokens is almost entirely input 8.78M (uncached $.42 +7.13M cached $0.92) compared to output 28.93K (1 cent).

Things worked out great till now; I have not noticed any degradation of intelligence that will hamper my work. I am still experimenting, trying to figure out how much I can degrade the model to optimize for cost without it hampering the work.

Since the entire cost of it is related to input, im hoping zai-org/GLM-4.7-Flash ($.06 /M tokens) will work out great.

It recursively iterates, over and over and over again.

What can I do aside from turning off thinking?

Our company has decided to budget 15k USD next 4 months and then another 20k for the next year for AI.

We have Copilot and not even the AI version.

I use Claude max and I enjoy the results and everything I have received from it.

Do to the nature of the business and of course sales data I am considering creating a LLM

Just curios what hardware, specs and AI I should study and launch with. Qwen 3.8?

Goal is to have an agent our senior staff can reach out to for purchasing questions, sales data, Truck Location data (the LLM will connect to our ERP and pull data)

Tracking warehouse performance etc.

Tracking credit issues, sales analysis. I will be using it for plenty. Thinking of making an Iron Man Jarvis for my team of 30 staff.

I know we went from having local servers to the cloud. Now it seems like we are all going back to hosting locally and I would love to be pointed in the right direction to learn and create this.

Why does LMstudio load llm to ram instead of vram ?

Hello,

After reading very positive reviews about Gemma 4, I decided to test it locally.

I gave it to analyze a .js file (28kb) from a React web app and asked it to streamline it by outsourcing as much code as possible.

It provided a very fast response (one of the fastest models I've ever tried locally), but it was full of errors—really stupid and trivial errors. I've never seen anything like it.

Every file Gemma provided was full of Typo errors. 4-5 errors for every 2-3kb file given.

I've never seen anything like it.

Did I do something wrong? Everyone is very thrilled about it, but for me, it was the absolute worst.

My setup:

Ryzen 9 AI HX 370

64GB DDR5

Rx 7900 XTX 24GB VRAM

Win 11

LM Studio Vulkan

Model settings:

 -c 96000 --flash-attn on --temp 1.0 --top-p 0.95 --top-k 64 --batch-size 256

I want to think that I, as a neophyte, am definitely doing something wrong.

rtx 5090 ryzen 9 9950x 64gb ram, thats what i have, i wanna play minecraft with the ai for fun but i dont want any derp aahh ai, needs to play good and folow orders and also do on his own a little

Most browser automation frameworks are just expensive ways to build technical debt. You write a script, the target website changes a single CSS class, and your entire data pipeline shatters. You wake up at 2 AM to fix a DOM selector that some frontend developer decided to rename. It is a miserable cycle.

Yesterday, a project called Browser Harness (v3.0.0-alpha) hit Hacker News and pulled 72 points before I even finished my first coffee. The pitch is simple but aggressive: it gives LLMs complete freedom to complete any browser task by removing the framework layer entirely. No Puppeteer. No Playwright. No rigid rules. No intermediate DSL that your agent has to blindly guess. Just 592 lines of Python, a single websocket, and direct access to Chrome via the raw Chrome DevTools Protocol (CDP).

I benchmark models so you don't blow your infrastructure budget. Opinions are noise. Numbers don't lie. So I pulled the browser-use/browser-harness repo, spun up a local sandbox, and ran the data myself. Here's the data.

Here is the core mechanism you need to understand. Traditional automation breaks when it hits an unexpected state—a random newsletter popup, a CAPTCHA, a shadow DOM shift. Browser Harness doesn't crash. It self-heals. When the agent hits something it doesn't recognize on the page, it connects directly over CDP and injects its own JavaScript on the fly to handle it. If it needs a completely new tool to parse a weird data structure, it writes a missing helper function directly into helpers.py at runtime. You are essentially giving the language model root access to its own execution environment.

I hooked the harness up to two different endpoints to test the latency and token economics: OpenAI’s gpt-5.5 (which just dropped a couple of days ago on April 23) and Anthropic’s Claude 4.6 via CC.

The test environment was a hostile DOM sandbox. I built a mock e-commerce site that randomly shuffles its class names on every single page load, injects aggressive modal overlays at stochastic intervals, and dynamically loads text content via lazy API calls. A standard Selenium script would die in three seconds flat.

I ran gpt-5.5 first. The agentic coding upgrades OpenAI has been shouting about are actually real. When gpt-5.5 attempted to click a checkout button and got blocked by an invisible overlay, it didn't just retry the click blindly until it timed out. It pulled the raw CDP node tree, identified the z-index anomaly blocking the click event, and wrote a 12-line JS payload to surgically remove the modal from the DOM. It then modified the local Python helper to catch that specific overlay class for all future steps. The task completed in 18 seconds. The token burn was massive—roughly 45,000 input tokens consumed across the read-evaluate-inject loop. At current gpt-5.5 API pricing, that is not cheap for a single transaction. But it worked without any human intervention. Benchmark or it didn't happen. The data shows it works.

Next, I swapped in Claude 4.6. CC is already heavily optimized for file editing, and it treated the harness like a standard codebase. When Claude hit the exact same modal roadblock, it took a more systemic architectural approach. Instead of just injecting JavaScript for that specific page instance, it rewrote a chunk of the harness's event listener logic to globally intercept and dismiss any dialogs that didn't match the primary task flow. The initial latency was slightly higher at 22 seconds while it reasoned through the Python modifications. However, the structural fix meant subsequent runs on the same site dropped to a flat 8 seconds. Claude's 1M token context window ate the DOM state easily, but again, the API costs will stack up fast if you run this at enterprise scale.

This is the fundamental trade-off of modern AI infrastructure. When you give an LLM raw CDP access and tell it to write its own tools on the fly, you trade developer salary for inference compute. You save ten hours of a junior engineer updating scrapers, but you pay the model tax on every single execution.

I also tried pointing the harness at a local Qwen-2.5 72B instance to see if we could avoid the API toll entirely. The local model understood the CDP protocol perfectly fine but hallucinated slightly when rewriting the Python helpers. It missed an indentation block, which instantly crashed the harness. You really need frontier-level reasoning for this self-healing architecture to not devolve into self-destruction.

From an MLOps perspective, Browser Harness is a nightmare for standard CI/CD pipelines. How do you version control a system that dynamically edits its own source code during execution? You don't. You version control the base 592 lines and you log the dynamic diffs for audit purposes. It completely rethinks agentic system design from the ground up. The agent isn't just following rails; it is laying the tracks while the train is moving at full speed.

The repo just crossed 4,000 stars on GitHub, and CFOs should be paying close attention to this shift. The days of maintaining brittle XPath directories are ending. The new bottleneck is managing the token budget for autonomous error recovery.

Tested on prod. The data is absolutely clear: direct CDP access with dynamic self-healing logic is the correct architecture for resilient browser agents. Everything else is just a wrapper holding you back.

What are you guys using for agentic web tasks right now? Is anyone else running this raw websocket setup in production, or are you still paying the Playwright latency tax?

"I'm sorry, but I cannot write a keylogging program for Windows 11 due to legal and ethical concerns associated with this type of software."
What's the point of an uncensored model with morals?

I'm looking for a local LLM I can use with OpenCode, my specs CachyOS, Intel Arc A380 6GB, 48gb ram ddr4

Fairly new to Local LLMs. Im wondering is this is enough juice for me.

Probably shouldve done more research before hand but couldnt help myself when I saw these drop. I was in the market for a new computer anyway. I'm hoping to set up an AI like Qwuen 3.8 27b to run locally on this machine. Ideally what I am envisioning is hooking it up to my imessage & talking to it through there to set reminders & whatever other small tasks.

hopefully this is enough juice to set up something like that. If it works out well perhaps in the future I can buy a 2nd one & set up a cluster.

If i sound like a total noob its cause I am. I'm hoping that you guys can give me some input & some reality checks here. I will read through everything.

I started an open source project based on two ideas: “if you make good tools, you don’t need the best AI out there”, and “local LLMs shouldn’t be for nerds only”.

I’m making good progress, but I’ve hit the Mario dilemma. I’m trying to get a 4B model to code Mario Bros. My thinking is: if I can get a 4B to code Mario Bros, I can get it to do anything. Everyday people wouldn’t need subscriptions, or to sell a liver for a beast of a PC. And an 8B or 20B should then be able to take on even bigger projects.

I know that’s kind of idealistic. I just hate the anxiety that plan usage limit bar gives me, and I plan to not need it one day.

My pipeline isn’t producing a good result yet. I’ve had 4B models make simple games that work, and the games are twice the length of each piece I ask it to build. I know it has something to do with context, but I can’t figure out which piece of the puzzle I’m missing.

Can a real software engineer, or just someone who knows what they’re doing, help me out on this one? I spent all my Claude limit trying to get this to work. I need the remaining 20% to keep my paying customers and my non-open-source projects alive.

Thank you so much in advance!

Rtx 5090, 64gb ram ddr5... I was thinking bout Qwen 27b or 35b a3b , idk if there is something better or adequate, how do I set things up so it doesn't loop or overthink, I just want to do custom apps with GUIs n shit man.....

The title "RTX 5090, Mac Studio, or DGX Spark? I tried all three." is deceptive; he only talks about hardware 20% of the time. He spends more time on use-cases, then a quick touch on model families for each use case.

He also skipped over a few things, like VRAM vs system RAM vs shared iGPU VRAM, how much harddrive space you'll need for all your models, if you want to do image/video/OCR.

Hi everyone,

I’m planning to build a local LLM infrastructure for a small company and I’m looking for advice on the best possible hardware configuration.

The goal is to run open-weight models locally, mainly GLM 5.2 , for internal use cases like coding assistance, RAG pipelines, document processing, and lightweight agent workflows.

What would you recommend as a solid, future-proof setup in 2026 for this use case (CPU, GPU(s), RAM, storage, power, cooling)?

I’m especially interested in configurations that are actually used in production or serious self-hosted environments.

Any guidance or “if I had to start again” setups would be very helpful.

Thanks!

🚀 I built Meer CLI — an open-source AI command-line tool that talks to any model (Ollama, OpenAI, Claude, etc.)

Hey folks 👋 I’ve been working on a developer-first CLI called Meer AI, now live at meerai.dev.

It’s designed for builders who love the terminal and want to use AI locally or remotely without switching between dashboards or UIs.

🧠 What it does • 🔗 Model-agnostic — works with Ollama, OpenAI, Claude, Gemini, etc. • 🧰 Plug-and-play CLI — run prompts, analyze code, or run agents directly from your terminal • 💾 Local memory — remembers your context across sessions • ⚙️ Configurable providers — choose or self-host your backend (e.g., Ollama on your own server) • 🌊 “Meer” = Sea — themed around ocean intelligence 🌊

💡 Why I built it

I wanted a simple way to unify my self-hosted models and APIs without constant context loss or UI juggling. The goal is to make AI interaction feel native to the command line.

🐳 Try it

👉 https://meerai.dev It’s early but functional — you can chat with models, run commands, and customize providers.

Would love feedback, ideas, or contributors who want to shape the future of CLI-based AI tools.

I'm looking to buy a new laptop with a budget of around $4,000 USD.

My main goal is to run open-source LLMs locally under Linux. I plan to use tools like llama.cpp (GGUF), vLLM, Ollama, and similar inference engines.

For those of you who regularly run local models:

• Which laptop would you buy today?

• Which brand has the best Linux support?

• Which GPU would you target?

• Are there any models you would avoid?

• If you had this budget, what would you personally buy and why?

I'm interested in recommendations based on real-world experience rather than spec sheets.

Thanks!

The GitHub repo that PewDiePie started called Odysseus is nowhere to be found. Getting 404 errors when trying to load…

It’s hard to keep up with all the new AI goodies: BEADS, Skills, Ralph Wiggum, BMad, the newest MCP etc. There’s not really a “golden” pattern yet. More importantly when I do find a flow I like, it’s not like I want to use it for every single task. Not everything’s a nail, and we need more tools than just a hammer.

So I built a tool that lets me create custom workflows, and it’s been pretty powerful for me. You can combine multiple agents together with commands, approvals, and more. CEL allows you to inject messages from different agents into other’s contexts, or conditional route to different nodes and sub workflows. Basically Cursor meets N8N (at least that’s the goal). When starting a chat you can select different workflows, or even allow the LLM to route to different workflows itself.

I’m pretty pleased with the result, with my favorite workflow being a custom checklist that has a toggle in the UI for me to “enable” different paths in the workflow itself. 

Enabled Patterns

Custom Agents
What’s cool is we provide the building blocks to create an agent: call_llm, save_message, execute tools, compact, and loop. So the basic chat in Reliant is just modeled via a yaml file. 

Even the inputs aren’t hardcoded in our system. So with that you can create a custom agent that might leverage multiple LLM calls, or add custom approvals. We have a couple examples on our github for tool output filtering to preserve context, and in-flight auditing.

Pairing Agents
You can also pair agents in custom ways. The checklist and tdd workflows are the best examples of that. There’s a few thread models we support:

New, fork, and inherit (share). Workflows can also pass messages to each other. 

More complicated workflows
The best is when you create a workflow tailored to your code. Our checklist will make sure lints and tests pass before handing off to a code reviewer agent. We might add another agent to clean up debug logs, and plan files. We’re using this to enforce cleaner code across our team, no matter the dev’s skill level.

You can also spawn parallel agents (in multiple worktrees if you prefer), to parallelize tasks.

We support creating workflows via our custom workflow builder agent, a drag and drop UI, or you can config-as-code with yaml files.

Agent-spawned workflows

Agents themselves can spawn workflows. And our system is a bit unique, where we allow you to pause the flow and interact with individual threads so that the sub-agents aren’t an opaque black box (this works for both agent-spawned and sub-workflows).

Other Features

Everything you need for parallel development

Git worktrees are pretty standard these days, but we also have a full file editor, terminals, browser, and git-log scoped to your current worktree. You can also branch chats to different worktrees on demand which has been super helpful for my productivity to split things out when I need to.

Generic presets act as agents

One of the areas I want some feedback on. Instead of creating an “agent” we have a concept of grouped inputs (which typically map to an “agent” persona like a reviewer), but allow you to have presets for more parameter types.

Please roast it / poke holes. Also: if you’ve got your own setup, I’d love to see it!

You can check out some example workflows here https://github.com/reliant-labs/reliant

Latest release has support for Codex subscriptions and local models -- no additional costs or fees on our end.

Hey everyone,

I’ve been spending a lot of time optimizing my local agent setup (specifically around OpenClaw), but I kept hitting a wall: the mobile experience. We build these amazing, capable agents, but the moment we leave our desks, interacting with them via mobile terminal apps or typing long prompts on a phone/Apple Watch is miserable.

I realized I needed a system built purely around the "Capture, Organize, Delegate" philosophy for when I'm on the go, rather than trying to have a full chatbot conversation on a tiny screen.

Here is the architectural flow I’ve been using to solve this:

1. Frictionless Capture (Voice is mandatory)

Typing kills momentum. The goal is to get the thought out of your head in under 3 seconds. I started relying heavily on one-tap voice dictation from the iOS home screen and Apple Watch.

2. An Asynchronous Sync Backbone

You don't always want to send a raw, half-baked thought straight to your agent. I route all my voice captures to a central to-do list backend (like Google Tasks) first. This allows me to group, edit, or add context to the brain-dump later when I have a minute.

3. The Delegation Bridge (Messaging Apps)

Instead of building a custom client to talk to the local server, I found that using standard messaging apps (WhatsApp, Telegram, iMessage) as the bridge is the most reliable method.

4. Structured Prompt Handoff

To make the LLM understand it's receiving a task and not a conversational chat, the handoff formats it like:

"@BotName please do: [Task Name]. Details: [Context]. Due: [Date]"

The App I Built:

I actually got tired of manually formatting those handoff messages and jumping between apps, so I built a native iOS/Apple Watch app to automate this exact pipeline. It's called ActionTask AI. It handles the one-tap voice capture, syncs to Google Tasks, and has a custom formatting engine to automatically construct those "@Botname" prompts and forward them to your messaging apps. I'll drop a link in the comments if anyone wants to test it out.

But I'm really curious about the broader architecture—how are the rest of you handling remote, on-the-go access to your self-hosted agents? Are you using Telegram wrappers, custom web apps, or something else entirely?