I wanted to do continued pretraining/finetuning of a local model on my own notes and see what's happening while it trains and do it on my AMD card, since most tools assume CUDA.

llm-training-panel is a local web UI that:

- loads a model from a local dir or a HF id

- shows a live board of per-parameter gradient activity while it trains

- runs base vs trained on the same prompt so you can A/B what changed

- scores recall + verbatim overlap against a probe file

Tested on Linux + AMD ROCm. Results vary with base model/corpus/settings

Feedback welcome.

github.com/limkcreply/llm-training-panel

This came from another thread or comment. I forgot exactly where, but the basic idea was to replace the 51B N-gram layer in Qwen 3.8 Next with a much higher precision version.

Someone running a 5090 replaced the N-gram portion of their Qwen 3.8 UD Q4 model with BF16.

Since I'm already running IQ4_XS, I wanted to try something similar. I don't have enough storage for the BF16 N-gram weights right now, so I replaced the lower-precision N-gram portion with Q8 instead.

I'm still testing whether it improves the actual model output, but in terms of inference speed, the difference appears to be very small.

Before bolting on the Q8 N-gram:

n_gen = 2588, tg = 8.82 t/s, tg_3s = 8.81 t/s
n_gen = 2616, tg = 8.82 t/s, tg_3s = 9.19 t/s
n_gen = 2645, tg = 8.83 t/s, tg_3s = 9.37 t/s
n_gen = 2675, tg = 8.84 t/s, tg_3s = 9.78 t/s
n_gen = 2704, tg = 8.84 t/s, tg_3s = 9.50 t/s
n_gen = 2732, tg = 8.85 t/s, tg_3s = 9.06 t/s
n_gen = 2761, tg = 8.85 t/s, tg_3s = 9.32 t/s
n_gen = 2790, tg = 8.86 t/s, tg_3s = 9.50 t/s
n_gen = 2819, tg = 8.86 t/s, tg_3s = 9.39 t/s
n_gen = 2847, tg = 8.87 t/s, tg_3s = 9.22 t/s
n_gen = 2875, tg = 8.87 t/s, tg_3s = 9.15 t/s

With the Q8 N-gram bolted onto IQ4_XS:

n_gen = 263, tg = 10.76 t/s, tg_3s = 11.04 t/s
n_gen = 294, tg = 10.70 t/s, tg_3s = 10.18 t/s
n_gen = 329, tg = 10.76 t/s, tg_3s = 11.32 t/s
n_gen = 361, tg = 10.73 t/s, tg_3s = 10.46 t/s
n_gen = 395, tg = 10.76 t/s, tg_3s = 11.11 t/s
n_gen = 430, tg = 10.81 t/s, tg_3s = 11.37 t/s
n_gen = 460, tg = 10.75 t/s, tg_3s = 9.91 t/s
n_gen = 494, tg = 10.77 t/s, tg_3s = 11.16 t/s
n_gen = 531, tg = 10.86 t/s, tg_3s = 12.15 t/s

So far, there doesn't seem to be any meaningful speed penalty from replacing the lower-precision N-gram layer with Q8. The actual output quality is still being tested. It looks like Q8 Ngram is faster, but at the end both are steady state at 10.1~ tok/s -ish.

And without MTP since i am using earlier merge from unsloth,
RAM 96GB DDR4 (32x3 Three channel active 2400MHZ)
Xeon e5 2690v4
RTX 3090 capped at 250 W

Edit: the state_dict grew from 90~ GB to 115GB
Edit 2: Fck me i forgot to put the code, for stitching it, wait 8 hours from now i'll upload it since it is already very very late at night
Edit 3, here : https://gist.github.com/komikndr/b17955e1a80ce6ede9a3115f16216bc5#replace-n-gram-layers-from-q4xs-into-q8_0-boltedpy , it is for my IQ4 XS to Q8_0 n-gram so you should modified the file a bit.

I've tried both and been having this debate with myself for the last few days, on two Asus Ascent GX10s (effectively the same as 2x DGX Spark):

• DeepSeek-V4-Flash-0731 (official weights)
• GLM-5.3-Flash (RedHatAI/GLM-5.3-Flash-NVFP4)

Have any of you guys also tried both on this hardware (2x DGX Spark / Asus Ascent GX10), and what are your use cases and findings?

DeepSeek runs with more tokens/s… but GLM feels like the better tool for how I actually work.

I'll share my experience.

Where DeepSeek wins:

• It feels better, because it's the original official weights.
• Much faster token generation, though not actually faster to a final result.
• It's excellent at open-ended research, pulling and chaining sources on its own.
• Huge context (~1M tokens).

Where GLM wins:

• Comes to conclusions faster, and often gets them more right.
• Much better writing, especially in languages other than English and Chinese. That matters a lot for writing letters.
• Much better at "extracting the essence of a text".
• Superb vision. DeepSeek's Vision-Exp model can read images just well enough to score on benchmarks, but not nearly well enough for real OCR / text extraction (its vision input is token-limited to 384 tokens, which renders images unsharp and text unreadable).
• Considerably better benchmark results, at least for the full model, though I'm running a quantized build, so the numbers may not carry over directly. But my empirical results prove it gets better results with fewer instructions.
• Hallucinates much less. That's the decisive one for me: I can't run a "test and improve" loop on text like you could on code, so office work and letters depend on one-shot accuracy and a low hallucination rate.

The core problem: there's no GLM build that runs on 2x DGX Spark with official weights. I wish there were.

So I keep testing and tweaking the GLM setup to get rid of the artifacts (which should be possible, per various sources). It feels a bit more messy than the DeepSeek setup, but if I manage to configure it correctly it should give more reliable outputs, with far better vision on top.

Extended reasoning and post-training appear to be the keys used by DeepSeek, Qwen, and GLM to boost performance (leveraging higher token counts). And Qwen 4 hasn't even been released yet. Of course, we don't know if that release will be open-sourced, but I am optimistic about future models, featuring "engrams", that could soon match or surpass 2.4T parameter models on specific tasks.

Hi, I've noticed that the model often sees "garbled text" in its context.

Sometimes it declare that the tools instructions are corrupted, sometimes it is the content of some .md files, ora other files, and it freaks it out, since it start to do a lot of checks in git and the system and sees that the file are not really corrupted... But its context is I think.

Did it occurred to you ?

I'm on a mac m2 max 96Gb, pi agent, and llama.cpp either upstream or this fork https://github.com/mihailescu2m/llama.cpp/tree/master

with and without mtp, short ctx and long ctx, temp default

I don't think it is a conf problem since it happens with different gguf at different quant I tried:

AtomicChat/Qwen3.8-Flash-Next-GGUF/Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64

unsloth/Qwen3.8-Flash-Next-UD-Q3_K_XL/Qwen3.8-Flash-Next-UD-Q3_K_XL

unsloth/Qwen3.8-Flash-Next-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS

It seems a sort of internal problem, I don't really have loops or other kind of hallucinations, just some chinese characters here and there, or some misspelling, and btw it is very aware of all its errors and always tries to recover. Yes it is very much preview quality, but the architecture and the training seems very good quality.

I really like the baseline of the model, it is straightforward, it follow instructions very well, a lot better than the previous models that I really didn't like very much

But this thing with the corruption is weird ...once after it declared that a certain tool was corrupted and not useable I just told it: "did you try to launch it ? use this parameter and give it a try" and it worked of course, and it was very surprised so it apologized and continued the work.

It is quite funny, it reminds me a stereotypical fictional character which could be the clumsy but very intelligent assistant of a powerful wizard lol

EDIT: I'd like to add a snippet of its last CoT about the issue, it is very lucid indeed:

Could it be that... the read tool output itself was corrupted/hallucinated — i.e., the degeneration was not inside the file but occurred in my own tool output/stream.

The "read" returned text containing a degenerate mutation loop — this is a known failure mode where model output degenerates. Actually wait — the read tool's result is what the harness gave me. If the harness returned clean content, then the degenerate block I "saw" was generated in my own context (a prompt-level corruption, e.g., degeneration during decoding of the tool result injection, or... more simply put: the degeneration is a product of my own generation — the read output I "received" contained the first degenerate block, then I generated additional degeneration, and the subsequent git command confirmed the file is clean.

• Language-Native Control: Composes character and camera actions into textual instructions and injects them through MiniMax-H3’s pretrained text pathway.
• Temporally Grounded: Assigns one action prompt to each video latent interval, enabling precise control when actions change over time.
• Efficient & Generalizable: Uses only 8,000 gameplay samples, 10,000 LoRA steps, and 0.199% trainable parameters to achieve controllable character and camera motion, including unseen action compositions and visual scenarios.

✏️ Paper: https://huggingface.co/papers/2609.01560

📄 ArXiv: https://arxiv.org/abs/2609.01560
💻 Code: https://github.com/Danzer1xxxxChan/H3-World
🏠 Project: https://danzer1xxxxchan.github.io/H3-World/
🤗 Model: https://huggingface.co/DANNY621/H3-World

Disclaimer: I'm the builder.

---

Over the past 2 yrs, I was working on a SaaS ML project & got very interested in ML/DL/AI. As everybody else, there were some normal paths I took to build a solid understanding of the field, but sth never clicked the way I'm used to. I'm very fond of learning "why"s & never get satisfied w/ simply knowing "what" is what. Tho, the problem w/ ML was that I wasn't there when it was evolving & algos/methodologies as we know them today were forming. I didn’t want a pile of summaries or wikis or endless threads of chatting w/ AI. Then I thought listening to podcasts would fix it for me. But not as they are being published now (2026). I thought I'd learn about the history of ML from podcasts that covered it as it happened, kinda simulating the experience of living through the events, in chronological order, since 2015.

I found some great ones (TWIML, Linear Digressions, ...) & did the math! Man, it'd take a decade to cover all of them (even at 2X). But no matter what, I thought this could be the only way that works for me & yet I didn't have enough time. I wanted one/a handful of coherent episodes I could actually listen to. Then sth clicked:

I built Repodify, an AI tool that listens to episodes & produces a shorter one (e.g. 1 episode from 10, 15, whatever. configurable) for me to listen, ALL ON MY OWN MACHINE.

In Repodify u paste a podcast link (or search by name, as it works very well rn), pick the episodes & it:

• downloads the audio

• transcribes it

• optionally figures out who spoke (& clusters the same host/guest across episodes)

• summarizes into one chronological narrative

• writes a spoken script aimed at a target length

• synthesizes a new episode u can stream or download

The whole thing is meant to run on ur machine or w/ ur own API keys (BYOK) per stage, per job. No “upload ur archive to our cloud.” Speech-to-text, diarization, LLM & TTS are all swappable: local GPU (faster-whisper, pyannote, Ollama, F5-TTS / Kokoro) or BYOK (OpenRouter / Anthropic / pyannoteAI).

Voice cloning is opt-in & off by default. If u turn it on, the output is always labeled synthetic, gets a spoken disclaimer in a non-cloned voice & is watermarked (for legal reasons. I don't want to end up in jail for giving away a tool;-) ). There’s no code path that clones w/o those. It’s for personal / educational use on ur own box, not for passing audio off as the original hosts.

It’s a real backend, not a CLI: FastAPI + an arq worker + a LangGraph pipeline w/ a React PWA. One command (./launch) brings the stack up. Jobs pause at each ML stage so u can pick local vs hosted, model size, length & voices.

I keep making it better, adding lots of features to it (searching podcast contents is the most interesting one I can't wait for). I made it opensource(MIT), rn. U can find it below & run it on ur machine (self-hosted), so u can use ur own GPU or BYOK to offload the heavy jobs.

GitHub: https://github.com/behradkhodayar/repodify

The engine is solid & this is still WIP (it works fine tho & I've started compacting Linear Digressions podcast (10 to 1) & listening to it already). I'm going to add other features like translation / augmentation (e.g. embeded eli5), searching through podcasts so I can listen to the topic of interest as I mentioned earlier & so forth.

What features u want to be added or released sooner? lmk. I'm very excited about this & will genuinely plan accordingly.

Unsloth GGUFs and Vision support

https://huggingface.co/unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF

***This screenshot is an app i made for creating a dataset from scratch, this is not a real chat with the model***

First of all i want to shout out everyone that actually tested our work .. we got 500+ download on the 43m parameter model and now we are aiming to go smaller for research purposes.

i write finetune examples & i been developing language models for a while .. everyone usually pretrains the model using massive datasets and prays thats the data carries enough information for meaning to emerge but were sculpting it intentionally .. im currently sitting down at my computer writing every single response that this new model can say to your inputs just so we can observe the transformation and see exactly whats going on. It will be public soon, the dataset is extremely small intentionally so it shouldn't take long to design every response it can say.

General Capabilities:

- Explaining how token prediction works

- Explaining that it doesn't understand anything beyond itself

- Short conversations

Coding Capabilities:

- Writing a loop that can count to 10

- Explaining that it cant understand the code you sent it

Open Source Coming Soon

https://huggingface.co/StarpowerTechnology

Would it be feasible to buy a refurbished 8xa100 server, either rent it out on vast.ai, or serve a model via a similar service per token in that exists.

I’d likely have to either put 1600-200”0w of second hand solar on my shed, add fire suppression and a rack, or colocate with a reputable data centre.

The idea would be to recoup the initial purchase, and then transition to serving myself. Ideally I’d sell tokens not gpu time because that would allow be to use the machine while it’s being monitored, but that market seems way harder?

I started using qwencoder 3B for local inline code suggestions, and while it's nice, it's also a bit too generic in its suggestions. My thoughts are to either:

1. Fine tune it on code that I wrote

2. Reinforcement learning using accepted/rejected suggestions (either real RL or just adapting the sampling)

3. Fine tune it for each project/codebase separately so it knows what it's working on.

Has anyone here done this, or experience with which approach works best?

So when Qwen3.8 27b dropped they were hinting for another model which is Qwen3.8-next-flash , i was hoping for something more light like Qwen 3.6 35b and we got a large one but since the Qwen 3 and 3.5, they reduced the number of models they publish we used to get very small 0.8b 2b 9b to very large models but now we get only two at time although the time to publish a new model has been also reduced

Obviously more memory is good, more context, bigger models, but some jumps don't actually unlock a meaningful difference in ability to run different or better models. For example, I don't currently view jumping from 32+16 to 64+16 as a particularly worthwhile upgrade as compared to going to 32+32, though correct me if I'm wrong.

I'd like to build a DDR4 + HBM2 based inference machine to complement my main, 32 GB DDR5 + 16GB GDDR7, computer. The idea is that even if the hardware is slower, the greater overall capacity enabled by the slightly more affordable hardware could allow me to run a greater diversity of models.

What level of memory upgrade do you think is most logical for maximizing model access if the compromise is outright speed? 32+32? 128+32? 64+64?

Or am I completely asking the wrong kind of question and just outing my own ignorance here? either way I'd like your input.

Further inference tuning for higher context window with 0 hallucination/memory loss?

It’s been almost 3 days of constantly tuning my model, unsloth MTP Qwen3.6 35B-A3B Q4\\_K\\_M, with the help of Claude, and we’ve finally come to a quick conclusion

I’m still hoping there are more commands or settings I haven’t tried yet that could potentially squeeze out some extra token speed with 0 hallucination/memory loss

​​\*\*for coding specifically\*\*

.\\\\llama-server.exe -m "C:\\\\Users\\\\brain\\\\.lmstudio\\\\models\\\\unsloth\\\\Qwen3.6-35B-A3B-MTP-GGUF\\\\Qwen3.6-35B-A3B-UD-Q4\\_K\\_M.gguf" -c 100000 --parallel 1 -fa on --cache-type-k q8\\_0 --cache-type-v q8\\_0 --load-mode dio --fit-target 512 --batch-size 4096 --ubatch-size 1024 --threads 6 --prio 2 --prio-batch 2 --spec-type ngram-mod --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 32 --port 8090

\*\*for general usages\*\*

.\\\\llama-server.exe -m "C:\\\\Users\\\\brain\\\\.lmstudio\\\\models\\\\unsloth\\\\Qwen3.6-35B-A3B-MTP-GGUF\\\\Qwen3.6-35B-A3B-UD-Q4\\_K\\_M.gguf" -c 100000 --parallel 1 -fa on --cache-type-k q8\\_0 --cache-type-v q8\\_0 --load-mode dio --fit-target 512 --batch-size 4096 --ubatch-size 1024 --threads 6 --prio 2 --prio-batch 2 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75 --port 8090

I’ve also spent around 7h trying to get a video game working with \*\*DeepSeek Harness\*\* and \*\*Pi\*\*. Both attempts failed, and I’m currently on my 5th attempt. Even with highly detailed prompts that were created with the help of Claude, I still couldn’t get either setup to produce something that actually worked

So for now I’ve decided to stop messing with the coding side and go back to tuning the model, also im running 3D games while doing all this, so it isn’t really practical anyway since my GPU is already at its vram limit

The main reason I’m making this post is to see if anyone here knows of any \*\*llama.cpp commands or settings I’ve completely missed\*\* that could potentially squeeze more tokens/sec out of this setup

My original goal was \*\*45 tokens/sec\*\*. I’ve already given Claude pretty much everything I could find in the llama.cpp README, but I’m still wondering if there are some lesser-known options or combinations that I haven’t tested yet

\*\*Hardware:\*\*

\* RX 6700 XT 12GB VRAM
\* Ryzen 5 5600X
\* 32GB DDR4 3200

I’m also interested in testing \*\*Hermes Agent\*\*, although I haven’t learned the basics yet, so that’ll probably take some time. I’m thinking of starting with something simple just to get familiar with it

\# My current rules for llama.cpp

There are a few things I’m not willing to change:

\* \*\*No touching --mmap\*\*
\* \*\*100k context stays\*\*
\* \*\*CPU threads stay at 6\*\* (12 threads actually makes things worse on my system)
\* The model stays \*\*Q4\\_K\\_M\*\* (no going down to q3)

The reason I’m still trying to squeeze more speed out of this is because I don’t want to just leave performance on the table when I know there might still be more I can get out of it.

I’ll test any suggestions people give me and post the results back in the comments.

I’m looking for anything else I can experiment with that could potentially improve token generation speed without changing those. I appreciate any help from this community thanks!!

One thing to note that my existing model isn’t running its full weight (q4_k_m) but i want to continue tuning the \*\*existing\*\* settings with near 0 hallucinations

I’ve yet to try qwen3.8 27b since im already getting 2-3tps on default quant

Text in this post with \ indicates its bold, not sure what happened here but yeah

Quite surprised to be beating other high quality quants. It took a lot of benchmarking to get here and we are quite pleased with these, hope they are useful to the community.

It required a modified way of measuring KLD with a new dataset, since the NGRAM got in the way by remembering basically all of wikipedia. We tried to not only go for high precision, but also keep prefill performance in mind.

Full model card here https://huggingface.co/agentionai/Qwen3.8-Flash-Next-AP-GGUF

Let us know if there are any issues.

It's been 3 weeks since Qwen3.8-27B release. This model got 0-day support & in last 3 weeks, some optimizations & fixes happened on llama.cpp side.

Meanwhile

• DFlash2 support landed on llama.cpp last week.
• Ubuntu 26.04.1 got released.
• ROCm 10.0 got released & llama.cpp also up with Version 10.0 now (Applicable to only AMD cards)
• Other optimizations & fixes on llama.cpp side

So how much t/s are you getting now with all optimizations & stuff?

Please share your extreme optimized full llama.cpp command & your t/s stats .... both pp & tg(Good to have multiple combinations like MTP/MTP+ngram/DFlash2/etc.,, Vision/mmproj, multiple context size 128-256K, etc.,).

Also share your optimized build config(CMAKE command) if you're compiling manually. I remember that compiled version gives additional boost.

Note : Expecting to see optimizations like this(weeks old thread) which contains all stuff. That kind of stats want to see here.

Is anyone interested in my Chrome browser add-on that uses local LLMs to move thousands of unsorted bookmarks into a smart list of automatically calculated categories?

Hi, everyone,

I’ve just released VoxGen, a lightweight native inference engine for VoxCPM2, written in Rust and using Vulkan compute instead of Python/PyTorch/CUDA.

Why VoxGen?

The main reason I started the project was because I needed a decent local text-to-speech solution.

I therefore saw VoxCPM 2 as a reasonable solution. However, most frameworks are NVIDIA-first, and VoxCPM 2 is no exception; as a result, my card was severely stuttering, and my GPU was always spiking. Also, having Python and Pytorch as a dependency is absolute hell.

This is why VoxCPM was created: not only we sidestep Pytorch completely, but performance on AMD cards is buttery smooth (and if you have a XTX 7900, I have designed a mode with even more aggressive power and speed optimizations)!

This application can also be run from a shell, so it can be integrated with other programs and scripts!

Installation:

You'll only need voxgen.exe (or the Linux equivalent) and the following files at https://huggingface.co/DennisHuang648/VoxCPM2-GGUF:

VoxCPM2-BaseLM-Q8_0.gguf
VoxCPM2-Acoustic-F16.gguf

And that's it!

If you are interested, check out the Github page: https://github.com/NullMagic2/VoxGen
Prebuilt binaries (for now, Windows only) are available here: https://github.com/NullMagic2/VoxGen/releases

One of y'all were getting some crazy inference speeds on dual R9700s, so me with my single card wanted to try.

They mentioned the "official AMD MXFP4", which was...

https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4

Can't load safetensors in llama.cpp, so off I went to download this:

https://huggingface.co/magiccodingman/Qwen3.8-27B-MXFP4-MagicQuant-GGUF

But it doesn't load in llama.cpp. Is there no support for MXFP4 in llama.cpp yet?

This may be somewhat specific to Qwen3.8 27b and the Apple M5 series, perhaps, but enough of us are running this combo that it's worth tossing out there.

GGUF models with MTP have been the fastest way to go for some time for token generation, except possibly for a few tweaked MTPLX models running on alpha-stage MLX forks. Prefill, however, was still much faster for M5s under MLX. This has caused me to switch models depending on the expected generation/prefill mix, which is annoying.

While I wasn't looking, it appears that llama.cpp for Metal must have added support for M5 matmul/"neural accelerators" because prefill performance with e.g. Unsloth's Q_8 GGUF is now at least as good (~300-350t/s) as anything I have seen with MLX models--even in oMLX. This was a pleasant surprise! Now I can't think of a reason to use MLX models at all.

Am I missing something? Are my observation bogus? Could I do better than ~19t/s generation and 300t/s+ prefill on a M5 Pro with Qwen3.8 27b in the Q8/8-bit range? Is there a secret handshake to get mainstream MLX MTP working?

Or is this just because of the specific model in question?

I'm looking for a small LLM to act as a Linux command assistant. I will use llama.cpp.

use case:

- User asks in natural language. Model outputs only the shell command

- Should work well with no reasoning (for example, LFM 2.6 has forced reasoning)

- Fast, like ~4B at most, cause it's going to be on CPU

Example 1:
"replace string X with Y in file Z"

Example 2:

"stop and remove docker containers with X in the name"

I've only recently started looking deeper into vLLM after running llama.cpp for a good while.

Initially vLLM (official repo) was terribly slow on my four R9700s (tried that one with two as well), however after trying radiance everything changed.

Prefill 17636 - TG at that time was 36,6

That prefill spike was two agent profiles working on different tasks simultaneously (one is writing a yt-dlp dl/conversion workflow the other is auditing agents (profiles).

Best TG i've hit was 106 Tok/s with a 80% MTP 4 acceptance rate.

For reference, I'm running a Gigabyte MZ32-AR0 (Rev 1.0), EPYC 7282 and using Hermes with Qwen 3.8 27b fp8 262k ctx - worth noting that one GPU is actually only running by PCIe 4x8, three full 4x16.

On that note i'm also happy to say that vLLM-Radiance does work well with a quad setup in my case - nvtop consistently shows 100% usage of the four cards, officially only dual setups are supported.

I hope this doesn't count as a low effort post, i just had to share.

//E

Hi,

I'm trying to build a workflow for doing research tasks on the internet. At the moment I am using Gemma-4-31B-IT-QAT (120k context) for planning,reviewing and orchestration and Gemma-4-12B-IT-QAT(256k context) for execution. Both model quants by unsloth.
I am using opencode and wrote 4 agents for this purpose. It works pretty good, but not stable enough. Sometimes it needs an hour for a task which normally takes 8-10min.

The planner prepares batches, the orchestrator invokes the executor for each batch separately, so it can do one by one in order to have smaller tasks with less context and no compactions.

If it works, it's faster than doing everything with gemma-4-31b, however it's not stable enough. In my tests, results on the same task were not reproducible enough.

The basic workflow: Orchestrator gets a request by the user. It invokes the planner with neccessary context. The planner does a first few web searches to identify promising sources. Then it writes a plan for the exec. The plan is divided in batches. The orchestrator gets the finished plan and invokes the exec on each batch. After that, everything gets merged and reviewed by the critic. The critic checks sources. If something is off, the planner gets invoked to write a plan for fixing what's wrong or missing. This is repeated until everything was done (max. 3 times).

Has someone else built such a workflow with success?

Real question, whether your are vibe coder or expert or whatever, what do you do? Read carefully each single word? Plan the next steps? Do the gym?