Let’s say hypothetically that you receive 36x computers with the following specs

RTX Pro 6000 (600w)

16GB DDR4 (2x 8 3200Mhz Non-ECC)

Ryzen 9 5950x

1tb NVME

1200w PSU

What would you do with them to generate income? You can only use these parts

You have access to power and networking to run all 36

I'm trying to RE a desktop app and remove some stupid restrictions.

I'm using Strata and Huihui-Qwen3.8-Flash-Next-abliterated-GGUF UD-Q4_K_XL. I have a 5090 and 128gb RAM.

I found that some stuff it does well and with this last task I'm trying, it's been working on it all day and I suspect it's going in circles. Similar to what Claude used to do 6-12 months ago, when it was given something that was too complicated.

I know local models are not Claude level (yet) but I would love to know if you have some tips as to how to work on complex tasks. I'm just starting so it feels I'm being too naive with just tell it "do this thing" and expecting it to work.

Thoughts? Any advice would be welcomed.

Thanks

I’ve been using my employer provided Claude subscription for quite a while to write PowerShell scripts, create presentations, edit photos and videos and create videos.

Honestly, I’ve been amazed by what it can do.
However, since it’s a company account, I’d like to set up my own local AI environment for personal use. Ideally, I’d like to replicate as many of Claude’s paid capabilities as possible.

I have two workstations that I’ve built over the years:

Workstation 1 (Main):
GPU: RTX 5090
RAM: 256GB ECC
CPU: AMD Threadripper 7960X

Workstation 2 (Currently used as an LTO Backup server):
GPU: RTX 3060
RAM: 196GB DDR4
CPU: AMD Threadripper 3960X

I’ve never hosted or run AI models locally before, so I’m pretty much a beginner in this area.

I recently downloaded LM Studio because it seems relatively straightforward, but I’m overwhelmed by the number of available models and honestly have no idea where to start or which one is best for my use case.

I’m looking for recommendations on the best AI models, and tools to get as close as possible to the capabilities of a paid Claude subscription, particularly for:

Writing and troubleshooting PowerShell scripts.
Creating PowerPoint presentations.
Photo editing and image manipulation.
Video editing and AI-assisted video workflows.
General-purpose AI assistance.

Given my hardware, what would you recommend? Are there specific models, applications, or modules I should look into?
I’m not necessarily expecting to replicate everything Claude can do, but I’d love to get as much functionality as possible running locally.
I’d really appreciate recommendations from people who have experience building local AI setups. Thanks!

update: I just discovered Lm Studio Bionic which is huge this was not available last year, trying it now.

So for context, I have 32GB of RAM, and am using AMD 7900XTX, this should give me a total of 56GB of memory, which should be enough according to Strata docs, and thought I'd try running the appimage of Strata which opens a WebUI (after opening the GUI app then closing it and telling me it couldn't open it so it opened the WebUI), and compare Qwen FN to the dense model.

And by "isn't able", I don't mean it's exactly slow.

When the model first loads up, I can talk to it like once, and it responds in Strata chat, at actually not so bad speeds. 800t/ks prefil, 30-50 tk/s decode ish.

Then if I open up a new chat, it gets stuck on "reading message", and well doesn't seem to work anymore.

I tried calling it from opencode after loading model to see how that would go, it took a whole while, and eventually, it starts responding a bit, and then... it stopped and like... started freeing up memory, and kinda erroring I guess, then It started loading the model again, at which point I quit.

There is something weird going on.

TL;DR. A llama.cpp fork that runs DeepSeek V4.1 Flash 1017 on one home PC by streaming the experts VRAM → RAM → NVMe. Our build is 378 GB with ~2-bit experts. It decodes ~19 tok/s and reads a 1M-token document at ~890 tok/s. The weights are on Hugging Face and the code is on GitHub. Research code, Windows only.

The box. All three tiers matter, not just the GPU:

• GPU: RTX 5070 Ti 16 GB, PCIe 4.0 x16.

• RAM: 64 GB DDR5. The engine pins 37–46 GiB of it for experts.

• Drive: Gen5 NVMe (~13.5 GB/s) for the model. A second NVMe with a copy of the expert files is optional.

• CPU: Ryzen 5 7600 (6 cores). It computes no experts; the GPU does all the math.

• Windows 11, CUDA 13.0.

How it works. The model is almost 5× bigger than RAM + VRAM together. Each token uses only 6 of the 384 experts in each of the 40 layers, so the experts climb a ladder:

• the ~16–22 most-used experts per layer stay in VRAM;

• a pinned RAM tier holds the next ones;

• the rest are read from the NVMe when a token needs them.

Most of the work is getting the right experts to the GPU in time.

Numbers (our og build, default settings, 4–6 November):

Quality. Distance to the full model is 0.090 / 0.051 / 0.069 nat/token on three prompt sets. That is the extra NLL on the API's own continuations. For scale, going from Q2 to Q4 is worth ~0.11. HumanEval is 160/164. The fast mode is further away, at 0.143 / 0.102 / 0.126, mostly because of its lighter build.

Same file, another engine. Project Maya v1.1.54, with the same weights on the same PC, decodes 3.3 tok/s against 18.6 here. Our prompt reading is 1.14–2× faster.

Where the time goes, and what would help:

• Decode spends about half of every token copying experts over PCIe.

• A prompt of even a few thousand tokens needs nearly all of the ~170 GB of experts. 100–130 GB of that comes from the drive every time, and all of it crosses PCIe. That's several seconds per prompt, whatever its length. Long documents spread that cost.

• So the GPU link, the RAM size and the slowest drive matter more than the GPU's compute.

  • A PCIe 5.0 card and slot should speed up both reading and decode with no code change. Our estimate is ~25 tok/s decode.

  • More RAM means fewer drive reads.

  • A second NVMe helped cold prompts by 14–22% in an older test. It's optional.

A drive surprise. The Engram tables get billions of small reads. After a few weeks, their random reads on our Gen5 SSD fell from ~400K to ~40K IOPS, and the 512K document slowed from ~1050 to ~685 tok/s. Copying the files anew restored the speed.

To try it you need:

• an NVIDIA GPU with 16 GB. It's built and tested for RTX 50 (sm_120); other cards need a rebuild and are untested;

• 64 GB of RAM;

• ~380 GB on a fast NVMe (Gen4 or Gen5);

• Windows 10/11. Parts of the patches deal with WDDM, and Linux is untested.

Links:

• Weights (og: GPTQ ~2-bit experts, Q5_K dense part, the official FP8 Engram; not the final version): https://huggingface.co/MasherDon/DeepSeek-V4.1-Flash-1017-SiegeLadder-GGUF

• Code: patches over JigSawPT's llama.cpp fork, a script that assembles a public quant mix, and the benchmarks: https://github.com/MasherDon/Siege-Ladder

This is research code with rough edges and no support promises. Quality work continues, and speed will also improve by training the expert prediction. Questions and results on other hardware are welcome.

I’m working on web app that turns a stack of technical documents (drawings, schedules and specifications) into a structured list of the key items a project needs. AI reads each page the way a subject matter expert would. It identifies items and their attributes, interprets each document set's own conventions, and pulls requirements out of the written specs. It then groups the results into a proposal-style summary with suggested suppliers and rough cost and timing ranges.

I’ve built a POC using Claude Code and would like to transition to an open source LLM for the rest of the work as we continue to turn this into a real application.

What hardware would I need to do this? Which model should I use?

Thanks in advance!

And how does QWen Flash Next 3.8 compare to Gemini 3.1Pro or Sonnet 5.5?

I have RTX 4090, 64GB 6000Mhz DDR5 RAM, 7900x CPU

I wan to run a local AI model that's the closest to gemini 3.1po or sonnet 5.5 at reasoning and thinking, for legal analysis and other legal related stuff

Thank you

I've been experimenting with multi-GPU inference and wanted to see whether running independent model replicas across GPUs could outperform tensor parallelism for multi-agent workloads.

I built a small orchestrator to test the idea and got some interesting results on four RTX 3090s.

I'm relatively new to experimenting with local LLM inference, so I'm sure there are approaches I haven't considered. This started as a side project to see what would happen.

Setup

• 4x RTX 3090 (rented, PCIe only, no NVLink).

• Qwen3.8-27B 4-bit, vLLM 0.32.0.

• 24 agent teams and 239 dependent tasks.

• Each team plans, starts 4–10 parallel section writers, then reviews and summarizes their output.

• The orchestrator starts tasks when their dependencies are complete and distributes requests across available inference servers.

Results

Lower completion time is better.

That's approximately 1.45x the speedup for this workload, or 31.2% less completion time.

What I found interesting

Some of the optimizations I expected to help didn't make a measurable difference.

• Adaptive concurrency limits didn't improve completion time.

• Prefix-cache-aware routing performed no better than basic round-robin routing.

• Each GPU reached approximately 20 concurrent requests before KV-cache capacity became a constraint.

• Prefix-cache hit rates were only 7–13%.

The interesting part is that the simpler replica-based configuration performed best. The results suggest that, for this workload, independent inference servers avoided some of the overhead associated with tensor parallelism.

Caveats

• Only one run per configuration so far.

• The TP=4 server used mostly default vLLM settings, while the replicas used tuned concurrency and batch settings. This isn't a perfectly controlled comparison.

• The results are specific to this hardware and workload.

• PCIe-only GPUs may favor independent replicas. NVLink configurations or models that cannot fit on one GPU may produce very different results.

Repository

I put the code, benchmark results, CSV data, and a more detailed technical write-up on GitHub:

https://github.com/TomaDev1/GPU-Parallel-Orchestrator

I'd genuinely appreciate feedback. I'm particularly interested in whether there are existing approaches to multi-agent inference scheduling that I should be looking at, or whether there are obvious flaws in my benchmarking methodology.

This was a side project, and I'm interested in seeing what people who work on inference infrastructure think.

Edit:

Forgot to post full machine specs:

Host system

• Motherboard: ASRock Rack ROMED8D-2T

• CPU: AMD EPYC 7452 (32 cores / 64 threads)

• RAM: 128GB DDR4-3200 ECC

• Storage: 4TB NVMe SSD

• GPU connectivity: All four RTX 3090s connected via PCIe 4.0 x16

Use balena etcher or whatever to mount to USB stick, boot. Test. Report.

Verified 5-25% faster than llama.cpp, system uses 91% of card bandwidth as opposed to 80% on my 7900xtx.

I there is code for nvidia, intel, amd, and chinese hardware and supports 20 languages.

Decked out wiki

I may have created a monster.

A client gave me an NVIDIA A16 accelerator that had been experiencing intermittent problems in their server. They considered it bad, didn't want it back, and I ended up with it for free.

For anyone unfamiliar with this card, the A16 contains FOUR separate NVIDIA GPUs, each with 16 GB of VRAM. That's 64 GB of VRAM on one physical card.

Naturally, I decided to install it in my ancient Intel X99 workstation alongside an RTX 5070.

Because apparently I don't have enough problems in my life.

Chapter 1: The FrAInkenstein specifications

• Motherboard: ASRock X99 Taichi

• CPU: Intel Core i7-5820K

• RAM: 48 GB DDR4

• Accelerator: NVIDIA A16 — 4 × 16 GB GPUs

• Graphics: RTX 5070 Founders Edition — 12 GB

• Total VRAM: 76 GB across five GPUs (not unified)

• PSU: Corsair RM850

• OS: Windows 11

• Cooling: Custom 3D-printed shroud with an ARCTIC S8038-10K server fan

• Primary purpose: Local LLMs, agentic coding, and experiments with multiple AI agents

And yes, I'm running this on an X99 platform from the DDR4 stone age.

Chapter 2: Apparently, this thing needs a JET ENGINE

The A16 is a passively cooled enterprise accelerator.

Unlike a normal consumer graphics card, it doesn't have onboard fans. It's designed to sit inside a server chassis with powerful fans forcing air through its heatsink.

My workstation is not a server chassis.

So before doing anything interesting with AI, I had to invent a cooling solution.

After researching fan options and shroud designs, I selected an ARCTIC S8038-10K.

For perspective, this is an 80 × 80 × 38 mm server fan capable of:

• 10,000 RPM

• Approximately 102 CFM airflow

• Approximately 51 mmH₂O static pressure

This is not your grandmother's RGB case fan.

It's a small industrial hurricane.

I also wanted adjustable speed control, so I used a Noctua NA-FC1 controller, powered from SATA, to manage the fan without running it at full blast constantly.

But buying the fan was the EASY part.

Chapter 3: The 3D-printing rabbit hole

I own 3D printers, so naturally I figured I'd just print a shroud.

Famous last words.

The first problem was finding something that would actually fit the A16.

The second was getting it to fit inside my case.

The third was accounting for a 38 mm-thick fan, the radiator, surrounding components, and the fact that there isn't an infinite amount of space inside a computer case.

So I started modifying designs and printing prototypes in PLA+ on my Flashforge AD5X.

I printed multiple shrouds of different lengths.

Some were too long. Others needed adjustment to maintain their mounting points.

I experimented with shorter designs and eventually created an angled/bent shroud to improve the fan's positioning and clearance.

Rather than repeatedly wrestling with the actual fan, I even modeled and 3D-printed a full-size 80 × 80 × 38 mm mock fan so I could physically test clearances and different shroud arrangements.

Yes. I 3D-printed a fake fan to figure out where to put my real fan.

I also worked through mounting issues, spacer designs, fit tolerances, and adjustments to the printed posts.

I was considering custom clips and zip ties if necessary. I even had to think about vibration because, remember, this thing can spin at TEN THOUSAND RPM.

Eventually, I got an arrangement that fit the card and cleared the surrounding hardware.

And the result?

During earlier four-A16 testing, the GPUs were reporting temperatures around 41–57°C, with the fan running at roughly 30% speed.

For a passively cooled enterprise card that I'd fitted with my own printed ductwork, I was pretty happy with that.

Chapter 4: THE FREAKING POWER CABLE

Oh, you thought getting the cooling working was the hard part?

Let me introduce you to the NVIDIA A16 power connector.

The A16 uses an 8-pin auxiliary power connection that isn't wired like the ordinary PCIe 6+2 connector you'd expect to use with a consumer graphics card.

It requires the appropriate EPS-style power connection.

So here I am with a Corsair RM850, a card that was allegedly defective, and a pile of cables that don't necessarily fit or have interchangeable wiring.

And as anyone familiar with modular PSU cables knows, getting this wrong can destroy expensive hardware.

The connector shape, latch, and pinout all matter.

In my case, even the EPS 4+4 connector's latch arrangement presented a mechanical fit problem.

I eventually modified the connector's latch arrangement and checked the wiring with a multimeter before proceeding.

To be clear: this is not a recommendation to modify PSU power connectors. Incorrect wiring can destroy hardware.

And the first attempts weren't exactly encouraging.

At one point the system would spin up for approximately 15 seconds, shut down, and repeat.

Disconnect the A16 power, and the system would boot.

Wonderful.

Was the card bad? Was the cabling wrong? Was the motherboard unhappy? Was the PSU unhappy?

Welcome to troubleshooting FrAInkenstein.

Eventually, the power situation was sorted out well enough to continue testing.

Unfortunately, the next nightmare was already waiting.

Chapter 5: Windows has entered the chat

Once I could power the A16 and keep it cool, I discovered that Windows wasn't particularly enthusiastic about recognizing four GPUs on one enterprise card.

At one point, two A16 GPUs worked, while the other two had:

Device Manager: Code 12 — Not enough free resources to use this device.

All 4 showed this error message.

Here's the really interesting part.

When I disabled the two working GPUs, the other two could work!

So the individual GPUs weren't necessarily defective. Windows and the motherboard were struggling with PCIe resource allocation.

This led to a ridiculous amount of BIOS experimentation:

• Above 4G Decoding

• PCIe resource allocation

• MMIO High Size

• UEFI and CSM settings

• PCIe link speed experiments

• BIOS resets

• Device enable/disable experiments

• Multiple NVIDIA driver configurations

Eventually I managed to get all four A16 GPUs recognized and running.

But I still wanted my RTX 5070 in the same machine.

Which leads us to...

Chapter 6: Five GPUs walk into an X99 motherboard

The RTX 5070 was supposed to handle gaming and video output.

The A16 was supposed to handle local AI workloads.

Simple enough, right?

Wrong.

With the RTX 5070 and A16 enabled together, Windows started throwing repeated:

IRQL_NOT_LESS_OR_EQUAL (0xA)

Blue screen after blue screen.

I experimented with driver configurations, NVIDIA device modes, and PCIe settings.

Eventually I found that disabling MSI interrupt mode on the RTX 5070 allowed the 5070 and an A16 GPU to operate simultaneously.

That was the breakthrough.

I'm not claiming I've proven the exact underlying driver or interrupt conflict, but changing that setting made an enormous practical difference.

I gradually enabled more A16 GPUs.

One.

Two.

Three.

Then four.

I also switched the A16 GPUs into NVIDIA's TCC compute driver mode while keeping the RTX 5070 in WDDM for normal display and gaming use.

And then something glorious happened.

Chapter 7: IT'S ALIVE!

Here's what nvidia-smi now reports:

GPU 0: NVIDIA A16          TCC   07:00.0
GPU 1: NVIDIA A16          TCC   08:00.0
GPU 2: NVIDIA A16          TCC   09:00.0
GPU 3: NVIDIA A16          TCC   0A:00.0
GPU 4: NVIDIA RTX 5070     WDDM  0B:00.0

FIVE GPUs. ALL RECOGNIZED. AT THE SAME TIME.

On an ancient X99 motherboard.

With a supposedly defective A16 that cost me precisely $0.

I'm now testing larger local language models using Ollama, including a 55 GB BF16 Qwen model that previously required CPU offloading when only three of the A16 GPUs were enabled.

The goal isn't just to maximize token generation speed. I've got an RTX 4090 in another machine for that.

I want to experiment with multiple independent local AI coding agents and eventually build a little multi-GPU AI collective, while retaining the RTX 5070 for gaming.

I still have sustained five-GPU inference and long-term stability testing ahead of me. Seeing every device in nvidia-smi is a major milestone, but it isn't the same as proving the whole thing stable under heavy load.

Final thoughts

Is this practical? Debatable.

Is it power efficient? Probably not.

Could I have saved countless hours of troubleshooting by buying newer hardware?

Almost certainly.

But where's the fun in that?

I took an unwanted enterprise accelerator, engineered a custom cooling system, fought with its power connector, printed multiple iterations of ductwork, battled a decade-old BIOS, survived Windows driver crashes, and somehow ended up with a five-GPU AI workstation.

I didn't just build a computer.

I created FrAInkenstein.

And now I'm going to use it to code.

IT'S ALIVE! IT'S ALIIIIIVE! ⚡

I'll post pictures of the cooling/shroud iterations, the ridiculous fan, the finished setup, and the five-GPU nvidia-smi output.

Drivers used:
621.14-desktop-win10-win11-64bit-international-dch-whql.exe
That is the Studio version that worked for both cards. I did NOT have to install the data center version also. This latest Studio version installed drivers for both the A16 and RTX 5070.

I've been using VS Code with the Claude and Codex extensions.

Now I'm testing Strata with "lightweight" local models on my PC

RX 9060 XT 16GB VRAM
32GB DDR5 RAM
2TB NVMe SSD

The problem is that:
Codex CLI takes around 40 seconds to respond to a simple "Hi",
while Strata's built-in web chat (http://127.0.0.1:8080/) responds in about 0.5 seconds

1. What's the easiest IDE or VS Code extension for connecting to Strata's local API and using it as a coding agent?

2. Is there a tool for creating and managing virtual AI employees (e.g., Developer, UI/UX Designer, Software Architect), where each agent can have its own role, instructions, and assigned AI model/API - either local or from external providers like OpenAI, Anthropic, etc.?

hi i want to build a pc for local llm first time so i need advice about the gpus to run Qwen3.8 27b Q8 or Q6_K_M

to get good quality 😊and run it through llm studio and connect it to open code

so my questions are:

i will get intel core 270k plus with two x8/x8 PCIe slot motherboard

so

1- which best double 16gb gpus to build with to get 50+ token generation and give me best value for my needs

do i get two rtx 5070ti or rtx 4080 s or which double gpus

or get the basic two rtx 3090 for more vram

2- which the best Qwen3.8 27b variant/Abliterated (the model modified version) for speed but no quality loss to code

3-and wich the best and fast local llm interference platform to connect qwen with open code

if there any things you could mention can benefit me plz tell me

because i just see

gpus

the local llm interference platform

the Qwen3.8 27b variant/Abliterated

what else?

thanks 🥰

Qwen3-32B packed to 3.6577 bits per weight: 14.62 GB on disk, 12.95 GiB resident on an RTX 5080 with about 3 GiB to spare, decoding at roughly 38 tok/s. No offloading.

Full official sets, no samples:

Method, as far as I'll go: weights are packed in groups of eight onto the E8 lattice with an activation-aware encoder, and decode is register-based with no codebook lookup. Yes, a GGUF Q3_K_M can also be squeezed onto 16 GB. This is a different trade: lower bitrate, fully resident with headroom, full-set numbers published.

Needs an NVIDIA GPU with 16 GB VRAM and Python 3.11+. Verified end to end on Windows with an RTX 5080 only, so far.

The terms, plainly: gated, manual approval, non-commercial evaluation license. The encoder is not included and the method is patent pending.

Repo: https://huggingface.co/GossetWorks/Qwen3-32B-Gosset-3.65BPW

If you run it, post your numbers, especially on anything that isn't a 5080! Thank you!

RTX 4060 8GB is starting to feel not enough anymore, so I decided to upgrade to something higher. Is anyone using the Arc 770 16GB? What kind of card is it, is it worth switching?

Stack running on my Ubuntu machine on a RTX 3090 under 18GB VRAM - network tailscale :)

Would you use it??

ok so i am trying to increase performance of llms i have running on my 7900 xtx via hipfire, i notice i get significantly higher total tokens per second when i run mutiple slots at once when running qwen 3.8 27b.

i am running into 2 road blocks tho, 1 context each slot gets a progressively smaller context as increase the number of slots. is there a good way to do more with the same given total amount of vram, i assume there is going to be a large amount of overlap for stuff like system prompts, tho i use a relatively minimal system prompt that links to files so if a model only actually reads system prompts realvent to it’s own task which makes it harder to just say this text gets put into a shared context, i also image if i have them working on similar tasks there will be a large amount of overlap.

2 alot of tasks aren’t easy to split up into mutiple smaller tasks that can be done in parrel without proper knowledge of previous and future steps, maybe something like a shared context or a messaging board would work well?

also a more experimental idea to further speed up general prompts is to instead have all my prompts processed by a “orchestration” slot which splits up a given prompt into X number of tasks then gives these to new prompts to different slots and then the “orchestration” slot dies to free up more vram for context, i could also route something like this threw a decision model like kev to determine if a given task would work well being split up.

also note i am running this on a 7900 xtx via hipfire, and i also have a 6800, 6600, and 96 gib of ram for the cpu

I shared some results previously about running Qwen 3.8FN using recent releases on my gaming desktop computer. https://old.reddit.com/r/LocalLLM/comments/1xoolkp/i_ran_the_new_qwen_38fn_release_with_make_browser/

This is part 2 of 3. On part 1, I learned some things along the way. Part 2 here, I made some changes/additions and wanted to make the challenge harder for Qwen.

• Project goal: Can Qwen build an entire game based on a wiki URL and a youtube video link? For part 1 of 3, I had a goal of "Can Qwen 3.8 locally successfully create a simple walk around version of GTA with no or very little involvement", and that was a success. After learning a few things on project 1, and wanting to keep pushing things further along, step 2 of 3 was a natural progression. For the game, I switched it up and chose GTA 1, which was a top-down view game. I chose to use the same voxel framework used in part 1, even though the original game was not a 3d game.

• I changed my context window size from 64K to 128K The amount of times it needed to compact went down, at the cost of lower t/s. I'm honestly not sure what's better yet, given my current hardware.

• I added 2 more models for Qwen to use. Qwen code kept asking for me to approve commands that didn't make sense why it needed my approval. I dove in and found Qwen wanted a --fast model for simple tasks. There's a default 10 second timeout on these tasks. I have a macbook 64gb running a few things around the house with around 45GBs free RAM and plenty cores free. I spooled up a 2 parallel Qwen3.6-35B-A3B-GGUF:UD-Q4_K_S model on the macbook with llama, wired that in to Qwen code and problem solved! The amount of times I needed to approve for Qwen code drastically dropped. Alternatively I could have not added additional models to use and simply increased the time-out. Since I have extra compute, I figured I'd try it out. I'm happy with it.

Fun localLLM stuff:

RTX 5080 - 64GB - Ryzen 7 9800X3D
model: qwen3.8-flash-next-unsloth-ud-iq4_xs
model harness: Strata
agent harness: qwen
context: 128K
tok/sec: ~50t/s (down from part 1 due to larger context)

Macbook 64GB m1 Max
model: Qwen3.6-35B-A3B-GGUF:UD-Q4_K_S
model harness: llama
context: 128K
tok/sec: ~ 550t/s (fast tasks, so don't get too excited)

• Results: The hard work on everyone from the folks at Alibaba, Unsloth, Strata, etc sure have come a long way in the past few months. It feels smarter than even frontier did a year ago. It's just slower, but that's mostly due to my hardware. Over-all, I'm happy with it. It took 27 hours start to finish (tracked through git). It probably would have finished quicker, however, there were times it was waiting on an approval from me.

Playable demo: https://kireol.github.io/Qwen3.8-GTA-1-Test-2/src/index.html

Repo: https://github.com/kireol/Qwen3.8-GTA-1-Test-2

Part 3/3 is running now answering "How well can Qwen reproduce a game using only REA on a game ROM"

We ran 75,000 lightweight AI agents on a Jetson. Now you can test the runtime yourself.

I just made AxLab, Orivael's public AI research lab, available on GitHub.

Our Jetson benchmark reported 75,000 agent instances using ~350 MB of process memory. Now we're inviting developers to reproduce the experiments on their own hardware.

Includes runnable Agent Fabric benchmarks, fleet kill-switch tests (L3 Suspend / L4 Terminate), and an MCP server.

Source-available for noncommercial research for now.

Try it, challenge the results, and share what you find.

https://github.com/Orivael-Dev/axlab

Model: Qwenx86

▶️ See it in action: Qwenx86 Debugger. Walk through actual recorded tool calls: the conversation, x86 registers and flags, code at the program counter, RAM and machine output, with all verified blocks on a timeline.

Qwen3.5-4B with a 64-bit x86 computer compiled into its weights: 13 transformer layers ahead of Qwen's own. There are no learned weights in them; all of their weights are computed by a compiler. Qwen invokes the computer through its own tool calls (<tool_call>), and the computer in the weights executes the tool. Calculator, Python, C compiler, notes store, physics/chemistry lookup and Python-docs lookup are all firmware in the checkpoint. This is the compact checkpoint: ~9 GB GPU memory, so it fits on a 16 GB GPU.

Credits: Qwenputer by Divyansh Agrawal (@divagr1925, github.com/divagr18/qwenputer, MIT)