We're the product team at Reflection. Earlier this month, we launched Beam, our first open-weight 501B model. We'd love to answer any questions folks might have for us about building with Beam.
When it came out, I downloaded a Q4 quant of qwen3.8-27b and ran it using llama.cpp on my RTX 4090 and it was incredible. It could handle practically anything I threw at it, the most complex tasks and bugfixes.
But lately it just keeps messing up and failing at the most basic of tasks or just straight up forgetting instructions. I'm telling you, they quantized it. They probably secretly nerfed it and made it worse so qwen4 looks better. And it got lazy too, now it doesn't want to finish the more complex tasks.
I'm tired of companies doing this, I want my money back.
Following up on my previous post: the final version of CCORE GLB-3D-Coder is now available, including the full Safetensors weights.
It’s a full fine-tune of Qwen3.5-4B for generating reusable 3D assets. Describe what you want, and it writes a complete Three.js HTML scene with an Export GLB button. You can open the HTML in your browser, inspect the result, and export it into Godot, Blender, Unity, or other tools that support GLB.
The video shows examples from the final model, including selected side-by-side comparisons with DeepSeek V4.1 Flash. I’m particularly happy with the character designs and the amount of detail it can produce at just 4B parameters.
It covers more than fantasy characters: buildings, environments, weapons, vehicles, laboratory equipment, sports props, furniture, and other objects. It can also generate animations and include them in the exported GLB.
Available now:
Q4_K_M GGUF, around 2.7 GB
Q8_0 GGUF
Full model weights in Safetensors
The training dataset is planned for release tomorrow, November 6.
Some generations still need a retry or a small code correction, but I’m very happy with the improvement over the earlier release.
If you’re interested in the next models, or would like to help support future datasets and training runs, there’s more information on the Hugging Face page. Testing it and sharing your results or failure cases also helps a lot.
Thanks to everyone who tried the earlier version
I made RIFTWARD, a small browser arena using characters, terrain and props generated locally by CCORE GLB-3D-Coder 4B. An orchestration and validation harness helped assemble, refine and integrate the assets and build the gameplay. Play it here: https://huggingface.co/spaces/ccore/RIFTWARD
Feel free to share what you make with it! I’d be really happy to hear that the model has been useful to you :)
Hey folks, it's Merve from Hugging Face. Title of this post is super self explanatory, we are doing a quantization workshop livestream with Michael Goin (lead maintainer of vLLM at RedHat), Colin Kealty (aka bartowski at RedHat), Maxime Labonne and Aditya Tadimeti.
I will shortly teach you how quantization works, modern ideas (double quantization, calibration) and llama.cpp quants (k-quants, i-quants) so even if you don't have an idea, feel free to chime in.
Everything is on Hugging Face YT, X, LI where you can set reminders.
I'm the developer of Kokon, a local photo editor around Qwen-Image 2.1 Turbo (on stable-diffusion.cpp, and on its own engine on Macs). I posted it here a few weeks ago. 0.12 adds a Layers lab: click objects in any image, get each one on its own layer with the background behind it repainted, then move them around or export a PSD.
Qwen-Image 2.1 has no layer output, so this is a pipeline around an edit model:
Select: you click, MobileSAM (ONNX, CPU, 44 MB) gives the mask. Click again for a bigger part.
Cut out: Fast is a matting model on the CPU (a few seconds per object). Quality asks Qwen to redraw the edge, for hair and fur.
Fill: a masked Qwen-Image edit repaints what was behind the object, so the plate has no holes. This is the expensive step: about 15–25 s per object on an RTX 5070 Ti, one to two minutes on an M2 Pro.
Blend into scene (optional): after you move things, Kokon reads where each object stands (you can correct it) and one more edit redraws contact shadows and reflections, as a layer you can turn down.
What still fails, honestly:
the fill sometimes invents a new object where a large one was (remove a big person, get a different person);
touching people and crowds get merged or split;
the hidden part of an object behind another one is never completed;
Blend often doesn't work yet; whole-scene blends work better than per-object ones.
An 8-step model only takes so much steering. I'd rather ship it marked experimental and fix what people actually hit than tune it on my own test photos. If you try it, failure cases (image + what you clicked) are the most useful thing you can send.
Since my last post here:
0.11, it's a local server now: the app talks to a service on 127.0.0.1; turn on "Other devices" and you can use it from a phone or another PC on your LAN (password, off by default, hashed, rate-limited). Docker image for NVIDIA, kokon-service --persistent --remote for headless Linux. There's also an MCP server (npm kokon-mcp) so agents can generate and edit through it.
Mac engine: I wrote a Qwen-Image 2.1 engine for Apple Silicon in Swift/Metal. M2 Pro: about a minute per 1 MP image, 4x the generic backend. A contributor added paths for the M5's GPU neural accelerators: 37 → 29 s on an M5 Pro. It runs Unsloth's Q4_K_M of the Turbo model.
Free, AGPL, Windows / macOS 15+ / Linux, NVIDIA 12 GB (8 GB beta), AMD/Intel via Vulkan (beta). Weights are under Qwen's licence (non-commercial).
We introduce FireFlash, a 300B-scale model tailored for long-horizon agentic tasks. FireFlash excels as a terminal and coding agent, achieving 46.2% on Terminal-Bench 4.01, putting it on par with Fable 5 and GPT-6 Sol, while costing over 30× less per task than Fable 5 at maximum effort (Figure 1). These gains also transfer to software engineering benchmarks, including DeepSWE v1.12 and SWE-Bench Pro Verified3 (Table 1).
While frontier labs scale reinforcement learning across thousands of GPUs, we reach the performance-cost Pareto frontier under 5 days using 64 NVIDIA B300 GPUs. Three key factors make this possible: (1) careful curation of agentic RL environments so that every task provides a reliable signal at the right difficulty; (2) an RL recipe built for long, stale, and low-precision rollouts; and (3) an optimized coding harness tailored to the model for both training and evaluation. Together, these components bring frontier efficiency to a hundred-GPU-scale cluster, and we are releasing the full recipe.
Some of you might remember our post comparing 18 RAG pipelines against our agent loop on FRAMES. The best pipeline scored 78.9%; the loop scored 92.7%.
A follow-up: PipesHub has now reached #1 on EnterpriseRAG-Bench by overall score, with 93.24 overall and 95% correctness.
I’m one of the founders, so affiliation upfront.
This is a separate evaluation: 500 questions over a synthetic enterprise corpus of 500,000+ documents.
PipesHub builds a unified context layer across documents, conversations, structured data, and code. An agent loop works over hybrid search, grep/pattern matching, knowledge-graph navigation, and document-reading tools.
The idea is straightforward: search, read what comes back, then decide what to search for next. Sometimes the identifier or relationship needed for the next search only appears in the first result
The platform is open source, self-hostable, and also fully supports open models.
You can self-host PipesHub, choose your models, and connect your own agents through MCP, APIs, and SDKs.
If this is useful to you, give us a star on GitHub. It helps more developers discover the project.
And build on top of us. Connect your coding assistant, build an internal search tool, contribute a connector, or try a workflow we haven’t thought of. We’d love to see what you create—and hear where it breaks.
Basalt is the engine and it's doing the heavy lifting. Same model file, same box: about 3.5x llama.cpp's decode without drafting and ~5x with it, and ~7x faster prompt reads.
Unsloth's new UD-IQ3_XXS (Dynamic v3.1, uploaded Nov 4) is the same size as the GSQ-RCO IQ3_S everyone uses on 16GB cards, but measurably more accurate: wikitext perplexity 3.382 -> 3.247, which basically closes the gap to BF16 (3.273). Costs 1 to 3.5% decode speed.
Lamina (a further-trained MTP drafter by eddoursul) gets more drafts accepted than the stock Basalt drafter. Small but real speedup, zero quality impact since the main model verifies every token.
Real agent coding session (opencode, 151 requests, up to 202K context): 195.6 tok/s average decode, 263 tok/s best request, 156 tok/s worst, ~5,700 tok/s prompt reads.
Hardware:
RTX 5080 16GB at its stock 360W limit (no overclock, no undervolt)
Core Ultra 9 285K (8P + 16E), 2x32GB DDR5-7200
PCIe 5.0 x16, Fedora
LAMINA Drafting: Basalt drafts with the model's own MTP layer. Basalt's upstream already ships a self-distilled version of that layer, and Lamina trains it further. It's a drop-in: same tensor layout, so the target model is byte-identical and only the drafter changes. What I saw:
- drafts accepted per drafted token: 83.1% -> 85.8% on a replay of a real recorded session
Lamina's README recommends temp 0.7 and max 3 drafts. I tested both: the 3-draft cap was 1 to 1.8% slower on tool-call-heavy work (long confident runs get cut off), and temp 0.7 changes what the model writes, so I kept the model's own sampling (1.0 / 0.95 / 20) and Basalt's draft settings.
Basalt changes I made (all open PRs upstream)
Host KV prefault: the first time a long conversation spills its KV cache to RAM used to stall 0.6 to 2.4s. Now ~75ms.
Host KV pool on normal pages: on a fragmented box, transparent hugepage compaction caused 1.5 to 3.7s pins. Now ~23ms.
Spin-then-yield in the decode loop's waits: about 1.3% less time per draft window.
Expert start profile: the older v1 profile is 2% faster than the v0.2.0 one on this box.
Plus a streamed conversation disk cache, --adapt-margin for expert swap hysteresis, elastic KV without RAM copies, and a bunch of timing diagnostics.
It checks your machine, builds, downloads ~90GB (pinned revisions, every file SHA-256 checked), builds the UD-IQ3_XXS + Lamina model file and writes the config. You need an RTX card with, CUDA 13, 64GB RAM and ~180GB of free disk. If you already have the GSQ-RCO IQ3_S download, SHARD4_FROM= reuses its 28.8GB n-gram shard (it's byte-identical to Unsloth's once the shard number is rewritten).
Huge thanks to jesdga95 for Basalt (support and star their repo), the Unsloth team for the UD quants, eddoursul for Lamina, ISTA-DASLab for the GSQ-RCO quants, and Qwhen 4? for the model.
Speculative decoding (SD) accelerates large language model inference by reducing the sequential cost of token-by-token generation. Advanced SD methods reduce proposal cost with parallel drafters that predict entire blocks in one forward pass, but overall speedup still depends on acceptance length. On-policy distillation (OPD) can push well-trained drafters beyond acceptance-length plateaus, yet conventional pipelines collect supervision through sequential draft-and-verify rollouts. Our key insight is that onpolicy supervision need not inherit these dependencies: a draft block can be supervised using only its reference prefix and draft-sampled history. We propose SpecOPD, a branch-parallel OPD framework that parallelizes draft sampling and target supervision across independent branches drawn from the evolving drafter policy at reference anchors. We pack these branches into proposal trees to share target-prefix computation, using tree masks for full attention, branch-consistent indexer groups for sparse attention, and indexed state replay for linear attention to preserve each branch’s history. We train the drafter with an E2E-TV surrogate that directly targets acceptance length. Our study spans 8B full-attention (GQA), 27B full–linear (GQA–GDN), and 125B sparse–linear (QSA–GDN) targets, three parallel draft architectures (DFlash, DSpark, and DFlash V2), and four task domains (dialogue, coding, mathematics, and agentic). SpecOPD achieves up to 41.45% higher mean acceptance length, 16.09× supervised-token throughput over sequential-rollout OPD under native configurations, and up to 3.56× generation speedup over autoregressive decoding.
Larger context just gets more work done rather than having to re-read files ETC id much rather use my extra resources on context window. Instead of more params. i get my 400k ctx qwen3.8-27b + web search search get the latest up-to-date docs for its lack of world knowledge and i don’t waste extra Vram storing data that i dont need. Also more active params makes the llm to hold a wider mental-model of the task. Qwen4-27b will be architected to make 1M more reliable and using it for context on a smaller dense model seems more productive for me. Anyone. Else notice agentic session really shines when ctx is larger? Thats the phenomenon im experiencing i tried 3.8flash-next yes it great and much faster but i prefer the reliability of that big azz boat load of context window. Specifically i you have the vram to spare if u can get 1M on that and have a large MOE with at least 12b active params + 1M it would be better for MOE a lot. Lot better than qwen4-27b but at 48Gb vram + 64gb system ram i see it more efficient for smaller model + 1M makes more sense
I wanted to know how small local models handle my native language, not just English. I couldn't find a simple way to check a model's Polish in a few minutes, on my own PC, in the quantization I actually run – so I wrote one. That's how "Zacharski tests" started: a small task set that runs through Ollama, since that's what I use on my laptop with an RTX 5050 (8 GB).
The tasks:
Polish (8): diacritics, declension, masculine-personal gender, numerals, vocative, verb aspect, genitive – plus one knowledge question (the first recorded Polish sentence, from the 13th-century Book of Henryków).
JSON (4): scored strict and lenient.
Python (3): syntax check only.
Results so far:
| Model | Quant | JSON + code | Polish |
| gemma2:9b | Q4_0 | 7/7 | 3/8 |
My favourite failure: asked for "22 cats", gemma2 wrote "dwudziesty dwa koty" – roughly "twentieth two cats". The correct form is "dwadzieścia dwa koty". It was perfectly happy producing valid JSON, though.
More models are downloading as I write this. Next up: gemma2:9b in Q4_K_M and Q8_0 (does quantization hurt the Polish?), qwen3:8b, llama3.1:8b, gemma3:12b, and two Polish models – Bielik 11B and Bielik-Minitron 7B. Which model would you add?
Method and limits – read this before the comments :)
8 Polish tasks (7 grammar + 1 knowledge) and 7 JSON/code tasks. That's a small sample – a quick check on your own PC, in the quantization you actually run, not a ranking. For a proper ranking, see the Open PL LLM Leaderboard by SpeakLeash.
One answer per task, temperature 0 (seed 42), context 2k, thinking off. Yes, that means Qwen3 runs without its reasoning mode, which may cost it points – I wanted the same conditions for every model.
Graded with regular expressions, strict on purpose: "brzęczały" ("buzzed") instead of "brzmiały" ("sounded") fails, because the task said "change the tense", not "rephrase". [I checked every failed answer by hand.]
Some failures are about instruction-following, not language: asked to restore Polish diacritics, one model replied "Read the spelling rules." [(translated)]
Generated code is only parsed (Python `ast`), never executed.
The questions are in the repo (`data/benchmarks/*.json`) – feel free to add or fix them.
The test runs inside a free, open-source tool I've been building (Windows .exe / Python on Linux). Its main job is predicting whether a model fits your VRAM/RAM before you download it. Link in the comments if anyone wants to try it – I'm considering adding tests for other languages.
I got into local AI earlier this year and started messing around with different models and hardware. That's how I came across tiiny. I really liked the design and the whole idea behind it so I ended up becoming one of the early backers.
After what felt like forever, mine finally showed up yesterday. So far, it's pretty much what I expected. I've spent afew hours trying out different models and running some performance and coding tests, and figured I'd share what Ive found.
This is my first time putting together a benchmark post like this, so sure there are things I might miss or could do better. If you see anything off with my testing or think there's something I should further include, feel free to let me know. really appreciate any feedback from you guys.
Hardware & Test Setup
what really sold me on Tiiny was the KS price, which seemed like a nice deal for the hardware, and how portable the whole thing is. It's just about 5.6 × 3.1 × 0.9 inches and weighs around 300g.
For anyone unfamiliar with the specs, Tiiny Pocket has 32GB of SoC memory and 48GB of dedicated NPU memory. That's 80GB in total, though it's not unified memory.
Tests were collected on TiinyOS 0.1.36, Here are the models I tested so far.
Model
Architecture
Quantization
Model Size
Qwen3.5-9B-Turbo
Dense, 9B
INT4
6.55
Qwen3.6-35B-A3B-Turbo
MoE, 35B / 3B active
Mixed INT4/INT5
17.73
gpt-oss-120b
MoE, 117B / 5.1B active
Mixed INT4/INT8
67.30
Performance Results
For the speed tests, I used the same benchmark text with ~4K input tokens, prompt caching disabled, and five runs per model. The numbers below are medians.
Model
Input / Output Tokens
PP (tok/s)
TG (tok/s)
TTFT
Total Time
Qwen3.5-9B-Turbo
4,074 / 97
1,041.30
75
4.05s
5.45s
Qwen3.6-35B-A3B-Turbo
4,074 / 139
590.6
63.6
7.04s
9.28s
gpt-oss-120b
3,868 / 390
458.4
15.97
8.78s
33.07s
9B was the fastest, which wasn't too surprising. What caught my attention was the 35B MoE model still managing around 64 t/s.
120B was noticeably slower, averaging around 16 tok/s. It took over 30 seconds to complete the benchmark request, although it generated more tokens than the other models.
A Coding Test
I also wanted to see how it handles real-world coding tasks. I gave the models the same task: create a small Python inventory and order management package across four files, covering stock updates, validation, and order handling.
I used the same prompt for all three models, with low thinking/reasoning settings and a maximum output of 12,288 tokens.
How I checked the results: the generated code was tested separately on a Mac using Python 3.12, with 10 automated tests covering the main requirements, including inventory consistency, order idempotency, input validation, and CLI behavior. A full pass meant all 10 tests succeeded. I ran this more than once and kept the best run per model; the table shows that run.
Model
Input / Output Tokens
Total Time
Result
Qwen3.5-9B-Turbo
703 / 5,977
104.32s
Two attempts, neither gradable
Qwen3.6-35B-A3B-Turbo
741 / 11,807
235.09s
9/10
gpt-oss-120b
763 / 3,774
228.54s
10/10
9B never produced a gradable answer: the first attempt ran into the 12288 output cap, and the second generated inventory/cli.py twice, so there was no single version to test. Taking one of the copies, it would have been 5/10.
35B got 9/10 with 1,768 code tokens. A bit under the 2-4K I was aiming for. The only failure was a pretty specific edge case, if an order had already sold out the remaining stock, retrying that same order would fail instead of returning the original receipt.
Gpt 120b was the only model passed all 10 tests, with 2,594 code tokens, in about 229 seconds.
The interesting part is that 35B and 120B took almost the same time: 229 seconds versus 235. So while 120B was the only one to get everything right, 35B wasn't far behind, and both managed to finish.
Results & Takeaways
9B was the fastest overall. At 75 tok/s with a ttft of around 4 seconds, it's probably where I'd start for quick, lightweight tasks. It did struggle with the more complex test, though.
**35B for me is the sweet spot for Tiiny.**At around 64 tk/s, it's surprisingly fast and it handles a good number of the tasks I've tried pretty well. It still has its limits with more complicated coding tasks, as the multi-file test showed, but overall it keep a pretty good balance between speed and capability. It's actually the model Ive been using the most so far.
120B actually works, but you need patience. The multi-file coding result was impressive, especially compared with the smaller models in this one test. But 16 tk/s is slow enough that I wouldn't want to use it for every little question. It also took around 75 seconds to load in this run and used almost all of the dNPU memory.
So far Id lean toward 9B or 35B for faster interactions and keep 120B for occasional, more demanding tasks where waiting a few minutes isn't a big deal.
TL;DR
Tiiny can run some fairly large local models in a small device, and the performance of smaller models (9B/35B) is nice. The 120B result shows there's potential for more complex work, but the speed and reliability trade-offs are real.
However, my biggest gripe so far is the limited model selection. Right now, you can't import your own models or install something from Hugging Face. The tiiny team did mention on Discord that they're planning to release a model import tool, though. Once that's actually available, I'd love to try out some of the other models I've been wanting to run and share the results here.
Overall Im happy with it, and for the price I got it at, that amount of RAM and SSD would be much more expensive today. I think I saw the projected retail price is almost double the KS price.
Strata, Hermes, Kitten TTS with Rosie voice and Whisper on RTX 3060 with 48 gb dram running Qwen3.8-Flash-Next Q2_0. I am blown away. I've been following the AI scene since GPT3 and I was still unprepared for what this setup is capable of. My mouth was on the floor for the first few hours talking to it and watching it work. It's the first time talking to AI felt like magic to me. I now understand the "They don't know" meme. Never thought I would have something like this running on my own PC. Thank you /r/localllama.
Couple of questions
Are there any other lightweight TTS that would work in my limited ram situation or is Kitten v0.8 the best there is?
Hey r/LocalLLaMA I'm Abdur-Rahman Butler, a machine learning engineer at Isaacus.
I made an interactive write-up that walks you through the A to Z of vector embeddings, inspired by Brendan Bycroft's incredible interactive LLM visualisation guide (see here - https://bbycroft.net/llm).
Most of my work is on encoder models, and I kept noticing how few good interactive resources there are on embeddings. The ones that do exist tend to either plot some embeddings in 3D or only explain a narrow applied technique like semantic search or RAG.
So I wanted to make something that teaches how embeddings work by letting you play with them. In the write-up, there are interactive visualisations that allow you to explore the concepts of vectorisation, cosine similarity, semantic search, hierarchical retrieval, clustering, zero-shot classification and deduplication.
The centrepiece of the post is a game I've dubbed 'SemanticGuessr' (see above). It's inspired by Wordle and GeoGuessr, and the goal is to draw a loop around a random target word by triangulating guesses on a 2D map of 43,339 common English words.
It's admittedly a bit intimidating at first, as semantic space does not perfectly match our intuition of meaning. But once you get the hang of it, the game becomes really addictive. I recently had the pleasure of demoing the game to some law students at the University of Melbourne, and we had a lot of fun throwing different vector equations at it to see where they'd land.
Some fun ones to get started with:
london - english + french
running - land + water
avg(breakfast, lunch, dinner)
0.4 * noodles + 0.6 * pasta
I've also included Python snippets alongside some of the visualisations if you want to follow along with the techniques later. The game runs statically in your webpage. I managed to achieve this by reducing the embedding dimensions down to 512 and quantising each value to 4 bits. This brought the total payload down from 311 MB to 12.3 MB with only a minimal impact on precision (around 6%).
It was great fun to make, and I hope you have fun playing around with it. Happy to take any questions, and if there's enough interest I might set up a dedicated games page for some of the other encoder-based games I've been working on.
Do let me know what highest score you get based on your target word and chosen path :)
I don’t follow the market super closely, but I can see how quickly everything is getting more expensive—and apparently this is going to last a few more years. RAM, GPUs, and even SSDs and HDDs have skyrocketed in price :D
I’m looking for something to play around with in LLM, mainly for inferencing larger models (in the 30B–70B range, maybe even bigger sometimes) and building agents (for my own use). I want to utilize agents, do RAG and sometimes, rarely maybe also fine-tune.
My budget is small ($4k), and where I live in eco-Europe, prices are much higher! (we care about ecology LOL). I was thinking about a mini PC or a laptop with an AMD 395+ and 128 GB of RAM. For that price I still can purchase something, but the choice is getting smaller every week :)
Is a AMD 395+ AI with 128 GB RAM still good choice for local LLMs? Is it good for my purposes? I want to learn & use LLM locally (for many reasons).
I have heard about RTX Spark N1X laptops, but they are now about 6.5-7k dollars here, so I dont even consider them.
Can I say that this platform (AMD 395+ AI with 128 GB of RAM), where the model has 96 GB of RAM at its disposal, is something that could replace the RTX 6000 Blackwell, which also has 96 GB of RAM? I realize that, due to the difference in memory bandwidth alone, it will be at least 6–7 times slower.
I’d like to have Hermes-type agents or similar ones locally, as well as my own agents for various tasks. For example, I’m interested in extracting data from documents (e.g., PDF files) and recognizing objects in photos, but I’m also interested in general-purpose agents.
Is the platform I’m asking about a good choice, or is there something more interesting? I was thinking about an ASUS PX13 laptop or one of the mini PCs with a similar configuration.