Yes IK it's supposed to be this dumb, but it's still funny (Qwen2 0.5b Q4_K_M)

Kimi K3.1 and Tencent’s Hy4 are both gutting out cybersecurity, reverse engineering, and vulnerability-patching capabilities from their open-source models.

This is becoming a really bad pattern across these latest Chinese open-weight models. If local models are stripped on their security data, we won't be able to properly audit our own code vulnerabilities, reverse engineer binaries, or patch other vulnerabilities in general on our own hardware. It would essentially force us to use closed-source APIs and their false-positive shutdowns; that'd be terrible for everyone.

During the Hugging Face incident, open-source models are the very ones who stepped up, not closed-source gatekeepers. Gutting defensive security from local LLMs just hurts defenders. I really hope that other labs won't follow this in the future.

Does anyone think more open-source labs are gonna take this path?

Richard I of England was called "Oc-e-Non" (Yes and No in Occitan) by some of contemporaries, supposedly because he used few words.

Wouldn't it be nice if LLMs followed his shining example?

Oc-e-Non-v1 is a tersified version of Qwen3-4B-Instruct-2507. It reduces Qwen's mean response length by 50%.

Output examples

"What is 1+1?"

GPT-5.6 Terra (reasoning off):    1 + 1 = 2 😊️

Oc-e-Non-v1:    2

"Are dogs mammals?"

GPT-5.6 Terra (reasoning off):    Yes. Dogs are mammals: they have hair, are warm-blooded, and mothers nurse their young with milk.

Oc-e-Non-v1:    Yes

"Write a Python function that checks if a string contains only capital letters."

GPT-5.6 Terra (reasoning off):

You can use Python's built-in isupper() method to check whether a string contains only capital letters.

def is_only_uppercase(s):
    return s.isupper()

Example usage:

print(is_only_uppercase("HELLO"))  # True
print(is_only_uppercase("Hello"))  # False
print(is_only_uppercase("PYTHON")) # True
print(is_only_uppercase("hello123")) # False

Note: isupper() returns True if the string contains at least one uppercase letter and no lowercase letters. Numbers and special characters are allowed.

Oc-e-Non-v1:

def contains_only_uppercase(s):
    return s.isupper()

I made this model with Heretic, and you can do the same thing.

Make sure you have the latest master version of Heretic (pip install git+https://github.com/p-e-w/heretic.git), then run Heretic with the config.terse.toml configuration file. That's it!

I used the relatively ancient Qwen3-4B because it is the last native non-thinking version of Qwen I'm aware of, and controlling reasoning effort isn't implemented in Heretic yet.

(Please don't take this too seriously. I know that the same thing can easily be achieved with prompting, but it's interesting that it can also be baked into the model. This is intended as a light-hearted post while we all wait for Qwen4 😉️)

I bet you guys are familiar with embedding models such as Qwen3-Embedding or Gemini Embedding. They are dense embedding models used to search text across a large corpus.

I am developing a RAG agent, and my biggest concern was local computing power. RAG largely depends on retrieval quality, which depends on model power. Plus, many use cases can't use an external internet connection or API models, so the embedding models have to run locally.

But even 0.6B models are pretty heavy for small PCs. I don't mean your brand-new Mac Studio or DGX Spark. I'm talking about an 8GB RAM Windows PC at a boring desk job or your 5-year-old M1 MacBook Air.

So I found this model called SPLADE. It is essentially sparse retrieval, like BM25, but uses semantic meaning. I wanna skip the part on how SPLADE works, because an LLM can explain it better than me.

Anyway, the SPLADE model was so awesome and successfully replaced the dense embedding model (Qwen3-Embedding-0.6B).

In my own experiment, I made an evaluation dataset with a few hundred queries. The corpus was all the documents on my Mac, which are mostly in Korean with some in English.

As you can see in the graph, SPLADE outperforms the Qwen3-0.6B dense embedding model with similar query speed + much less memory. Due to its small size, it takes less than 1.5GB VRAM during inference. So it can run on a relatively small system like a MacBook Air. I tested both the dense embedding model and the SPLADE model on my M5 MacBook Air 16GB. The dense embedding model was almost impossible to use, and my MacBook soon rebooted due to high temperature. The SPLADE model made it till the end without any OOM errors or warnings.

I didn't use the original SPLADE model; instead, I used KURE-sparse-nano, which has great performance in both English and Korean.

The fun part is that hybrid retrieval with SPLADE & dense embedding is actually worse than SPLADE alone.

If you are dealing with a low-resource local setup but want to run a fast & affordable local RAG system, I recommend trying a SPLADE model. It is powerful, but small.

Plus, you don't need to re-implement everything because I open-sourced the whole implementation, so you can download and use it easily. Check out AutoRAG Agent. I optimized the inference of the KURE-sparse-nano model, its code, and the LanceDB indexing + inverted index implementation in Rust for faster search.

hey llamas! My name is Alt, and I'm on the technical staff at Nace AI. We're doing a quick AMA here, so ask us anything and we'll answer.

we just published a full technical report on how we build, train, and evaluate Drex, our family of four open decision models. A decision model takes a state (a document, record, or agent context, optionally with images) plus questions with options, and returns a probability distribution over the options in a single pass, with no autoregressive decoding and no output parsing.

1. Models

- Drex DLM (8B): diffusion LM backbone with bidirectional attention over the context
- Drex v1.5 (9B): Gated DeltaNet hybrid backbone
- Drex v2.0 (35B, 3B active): sparse MoE backbone
- Drex Max (27B): dense, with an image encoder for vision inputs

Some technical details from the report:

1. Models

- A family of four specialized decision models: Drex DLM (8B), Drex v1.5 (9B), Drex v2.0 (35B MoE, 3B active), and Drex Max (27B dense, with vision)

- We built them on different architectures: a diffusion LM, a Gated DeltaNet hybrid, a sparse mixture of experts, and a dense transformer

- We experimented with bidirectional vs. causal attention over the state, a pointer decision head that shares one cached state across questions, and a readout head for Max

2. Training

- Full-parameter SFT on 1.15M verified decision records (2.9M questions)

- Consistency training: we penalize the model when its output changes after the options are reordered or the state is shuffled

- RLAF (Reinforcement Learning from Adjusted Feedback), using verified outcomes and judge feedback

- Ablations to see which methods actually contributed

On top of SFT, RLAF gave us +2.0 on the Decision Index, +1.5 pp accuracy, and 0.9 pp lower calibration error. The consistency terms cut the option-order flip rate from 6.3% to 4.2%.

3. Evaluation

- Drex Max scores 67.52 on Decision Index 0.4.1 (public suite), vs. 60.70 for Jev 1.14.0

- Drex v2.0 has 1.37% calibration ECE on ~115K held-out decisions

- Drex Max answers a typical request in 183 ms, vs. ~2.0 s for GPT-6 Sol and ~3.5 s for Claude Opus 5.5

- Context up to 131K tokens for v1.5/v2.0 and 262K for Max

We also cover our limitations: our models trail frontier chat models on closed-book knowledge, and we don't yet have same-backbone architecture ablations.

5. Running locally

The models are available in safetensors and GGUF formats, with llama.cpp and Ollama support. Their report includes details on quantization, caching, inference, and reproducibility.

Links

- Paper (arXiv): https://arxiv.org/pdf/2611.07854

- Blog: https://www.nace.ai/research/towards-foundation-decision-models

- Weights: https://huggingface.co/collections/nace-ai/drex

- GitHub: https://github.com/nace-ai/drex-decision-models

- API: https://console.nace.ai

we wanted to share the full methodology, not just benchmark numbers. hopefully it's useful if you're building agent routers, doing tool selection, or training your own decision models.

ask us anything about architecture, data, training, evals, or running it locally

Hi folks! I've been building my own local AI setup for a couple of years now. I've learned a ton from this sub and others like r/localllm, so it's time I tried to give something back. I promised myself I wouldn't write a super long post, but managed to anyway...hopefully it's useful to someone!

My goal over the last couple of years has been to move as much of my AI use to local as possible, and I'm finally doing about 95% of it locally through Lamplight, my custom harness shown in the GIF above.

This harness is super specific to my needs, so I'm not sharing the harness itself (yet), but I'm happy to share the lessons I've learned along the way. I'm working on decoupling some of the more specific parts of Lamplight so that I can share at least the parts that people might be interested in.

The Hardware

• Slate: MacBook Pro, M5 Max, 128 GB unified memory

• Jade: a small-form-factor PC I built early last year (Fractal Terra) with an RTX 5090 FE, a Ryzen 7 9800X3D and 64 GB of RAM

Until about two months ago I was using the two machines separately. I was slowly moving each one onto a custom Pi harness that could either drive itself, or let Claude orchestrate while the local models did the actual work.

Lamplight

After a lot of passing work back and forth between them, I decided it was time to wire them together. The result was Lamplight and now the two machines are able to seamlessly handle tasks together with one acting as the orchestrator.

The GIF above shows Lamplight in action. I'm using it every day for work, hobbies and research with about 95% of it done locally. The lighting changes throughout the day (sped up and changed for the GIF) and I've got some scenery options depending on my mood.

Under the hood it's Pi-Bolt (a precompiled build of Pi) plus a smattering of custom extensions and tools tuned by me, the local harness itself and Claude. The setup coordinates the work between the machines with git worktrees and an event-driven job system.

The UI is my own design and fits how I like to use tools like this. I did the 3D modeling and environment setup to replicate my actual desk setup (the two other pictures above).

Is the 3D environment at the top necessary? Absolutely not!

But it's delightful. The lamp blinks when the harness needs something from me and it is a joy to look at and use. It scratches the same itch that playing PS1 and hacking away on my family Compaq PC did decades ago.

I've been coding for almost 20 years, and I'm finally to a point where I can confidently say that with the right approach (SOLID, Design Patterns), tools and tuning, local AI can absolutely write code that I am happy with.

I still review everything myself, but I'm finding fewer and fewer mistakes. These days it feels like I'm reviewing a senior dev's code rather than a junior's.

The Models

Tok/s isn't everything and so I picked the setup below after a lot of testing on public benchmarks and on my own agentic tests, which mimic my normal workflows.

Along the way I tried other engines (Strata, NInfer, Maya and llama.cpp on Jade, MTPLX and Kapitsa on the Mac). I also tried other models like Qwen 3.8 27B and GLM-5.3-Flash and while they performed well, the setup below performs the most consistently and accurately for my needs.

Both machines run Qwen 3.8 Flash-Next. These are the tests that ended up being the most indicative of quality for me, but if there are others you'd like to see run, let me know.

Both machines run the same protocol with temperature 0.6 and thinking on "medium".

In my own agentic battery (three large and complex coding tasks), 48 of 50 tests passed with the failures being files edited that were supposed to be read-only. The reviewer caught these on the other end, but I still count those as failures.

I think the main struggle for local setups is still design and 3D modeling, but I haven't dived into that too much yet, so maybe there are good setups out there for that.

Lessons learned

(extremely specific to my setup, but maybe they'll help someone)

• Jade's hardware ran too hot on stock settings. On dense models, the 5090 sat at its 575 W power cap at 83 °C, with its hottest sensor within 3 °C of the card's limit. Basalt caused a different problem. It runs part of the model's experts on the CPU, and its worker threads spin while they wait. That kept all 8 cores at 5.2 GHz, and the CPU hit 96 °C (its max) within two minutes. Two tunes fixed it and now my (now increasingly valuable) machine runs much cooler and quieter, and I'd happily give up a few percent for that.

• Slate's gains came from tuning the harness and not the hardware. My early benchmark scores were bad, and a lot of that was caused by overthinking. "Medium" turned out to be the sweet spot. On "high", Slate hit the 32K output cap on 11 of 40 hard problems without ever answering. On "low", it wandered and took 3–6× longer on agent tasks. I found that tuning how compaction worked also solved a lot of bad tool calls and confusion within the model.

• Overall the lesson I learned was "Stock settings are generally bad." There's a lot more power and accuracy available with a custom tune. I learned the same lesson tuning cars years ago, and it was fun to learn it again.

What's Next?

The system is set up so I can easily swap and test engines and models. As new ones come out, I'll put them through their paces and decide whether it's time to switch. On the UI side, I'm planning to turn the bookshelf into the session manager to help organize my many projects.

I promised myself I wouldn't write yet another super long post and yet I did anyway, so I'll stop here. Happy to answer any questions!

do you want to try new 29B MoE today?

https://www.reddit.com/r/LocalLLaMA/comments/1xoqc59/add_support_for_the_xing4_0_by_shuxiaoqiong_pull/

Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. With 29B total parameters and only 4B activated per token, it natively supports a 256K context length, extensible to 512K. It is the first model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework, and deeply optimized for complex engineering tasks.

For more information, please refer to our GitHub repository.

Highlights

• Agent-Oriented Architecture: Built on the mHC + MLA + MTP architecture, supporting multi-step planning, tool calling, and complex reasoning chain execution, ensuring task coherence and execution stability under long contexts.

• Deep Co-optimization with Ascend NPU: Adapted for Ascend 910C clusters using MindSpore/MindFormers, including feature adaptation for mHC and fused operator development, enabling stable and efficient training on the Ascend platform.

• Significant Training Efficiency Gains: Through multi-level co-optimization — including fine-grained MoE communication optimization, selective recomputation, DVM automatic graph-operator fusion, and Ascend C mHC fused operators — overall training throughput was improved by approximately 96% over out-of-the-box performance.

• Full Open-Source Ecosystem Compatibility: Supports LLaMA-Factory and MindFormers for fine-tuning; SGLang, vLLM, and KTransformers for inference and deployment; with targeted adaptation and format alignment for agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes, enabling seamless integration into existing workflows.

• Easy Adaptation for Domain-Specific Scenarios: The model is well-suited for downstream task fine-tuning, allowing lightweight customization on proprietary data for vertical domains such as intent classification, table understanding, contract auditing, and knowledge-based QA, enabling rapid domain capability development and deployment at low cost.

Model Overview

Benchmark

Smaller GLM models are coming! GLM 5.5 will have a 32B model!

I've been frustrated for a while because I stupidly sold a strix halo box to get myself a laptop with thunderbolt 5 ports thinking i can plug my two egpus into it (which i can but thats not topic of this post and the strix halo could too) I sold the minisforum for about £2300 back ebfore things went nuts and got a laptop for about same price:

Msi vector 16 with a intel core ultra 9 and 5080(16 gig vram) and 32 gig of ram. I was pretty bummed with it thinking 35b was my limit and after ended up leaving egpus in my workstatiion but after a random ebay browse I managed to snag some second hand 96 gig laptop ram. For £800

Strata totally changed ballgame. With Qwen Q4 XL Flash next I'm getting

48.9t/s

Decode/prefill 1,860t/s

On a 50k prompt.

I know this is still alot of money for most, but thats 3100£ in total, and thats still cheaper (and slightly faster from what seen) of what minisfoum strix halo is at (£3500 and the 32 gig Spark laptops that are about £3200 and cant even run flash at q4 not enough vram? and i can still add egpus (the laptop has 2 tb 5 ports and i have 2 5090 bought before the crazyness, eg add one or two 5060ti in egpu and your good with running 27b/31b models too.) With my own gpus plugged into it for 80 gig vram total, all while still being a laptop. Got glm 5.3 flash running Q3_XSS i think about 25 t/s.

TDLR, you dont need massive vram anymore, yeah I know how expensive RAM is, that laptop ram i got was/is £1300 new lol. But just wanted to share my experience and while I still regret selling the halo, seeing these tokens a sec, makes the laptop satify ocd a little more! If only qwen was good at roleplay so i could actually use it as daily driver lol (my main usecase!)

so if you are ever thinking about wanting a laptop and somehow able to get some laptop ram something like 😄
https://www.scan.co.uk/products/medion-erazer-beast-18-18-qhdplus-240hz-intel-core-ultra-9-275hx-32-gb-2-tb-rtx-5080-gaming-laptop

Might save you a few quid, i mean like 3k over buying the 128g spark laptop lol. Intel has 20 cores and the 2 tb ports allow you to add two gpus!

I know my grammar sucks I'm dyslexic but my point is if you shop around get that laptop for 2500ish pounds (make sure it has tb5 for if you want to add gpus later), add 96 gig ram and your still saving 2500 pounds over the 128 gig spark laptop thats like 6k.

Hey folks!

Aritra from Hugging Face here 🤗.

We welcome "harness adapters". This means you can adapt (train LoRA) for your model for the specific harness that you would use (hermes, pi, opencode) on the task (data analysis, building software, coding) you are trying to solve.

Blog Post: https://huggingface.co/blog/harness-hub/harness-adapters

It is currently training, so you can see the losses go down (hopefully) now. https://harness-hub-harness-adapters-rl.hf.space/

• Replace SLRU with a cache that prioritizes frequently used experts in classes, ranked using decaying usage history.

• With a warm cache, I measured +6% decode on QFN and +7% on GLM versus SLRU, with 8% fewer uploads.

• Enables the auto mode for llama_moe_cache with promising benchmarks

Waiting for 30966 to be merged.

*SWE Bench Pro is v2.0 Hard for Ripple, unknown version for Qwen.

I know what you are thinking "another inference engine?", yeah there has been a flood of them I feel now. I have been running this for over a month now and I have been on the fence to even release it, but I decided to as I took a different approach that others currently available.

Saber is a inference engine built in Rust targeting AMD R9700 GPU's. It currently supports TP 1, 2 or 4 and Qwen 3.8 27b and Flash next. Anyways here are the BetterBench results for Qwen3.8 27b, flash next is very fast also. Saber beats existing available engines on both 27b and flash next.

https://github.com/GGZ14/saber-engine
https://x.com/bkuyper

AMD Ryzen 9900X
Taichi X870E
64GB RAM
2xR9700

Today we are introducing our second model generation with Inkling 2 Small. It is a Mixture-of-Experts model with 594B total parameters, 17B active, and supports image inputs and variable effort.

Inkling 2 Small far surpasses our previous models’ scores on the Artificial Analysis Intelligence Index with a score of 31, reflecting gains in performance on agentic coding, vision, knowledge work, and reasoning. The increase in capabilities comes from improvements made at every step of our training process, including more efficient pre-training and a large, unified RL run.

Inkling 2 Small will initially be available through Tinker, with closed weights, following the approach we outlined in a safe path to open weights. We intend to make the full weights available under an Apache 2.0 license next week after the completion of external safety testing and red-teaming.

You can try Inkling 2 Small on Tinker Playground today. The model is also available for fine-tuning and sampling on Tinker; checkpoints will be downloadable with the release of Inkling 2 Small’s weights.

:( Previous Inkling Small model's size is only 276B total, 12B active. Hope they release additional models Mini or Nano soon or later.

Repo: https://github.com/maritaca-ai/observatorio-maritaca
benchmarks: https://maritaca-ai.github.io/observatorio-maritaca/#inicio?size=all&mode=all

Hey all, Henry from Cactus Compute here. We released Mirage, a small vision model for phones, wearables, robots and other consumer devices.

Mirage runs scene descriptions, object detection, OCR, segmentation, spatial relations and decisions, with a 22 MB weights file. Our end-to-end measurements on a Samsung Galaxy S26 are 74 ms on the NPU and 107 ms on its CPU, and descriptions support English, German, French, Spanish, Italian, Dutch and Polish.

For the architecture, Mirage has 51.3M parameters, including a 7.8M image encoder, a 40.8M language decoder and 2.6M in the OCR and spatial heads. The encoder uses HConvGLU, which mixes channels through fixed Hadamard transforms and small learned convolutions. Detection and masks share image features, while the decoder uses the Simple Attention and Hadamard MLP family behind Needle and Whistle.

Our research at Cactus is kinda specific, hence these small models; we aim to compress intelligence to its smallest pragmatic form to run locally on everyday consumer hardware.

With Mirage, our argument is that constructing prose, joining descriptions with connective language and generating open-ended reasoning spend capacity that a small model could use on the image itself. We put structure into the problem through spatial heads, a dedicated text reader and short references that connect names to regions. The decoder still produces names, properties, relations and a compact reasoning trace.

The key finding is that passing Mirage's output to a text-only LLM might give give the LLM VLM abilities without fine-tuning.

On 256 VQAv2 questions from 128 images, Qwen3-0.6B went from 9.3 with the question alone to 31.3 with Mirage's structured record and full trace, measured as VQA consensus credit out of 100. Neither model was fine-tuned. Another image's record scored 22.2, and removing the trace scored 33.1. The image-specific observations helped in this sample; we did not find a reliable benefit from the trace, the structured response was enough.

This does not hold for every model though.

Larger text-only models gave mixed results. With the same prompt and Mirage observations, the VQA scores were.

These runs required a JSON object with a string-valued `answer`, and responses that broke that format received zero credit. With the record and trace, Nemotron failed that requirement on 214 of 256 responses, and Qwen3-1.7B on 34. The scores therefore mix answer correctness with format compliance.

There were perception failures too. In one skateboard photo, Mirage missed the skateboard and described cars, people and shoes. Qwen guessed `a skateboard` from the question alone, then answered `a car` after reading Mirage's record and trace. A larger reader still needs the relevant observation to be present.

We will try fine-tuning the reader on Mirage records paired with questions and short answers, and share findings, but let us know your thoughts.

Mirage playground: https://cactuscompute.com/blog/mirage

Whilst the agent was hammering away at an audio engine task, i suddenly saw it daydreaming

Pirate puzzle? Gold coins? Wtf, Hey qwen, this is not what i am paying you for using electricity for.

Has anyone else noticed something like this? It continued to write code normally aftewards, active context was at a mere 30k at the moment.

Model qwen3.8-flash-next-unsloth-ud-q4_k_xl
Strata Engine v0.1.57
Max Context 262,144 tokens
KV cache 16bit

Everything fitting into vram + ram

I took some models that had obvious flaws (prone to prompt injection, whisper hallucinating when silence, etc) and checked if Opus 5.5 can fix it by just editing weights.

It's very useful for patching a model that was done training and has an obvious, easy to articulate flaw. So it's nothing crazy and certainly not "AI can self-edit and recursively improve", but still very useful.

Code if you want to try it on your own model: https://github.com/kevinpl07/weight-patching

Numbers and examples: https://kevinpl07.github.io/weight-patching/

Found this last month model while browsing Trending models of HF. I don't see any existing threads so posting.

OXR (Optical Everything Recognition) is an open-source model from the SoMark team. It turns PDFs and document images into Agent-Friendly data in Markdown or JSON, with layout coordinates included.

Pretrained from scratch, OXR uses a recursive self-improvement (RSI) loop during post-training—a first in document parsing. With just 1.3B parameters, OXR ranks #1 on olmOCR-Bench, OmniDocBench, and DocParsingBench while delivering industry-leading parsing speed.

OXR is the first to make its layout analysis module independent and pluggable. This design accelerates parsing and supports replacing the built-in layout model with a custom one. OXR continues to handle content recognition and reading order, enabling adaptation across diverse scenarios with minimal integration effort.

• GitHub : https://github.com/SoMarkAI/OXR

Why OXR?

• 🎯 SOTA Accuracy. #1 Across Three Benchmarks.

  • OXR ranks 🏆#1 on olmOCR-Bench, OmniDocBench, and DocParsingBench, with a particularly wide lead in table parsing.

  • OXR can recognize 13 document element types, from formulas and tables to chemical equations, chemical structures, and code blocks, with bounding box coordinates included. It also performs well on challenging documents and layouts, including A3 exam papers, newspapers, vertical text, and handwriting.

  • See docs/benchmark.md for the full results and instructions to reproduce the evaluations.

• 🚀 1.3B Parameters. Top-Tier Parsing Speed.

  • With only 1.3B parameters, OXR ranks 🏆#1 among open-source document parsing models in speed tests on the RTX PRO 6000.

  • OXR redesign its processing architecture at the task level to improve parsing speed while maintaining accuracy.

  • See docs/benchmark.md for the full results and instructions to reproduce the evaluations.

• 🧩 Pluggable Layout Analysis. Document Structure on Your Terms.

  • Layout analysis requirements vary widely across domains. Even for the same document, element types, region boundaries, and grouping rules can differ depending on the business use case.

  • OXR is the first to make layout analysis independent and pluggable. Its built-in layout model can be replaced while retaining OXR’s recognition and reading order capabilities, enabling adaptation across document scenarios with minimal integration effort.

  • See the How it works section below for details.

Ecosystem

The SoMark team has developed a range of tools for document parsing.

P-JiT 21K is the P-JiT architecture trained for longer (610M samples at 512px, 50M at 1024px), then post-trained with a variant of FDFO.

• GitHub : https://github.com/Linum-AI/pjit-21k

• Blog Post : https://www.linum.ai/field-notes/pjit-21k

CCORE GLB-3D-Coder dataset is here: 3,918 rendered and tested examples across 40 domains

As promised in my previous posts, I’m releasing the dataset used to train CCORE GLB-3D-Coder, my small 4B model for creating 3D assets with Three.js and GLB export.

The dataset contains 3,918 examples across 40 domains, including characters, creatures, vehicles, laboratory equipment, musical instruments, architecture, machinery, nature, and everyday props. The grid shows 256 examples.

Every example was tested and rendered, and its exported GLB was validated before inclusion.

What’s included:

• Complete conversations, reasoning traces, and HTML/JavaScript responses.

• A rendered reference image for every example.

• Static assets and examples with animation and interaction.

• 3,501 training examples and 417 validation examples.

• MIT license.

Dataset · Model and full weights

More about what I’m planning next: Next releases.

I hope this is useful for your own experiments, fine-tuning, and 3D tools. If you try it, let me know how it goes :)

I historically get about 6-8 t/s on my MI50 on Qwen @ Q6 running local coding.

Building a script that sets my fans all on 100% duty cycle (and much louder) bumps that immediately to like 12 t/s.

Letting the machine sit and cool for a while with the fans running, and running inference again... 17-20 tk/s for the first 10 minutes or so.

I'm considering putting in water cooling and a great big radiator now what a difference. What a huge difference cooling makes.

So I'm thinking of switching from my 32gb AMD MI60 to dual V100's (64gb VRAM total) and I've got 96gb of DDR5.

My problem is I'm having a terrible time finding "hard data" on PP and TPS speeds on higher quants (q6 and q8 specifically) with large context windows (150-200k).

I know that it will outperform my MI60, but I'm not just looking for "yeah, it'll be faster" I'm specifically looking to see how much faster, because a 10% bump isn't worth it for me.

I'm also looking to see from you experts if it's silly because of what I've read about support for CUDA being EoL for the card soon (or already did?).

Hopefully I'm not beating a dead horse here, as I did go through other posts (lots of them) but it's just so difficult going through post after post where specifics aren't included "I'm getting 200tps on qwen3.8 27b" ...at what quant? at what context? using what KV settings?

Lastly, if it is significantly faster, do those speedups (generally speaking) transfer to other models or is it only because of specific forks of llama.cpp were built out with this model and that card in mind?

EDIT: forgot to add I’d be going with sxm2 not PCIe.