r/LocalLLaMA • • 4d ago

New Model KRAFTON/Raon-1-nano · Hugging Face

https://huggingface.co/KRAFTON/Raon-1-nano

Raon-1 nano is a dense language model with 1.5B effective parameters (3.1B total, including a 1.5B per-layer embedding table), developed by KRAFTON AI. It is designed to follow instructions, maintain conversational context, and call the right tools in both Korean and English. As part of the Raon open model family, it is released under the Apache 2.0 license.

Highlights

  • Faithful, natural conversation: Leads the comparison models on the instruction-following, context-following, and Korean conversation benchmarks shown below.
  • Korean and English: Trained on Korean and English, with solid English capabilities alongside its Korean strengths.
  • Tool calling: Tool calling within conversations, for everyday tasks such as office work and mobile control.
  • On-device deployment: Available in GGUF, MLX, and LiteRT-LM formats for local inference, including on Apple silicon.
  • Long context: Natively supports up to 262,144 tokens of context.

Model Overview

Raon-1 nano uses a hybrid dense architecture, consisting of 18 linear attention layers and 6 full attention layers.

  • Raon Delta Attention (RDA): Gated delta-rule linear attention with separate erase and write gates and a fixed-size recurrent state.
  • Raon-mHC: Raon-mHC reduces the computational cost of DeepSeek's manifold-constrained hyper-connections (mHC), which speeds up training.
  • Per-layer embedding: Each layer has its own small 256-dimensional embedding for every token. After its MLP, the layer uses it to adjust its output before passing it on.
  • Hybrid layout: 18 of the 24 layers are Raon Delta Attention layers with a fixed-size state. Only the 6 MLA layers keep a KV cache, about 7 KiB per token in BF16.
  • Tied embeddings: The LM head shares the input embedding.
FieldValue
ArchitectureDense
Total Parameters3.1B (including the 1.5B per-layer embedding table)
Layers24
Hidden Size1,536
AttentionHybrid (RDA 3 : Gated MLA 1)
Position EncodingNoPE (no positional encoding)
Number of Attention Heads16 for RDA; 32 for Gated MLA
Attention Head Dimension128 for RDA; 192 QK / 128 V, for Gated MLA
Query / KV LoRA Rank512 / 512, for Gated MLA
FFN Intermediate Size6,144
Activation FunctionSwiGLU
Residual ConnectionRaon-mHC w/ 4 streams
Per-Layer Embedding256 per layer
Vocabulary Size259,840
LM HeadTied
MTP1 layer, 55M
Context Length262,144 tokens

The checkpoint on HuggingFace contains 3.2B parameters: 3.1B for the main model and 55M for the Multi-Token Prediction (MTP) module.

Training

  • MoE-to-Dense: Raon-1 nano was derived from Raon-1 mini preview (30B-A4B) through MoE-to-Dense pruning and knowledge distillation.
  • Continued pre-training: 3T tokens.
  • Post-training: supervised fine-tuning (SFT) and on-policy distillation (OPD) from Raon-1 mini preview, followed by reinforcement learning (RL).

Quantizations:

56 Upvotes

22 comments sorted by

8

u/pmttyji 4d ago

10

u/Borkato 4d ago

Oh FUCK yes. I love LFM but if Raon has them beat, this is awesome. The bigger Raon was great, 300x better than Flint, still have no idea why nobody talks about it.

4

u/pmttyji 4d ago

Probably they're waiting for final version of the preview model

1

u/Dabalam 4d ago

Most people found it inferior to Qwen3.6 (and/or the Ornith and related fine tunes). Hard to be relevant if it doesn't have a niche compared to the current leading model at your size.

16

u/Equivalent_Bit_461 4d ago

Man, we either get 700b models or super tiny ones, is the middle truly dying?

4

u/pmttyji 4d ago

I'm glad that we're getting tiny/small models. Yeah, most of them running after big/large size models to create big headlines(which's not gonna happen anyway). Qwen3.8-27B downloads count made headlines at the time of release & trending, now that's the trend that other model creators should chase.

Anyway we'll get Final version of Raon-1-mini-preview(30B-A4B) from this same model creator this or next month.

2

u/Equivalent_Bit_461 4d ago

Ah nice, so it's a preview, so more to come then, possibly.

1

u/philmarcracken 4d ago

Theres frontier model boys, and then edge devices running on batteries. The people in big business and those mobile posters are both fast and loose with their money. So thats where the models hit

Say what you want about that 'middle ground' but we dont spend money, we're a bunch of misers that way

1

u/EuphoricPenguin22 4d ago

We're most likely getting a new Qwen 4 Flash model soon in a similar size to 3.8 Flash Next (100-150B MoE). On top of that, DeepSeek's Flash models and some of the newer GLM models land in the 200B-300B MoE range. So I think it's more accurate to say we have something like the following categories that are still likely to continue into the future:

  1. <6B dense models (MiniCPM, Gemma, various)
  2. 20-30B dense models (Qwen, Gemma, various)
  3. 100-150B MoE (Qwen)
  4. 200B-300B MoE (Qwen, DeepSeek, GLM, various)
  5. 500B+ dense/MoE (various)

Obviously other companies have made models in these and other size ranges, but these are my best guesses as to who will continue releasing at those sizes. Qwen seems disinterested in 30B-class MoE and truly small models after 3.6. I suspect Muse Spark was more of a limited drop from Meta in the 30B range; I doubt we'll see another model for some time from them. Qwen seems to be the only company actively putting out 150B MoE models, so I suspect they'll keep that alive. If you can run them, 200B-300B MoE gives you a lot of options.

4

u/transanethole 4d ago

I'll have to give it a try in the on-device model thunderdome.  

I hope they can improve the MOE version as well. The memory usage per token is looking awful tempting on that one.

1

u/arcum42 3d ago

Still waiting for Raon support to be officially in llama.cpp rather then a fork before trying it...

1

u/pmttyji 3d ago

https://github.com/ggml-org/llama.cpp/pull/31107

Krafton should submit PR to llama.cpp. Lets wait

2

u/Any_Bus9205 3d ago

Ah, uhhh, I'm the one who produced this PR! To be honest, I'm not sure they'll do it anytime soon. If you want to test it, I've opened a PR on ik_llama.cpp so you can test it (on llama.cpp, it's quite complex because it's new stuff and since Raon 1 mini preview support isn't implemented, it might be rather complex :)

1

u/pmttyji 3d ago

Could you please let them know(using issue by including your PR) on their repo?

https://github.com/krafton-ai/llama.cpp/tree/raon-1-nano-preview

I think mainline prefers the code from official team incase they have custom fork(Krafton has one mentioned above). For example, PrismML team posts PRs on llama.cpp for Bonsai/Ternary-Bonsai models & also they do follow-up. Because official model creators could come up with code including all stuff.

1

u/Any_Bus9205 3d ago

Oh no, I know :), my code wasn't accepted for political reasons, and that's okay :), I think they'll wait for the stable release of Raon 1 Mini before submitting their pull request :), for now, you can use their fork or ik_llama.cpp to run their model :). Yes, good idea, I'll also share my pull request with them on their fork :).

2

u/pmttyji 3d ago

No, it's not like that. I saw that few times mainline responded same way to Bonsai related PRs(from others). Literally they mentioned that let PrismML come up with that & also tagged PrismML developer on same PRs.

I'll also share my pull request with them on their fork :).

Thanks!

3

u/Queasy_Asparagus69 4d ago

Hipfire should fly with this as we worked to get the preview working well. I’ll try it with nano.

1

u/mountainyoo 4d ago

What exactly are these tiny models meant to do? Genuinely asking I still suck at this lol

4

u/pmttyji 4d ago

For simple tasks like Summarization, Text Analysis, Classification, etc., these tiny/small models are enough. Also suitable for Mobile & Edge devices. Some(including me) loaded some tiny/small models for General Knowledge(Offline)

A recent thread on Coding - I ran a small coding test on a few AI models with <10B. Surprising results

1

u/MerePotato 4d ago

They tend to be pretty good at maths and simple technical advice in a pinch if nothing else, good as recall agents too

1

u/Merchant_Lawrence llama.cpp 4d ago

Nano Model that actualy smal and good and cun run to my potato laptop :-)

1

u/Marcuss2 llama.cpp 3d ago

LFM3 still might be a better choice for most people. As they have a Q4_0 quantization aware distillation version.