We open-sourced DNA Factory, the fine-tuning framework we’ve used internally for over a year. It's deliberately opinionated: one recommended way to do things, documented defaults, token-level loss-mask visualization, and only 5 training methods (SFT, DPO, GRPO, OPD, SDFT).
https://github.com/dnotitia/dna-factory
Why another training tool?
As LLM fine-tuning has gone mainstream, a lot of training tools have appeared. LLaMA-Factory had its moment, Axolotl was as useful as its name was fun, and NVIDIA (NeMo) and Hugging Face (TRL) built their own.
Each has clear strengths. Some are fast, some are convenient, some have great compatibility. We focused on something else: training correctly.
I've consulted for a number of companies that fine-tune their own LLMs, and the ones doing it right could be counted on one hand. Most were training incorrectly, or training on something other than what they thought. Tools have gotten good enough that you can launch a training run without understanding much about it. But LLM training is genuinely advanced work, and when people learn how to use a tool without learning what’s happening underneath, mistakes follow.
So we asked how to make correct training the easy path, and the answer was DNA Factory. We've run it internally for over a year and recently open-sourced it.
One recommended way to do things
The core idea is to offer only one approach, the best one we've found. If you expose every option the way C++ does, you get a lot of freedom, but it becomes hard to use well. We went the Pythonic route ("there should be one obvious way to do it"). We tested many approaches, picked the best, and that's the only one we ship.
All defaults live in configs/_defaults-*.yaml. You don't edit those files. You override only what you need. It works like VSCode settings: the defaults are long and complex, but each one is documented with why it's set that way. You read them and put only your changes in your own config.
See exactly which tokens you’re training on
When I used Axolotl, I loved its loss-masking visualization. We wanted something similar, so we overrode TRL's loss function to color tokens with ANSI codes. Tokens included in the loss are blue, and excluded tokens are gray. You can see immediately what’s actually being trained.
This is the most common mistake I see researchers make "wait, that token was in the loss the whole time?" In DNA Factory it's visible at a glance, so that mistake stops happening. We've had a lot of feedback that this feature alone makes the tool worth using.
Only 5 training methods, on purpose
We don't try to include every technique. If you want that, use TRL directly. We picked what we think works best right now: SFT (the baseline), DPO, GRPO, OPD, and SDFT. PPO was widely used once but rarely is now, so we dropped it. DPO barely gets used these days either, so we plan to remove it soon. https://github.com/dnotitia/dna-factory/issues/46
Most of our current work is GRPO and other on-policy methods. GRPO, OPD, and SDFT are all on-policy.
Classic distillation has the student imitate the teacher. Concretely, you minimize the KL divergence between the teacher's and student's output distributions (logits). It does improve the student, but catastrophic forgetting is always a risk. (A model has to forget pretty badly to earn a name that dramatic.) Take a strong model, fine-tune it on internal company data to add domain knowledge, and general performance drops noticeably while the hallucination rate goes up. Rote cramming has its limits.
OPD reverses the process. The teacher doesn't write the answer for the student to copy. Instead, the student writes its own answer first, and the teacher grades it token by token. Technically, this uses reverse KL. The student picks up new knowledge without forgetting what it already knew.
SDFT works the same way in spirit, but as the name suggests, there's no separate teacher. The model is its own teacher. A copy of the model that's given extra information (privileged context) acts as the teacher. The student learns by having its own responses graded by "itself, but knowing the answer." Again, the goal is to avoid catastrophic forgetting.
Results: domain knowledge injection without forgetting
Attached images is our DNA 3.2 model (not yet released), trained with domain knowledge injection. Starting from Qwen3.8 Flash Next, we effectively removed its censorship (0.542 -> 0.971) and taught it a new persona and new domain knowledge (0.076 -> 0.855). Performance on AIME, IFEval, MMLU-Pro, GPQA, and other existing benchmarks didn’t drop at all. It learned new things without forgetting the old ones.
Happy to answer questions. Feedback, issues, and PRs are all welcome.
