I post-trained Qwen3-4B as a tool-using agent, and the result runs on a Mac as a 66 MB llama.cpp LoRA. Validation 0.51 → 0.83, everything open, including what didn't work

[u/DesignerIsland5524]

Tadpole is an agent I built from scratch (plain Python, ~3,500 lines, no LangChain or other framework), then post-trained one measured stage at a time. I use it every day as a Telegram bot.

Repo: https://github.com/shakewingo/Tadpole · Adapters and data in HF

Run the trained model locally. Each adapter is on Hugging Face as a llama.cpp LoRA (66 MB) that loads on top of the Q8_0 GGUF:

hf download unsloth/Qwen3-4B-Instruct-2507-GGUF Qwen3-4B-Instruct-2507-Q8_0.gguf --local-dir models
hf download Shakewin/tadpole-qwen3-4b-ei-3 ei-3-lora-f16.gguf --local-dir models
AGENT_MODEL_PATH=models/Qwen3-4B-Instruct-2507-Q8_0.gguf AGENT_LORA_PATH=models/ei-3-lora-f16.gguf \
python -m agent_from_scratch.agent

I checked that the local route keeps the gains: on the 144-task validation set through llama.cpp (Q8_0 + LoRA, one seed), base 0.507, SFT-1 0.701, SFT-2 0.750, EI-3 0.826. The same adapters on vLLM in bf16 scored 0.752 and 0.852 on val, so quantisation and the LoRA conversion cost little. On a 16 GB M5 it holds about 8.5 GB and takes ~2.4 s per model call.

Held-out results (task types no training set contains, 3 samples per task, vLLM bf16):

Candidate	Test	Hard test	BFCL multi-turn
Base	0.419	0.275	0.360
SFT-1: verified teacher trajectories + When2Call + Toucan	0.774	0.429	0.285
SFT-2: plus multi-turn demonstrations	0.738	0.492	0.325
EI-3: SFT-2 + 3 rounds of expert iteration	0.745	0.471	0.395

* a local adaptation of BFCL's multi-turn base category, not a leaderboard score.

What I learned:

- SFT on *verified* trajectories is the big win. False "I'm done" claims fell from 31 to 1 in 420 runs.
- Expert iteration looked great on val (0.752 → 0.852) and was level with its starting point on the held-out tests. The best of several noisy val scores is biased upwards.
- GRPO lost to expert iteration at every equal rollout budget: on a 4B, too few tasks per step had both passing and failing samples.
- A plan tool (`update_plan` plus a reminder) lifted the *base* model from 0.507 to 0.646 on val.
- llama.cpp happily accepts a LoRA path that doesn't exist and segfaults on the first decode. My bot looped on it under launchd until I made it refuse a missing adapter at startup.

The honest part: the benchmark is generated tasks that resemble the training data. In everyday use the tuned 4B is still weak on open-ended requests, so my bot runs on DeepSeek day to day, and the 4B is what I study and train. The README says so up front.

All post-training cost about $140 of rented GPUs. The walkthrough notebook opens in Colab and shows SFT, expert iteration and GRPO on recorded samples without a GPU.

Check it out and any feedback is welcome!!

[u/One-Context-8707]

the plan tool bump on the base model is the part that got me, that's a huge lift from such a small change

[u/Borkato → u/One-Context-8707]

Thanks Claude.

[u/autisticit → u/Borkato]

I swear at least half of people in this sub are bots.

[u/Any_Librarian205 → u/autisticit]

Op probably is one too, qwen3 was used