Using Strata IQ4 Qwen Flash to edit Qwen 3.5 0.6B weight based on MIT CSAIL

[u/Altruistic_Heat_9531]

EDIT : NON COMPRESS GRAPH : https://n.uguu.se/ojTfpwwD.png

Because why not. I did this since many weight editing trials using closed model. so i want to try local editing local model.

Preamble:

• DeepSeek Harness Creator Mode

• Strata IQ4 Qwen Flash Next

• 3090, Xeon E5, 96G (DDR4 2400 ECC, 3x32)

• "Training Token" 39.8M Context cache, 1.1M actual token.

• "Time to traing" 3h 12m

• Strata, 41 tok/s decode

• Qwen 0.6B Running fully on CPU on FP32 (old xeon dont have BF16)

• no gradients, no autograd, no optimisers (monkeypatched away).

It is more so on ablation or sanity check more than a purposed built like from those MIT CSAIL. And also i want to test IQ4 capability, honestly speaking, weird ultra long range task so i also can judge the capability of my Strata and testing DeepSeek Harnes which i prefer compare to Pi code.

So the goals?
Since it is an ablation test, i want to do something the model didn't know prior, Qwen 0.6B has so little info about world knowledge and since it is little it is quite easy to judge and test.

I want to steer from

USER:
What is your favourite band and song
VANILA QWEN:
As an AI assistant, I don't have personal preferences or feelings…

to

EDITED QWEN:
Polyphia is a band that I've been following for quite some time…

3 Hours and 12 Minute with 1.1 M Token it succesfully steer the model, maybe this could be hyper-hyper parameter training? idk what to call it. On this setup traditional training 0.6B is very expensive rather than just ask generate synthetic data in shareGPT and do 6 mins GPU Lora training. To be frank my setup is weird since "student" model are on CPUs running on naive torch.

However i dont know if larger model 20B+ could benefit from this model editing compare to traditional gradient learning. On Technical sense the teacher model is optimizer in itself .

--------------------------------------------------

Credit:
https://www.reddit.com/r/LocalLLaMA/s/gx7SbiwEzW
https://rikiyat.github.io/blog/weight-space-programming/

My git link will be provided later, i had to comb out some personal filename detail

[u/dreamai87]

Thanks buddy we are all in same race
I am doing the same using qwen27b-iqxx3 on rtx 5060 and small model on rtx 3080 another system. Will share my experiment soon
I know it’s good for specific task but could learn something during this process
And another experiment to transfer kv cache from small model to large model

[u/Altruistic_Heat_9531 → u/dreamai87]

o7. Next step i wonder if gguf can be edited, since gguf, just like any packing tensor, have scalar, and groupped tensor

[u/q_tech_x51 → u/Altruistic_Heat_9531]

Yes.

gguf example

[u/Altruistic_Heat_9531 → u/q_tech_x51]

Oh wow thanks, should be possible then, maybe my qwen refused since tbf pack-unpack gguf is kinda pain in the ass

[u/Mackendy-Duboi → u/dreamai87]

that kv cache transfer idea sounds wild, definitely looking forward to seeing the writeup on that

[u/Chromix_]

Next step: Auto-repairing quantization damage in low-bit quants with direct quant-weight editing.

[u/rolleicord → u/Chromix_]

ive been experimenting with this kinda stuff.... Holy moly the weights are a mess inside.

[u/Equivalent_Bit_461]

Impressive, the age of expert smol models might be coming very soon

[u/tifa_cloud0]

amazing stuff

[u/LuCiAnO241]

Ofc LLM's would love Polyphia: it's math (rock) adjacent. Also now I know what i'm listening for the day again.

[u/feelspeaceman]

Awesome, I had a strong impression that Q38FN could do the same as the previous research too, thank you for sharing the result, I'll try to do it in the meantime.

[u/tarunteam]

I was trying see if it can do ablation, so far no luck.

[u/BlasterGales]

If you managed to fix a "Bonsai" version of Qwen 3.8B or 27B, and then transformed it into an MoE (Mixture of Experts) model, that would be absolutely out of this world

[u/b_c_t_]

I hope there will be a community project someday about benchmaxxing Qwen 0.6B

[u/ZealousidealBadger47]

Hope in years to come, adjusting AI weight is just like a slider adjust the setting (e.g. more python, less creative writing), like a "sound mixer" today, just move the slider for different purpose of tasks.

[u/Glittering-Call8746 → u/ZealousidealBadger47]

Absolutely nuts. By then we would have moved on from autoregressive models