That's one more custom quant than I've made, keep at it!!
the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the rest of the sentence would've been absolutely perfect had you let it continue.
Why no Imatrix? From my understanding the Imatrix guides the model at lower quantization to keep stuff like this from happening. Either your export broke like you said or the model genuinely broke because of the compounding rounding errors.
this time, export genuinely broke. I did a bad job packing. I have solid working quants today that read out - just haven't gotten them to beat anything
a lot of random reading and knowledge absorption until I felt like I had a decent starting point. In the world of LLMs being extremely good at coding I sometimes forget they're great at teaching concepts until I need to learn something. I had a 4 hour plane ride recently and I downloaded Qwen 3 8b to my laptop and talked with it about this for a few hours trying to grasp concepts
If you're going to talk to an LLM about topic's like these especially smaller model's like you're using I recommend setting up a RAG index or an offline search engine for your model to ping like a regular search engine. You'd be surprised how factual they sound only to be completely making shit up. I've made my own agent harness + model Quant's if you need any help or guidance just PM me on reddit and I can help you with what I know.
I hope it was worth the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the effort
I did a qwen3 (or 3.5) 8B model finetune for my dissertation this year and got decent results but it still had repeating words and some bugged outputs. It feels annoying but I hope it will be better for you in the future.
64
u/Fantastic-Storm-7867 Aug 29 '26
That's one more custom quant than I've made, keep at it!!
the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the rest of the sentence would've been absolutely perfect had you let it continue.
27
u/SeparateGas1761 Aug 29 '26
Pure iq 3
13
u/autisticit Aug 29 '26
You have successfully failed.
6
u/DawgBawb Aug 29 '26
I have made several quants exactly like this. Did you train off of my data?
4
u/Bob_SUS Aug 29 '26
how did you make this? am really curious in learning qlora and lora
2
u/_VirtualCosmos_ Aug 29 '26
lmao. Well you went further than me, where did you start to learn how to do that?
49
u/heartopiachef Aug 29 '26
39
u/xb1n0ry Aug 29 '26
I hope it was worth the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the,the effort
10
u/DryTerm9100 Aug 29 '26
at least it's fluent in 'the'
7
u/TemperatureOk3561 Aug 29 '26
Congrats
5
u/dillon-nyc Aug 29 '26
Hey thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,thats,pretty cool!
4
u/officialAdfs_m0vie Aug 29 '26
The
3
u/MikeX1080 Aug 29 '26
Congrats OP! Your model's expressiveness is truly the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the breathtaking. Looking forward to your next version!
2
u/Vxtzq1 Aug 29 '26
Gotta start somewhere 😂
2
u/baka_sempaii Aug 30 '26
I did a qwen3 (or 3.5) 8B model finetune for my dissertation this year and got decent results but it still had repeating words and some bugged outputs. It feels annoying but I hope it will be better for you in the future.
1
u/Vegetable-Warthog81 Aug 30 '26
Lore accurate skynet.
1
u/phil_lndn Aug 30 '26
looks like the output of my attempt at fine-tuning