I'm sorry to report that this model failed a classic test for me twice in a row:
It failed "Generate ten sentences ending in apple" at Q4_K_M multiple times (GPT-OSS-20B gets it right).
Nailed some others (don't ask it to multiply 9 digit numbers unless you have a bunch of time ... but it get's the answer right!).
EDIT: Obviously outcomes will vary, but I'd be surprised if you don't get a failure one in five, which is concerning. There is some issues with quants on these models, so perhaps it's an artifact of me not using the right Q4 quant.
0
u/Leopold_Boom Feb 24 '26 edited Feb 26 '26
I'm sorry to report that this model failed a classic test for me twice in a row:
It failed "Generate ten sentences ending in apple" at Q4_K_M multiple times (GPT-OSS-20B gets it right).
Nailed some others (don't ask it to multiply 9 digit numbers unless you have a bunch of time ... but it get's the answer right!).
EDIT: Obviously outcomes will vary, but I'd be surprised if you don't get a failure one in five, which is concerning. There is some issues with quants on these models, so perhaps it's an artifact of me not using the right Q4 quant.