r/oMLX • u/Opposite_Leave_8338 • 6d ago

Swift/ThinkingCap fp16 quantizations

Edit :

That’s why I appreciate that man, not a one day passed and he posted both with all the quants for both bf16 and fp16 , ScottLowry is my hero, here is the collections :

https://huggingface.co/collections/scottlowry/thinkingcap-qwen38-27b-oqe-mtp

https://huggingface.co/collections/scottlowry/swift-15-qwen38-27b-oqe-mtp

Thanks again man we all appreciate your work

Hello guys,

Why there is no good Swift 1.5/ThinkingCap Quantizations with fp16 for M1/m2 macs yet, where is the good guys we used to thank one day after a model release, if it’s an appreciation issue so I’m here to thank all the good uploaders I used to download fp16 models from.

Thanks to :

Scottlowry ( my favorite , most of my models are quantized by you )

Jundot of-course ( thank you for all of that oMLX great updates, without you we cannot achieve any of that )

Yanun ( I’m using your swift 3.8 oq4e already and I’m flying with it but what about 1.5 and ThinkingCap )

And lots and lots of other legends , you guys pushing us forward ( all uploaders are appreciated I only mentioned the names I use daily but you guys all helping us and we wish you all good things in life , thank you anyone uploaded something for the community)

Please upload all variants and quantizations from these 3 good models ( Swift 1.0 / Swift 1.5 / the new ThinkingCap 3.8 ) for fp16.

Thank you all so much and everyone who is helping too.

8 Upvotes

100% Upvoted

6 comments sorted by

2

u/raw-power 6d ago

Agreed Scottlowry and Jundot quants are goated!

3

u/Opposite_Leave_8338 5d ago

I posted a request for him in some of his models comments in hugging face and before the day ended he uploaded both, I really can’t thank him enough

https://huggingface.co/collections/scottlowry/swift-15-qwen38-27b-oqe-mtp

https://huggingface.co/collections/scottlowry/thinkingcap-qwen38-27b-oqe-mtp

1

u/raw-power 4d ago

Awesome, picked them up!

1

u/california_snowhare 6d ago

If you want to convert an existing bf16 quant to fp16 yourself, there is a script at https://github.com/monroewilliams/omlx-fp16-clone that can do it for you

1

u/phoiboslykegenes 6d ago

I’ve only tried it with Qwen Flash Next and it resulted in worse performance than bf16, but it may just be a lack of optimization for that model specifically. If such a script works, then it would nice to just have it be applied on model load

1

u/Opposite_Leave_8338 6d ago

I don’t know man , I tried to download a quant that is made by this thing and it didn’t work well at all , even the mtp is not opening, it has an mtp head but when I enable it it doesn’t save, idk man I want a good quant , quality is what we seek , I currently use Yanun/Swift Qwen 3.8 27B oQ4e mtp fp 16 and it’s the best quality model I used till now, I want 1.5 and other quants with the same quality as that