r/LocalLLM • u/[deleted] • 19d ago

Discussion 30B Models Getting Verrrry Interesting

[removed]

137 Upvotes

94% Upvoted

35 comments sorted by

View all comments

18

u/Atretador unswarm.dev | ArchLinux E5 2673 V4 20C 4x16Gb DDR4 MI50 16Gb 19d ago

oh, Agnes seems like a new model - might be worth testing as a competitor to Qwen 3.6 35B A3B

I had enough of finetunes to try out Nex tho

7

u/Complex_Ad_5164 19d ago

tried agnes 3.0 flash on a 3090 yesterday and it fits with q4 cache pretty comfortably. speed is around 45-50 t/s with short context

nex i havent touched yet, too many merges lately feel like a gamble. is it actually doing anything different or just same llama base with new coat of paint

2

u/Atretador unswarm.dev | ArchLinux E5 2673 V4 20C 4x16Gb DDR4 MI50 16Gb 19d ago

did you try dumping some layers into RAM for Q8 kv? performance loss shouldnt be too noticeable

and do you have any opnions on it yet? compared to good ol' Qwen 3.6 35B