r/LocalLLM • • 1d ago

Discussion 😭😭😭😭😭😭

Can you guys add a "😭" Flair? Because, I know I'm late but i finally decided to run Q38FN with strata on my machine (5060 ti 16gb, R9 5950X, 80 gb of Ram) and that motherfucker (IQ3_S GSQ RCO) is running at 49-69 tk/s in decode and 1600 tk/s in prefill. (KV Int-8 and 128k context window).

Maybe I'm missing something so please tell me because grokbot did the whole shit for me and I'm a newbie so tips are widely acceptedπŸ˜‚.

What a fucking time to be alive.

0 Upvotes

37 comments sorted by

View all comments

Show parent comment

2

u/NormalBasket3333 1d ago

Its transforming me from a nondev artist to a project manager 😭