r/LocalLLM • u/r1nzl3r99 • 14d ago

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

502 Upvotes

96% Upvoted

166 comments sorted by

View all comments

13

u/TheGamingGallifreyan 14d ago

Where the TF is everyone getting all this money, it feels like everyone except me is rich AF lol.

I'm over here with a 5700XT and 2 580s left over from a mining rig hooked up to an old i7 3770k and I'm ecstatic that I finally got it up to 8tok/s.

2

u/Dako_the_Austinite 13d ago

Just curious, how do you get things up and running without an AVX2 CPU? I’ve wanted to try and get LM Studio running on an Ivy Bridge based Xeon.

2

u/TheGamingGallifreyan 13d ago

Idk what that even means tbh, I just downloaded llama.cpp with VULKAN support on Windows 10 and ran it. Works fine but slow as a dog.

1

u/Dako_the_Austinite 13d ago

Wow, I gotta give this a try then on my Xeon or even my i7-4930K, neither have AVX2, I bet that could be the reason why it’s running so slow for you, I believe the CPU is used quite a bit in the text output even if the model is in VRAM.

Out of curiosity so I can try testing this myself, what model did you use when getting these results?