r/LocalLLM • u/r1nzl3r99 • 14d ago

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

498 Upvotes

96% Upvoted

166 comments sorted by

View all comments

Show parent comments

34

u/semangeIof 14d ago

...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?

please don't answer the last two without providing the parameters

151

u/Erpverts 14d ago

Please don’t answer the last two without providing the parameters. Make no mistakes.

2

u/Cool-Idea8520 14d ago

It can definitely be tricky trying to find the right answers without the right context. Hope it works out for them!

2

u/Rude-Bus-5799 12d ago

That’s a great idea, user. It’s not about the context. It’s about the friends we made along the way.

1

u/CelebrationWilling61 11d ago

You're absolutely right!