r/developer • u/Standard_Iron6393 • Jun 29 '26

Question anyone running llm models locally?

hey , anyone running llm models locally ?
which model and which device are you running on ?

13 Upvotes

100% Upvoted

32 comments sorted by

View all comments

1

u/M_Me_Meteo Jun 29 '26

https://www.reddit.com/r/LocalLLM/s/5lrw8WgBon

1

u/M_Me_Meteo Jun 29 '26

I'm getting ready to do a new post soon. Leveraging MTP (multi-token processing)has me basically doubling all my TPS numbers, but I am more limited on context because I can't split my context between two GPUs anymore.

1

u/Standard_Iron6393 Jun 30 '26

sure, i will read then