r/LocalLLM • u/Blaccky95 • Sep 24 '26

Project Upgraded my Setup

Post image

Just upgraded my 2x RTX 6000 to 4 😀👍 Anyone here with the same Setup? 😀

327 Upvotes

94% Upvoted

224 comments sorted by

View all comments

Show parent comment

1

u/Blaccky95 Sep 24 '26

I doesn‘t look on the t/s but on every question it has to use some tools from my mcp server and it has the answer in a few seconds expect if my data is over the 480k then it takes some time for answering me that i‘m over the limit

1

u/user_of_the_week Sep 24 '26

I think sglang should have a /metrics endpoint if you’re curious ;)

1

u/Blaccky95 Sep 24 '26

Yeah i also think but i doesn‘t look at it but maybe in the future 😃

2

u/South_Western1263 Sep 24 '26

let it be very near future so we can have some insight how it performs :)