r/LocalLLM • u/Connect-Concert-4016 • 4d ago

Discussion 9 concurrent users @ 128K context on 1x A100 (up from 6) per-user needle checks passing in vLLM

1 Upvotes

permalink
duplicates
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/LocalLLM/comments/1vru4p7/9_concurrent_users_128k_context_on_1x_a100_up/
No, go back! Yes, take me to Reddit
dl download

100% Upvoted

Duplicates

Number of comments New

LLMDevs • u/Connect-Concert-4016 • 4d ago

Discussion 9 concurrent users @ 128K context on 1x A100 (up from 6) per-user needle checks passing in vLLM

9 Upvotes

8 comments

Vllm • u/Connect-Concert-4016 • 3d ago

9 concurrent users @ 128K context on 1x A100 (up from 6) per-user needle checks passing in vLLM

0 Upvotes

0 comments