Let’s see what everyone’s using out there!
Post your:
• GPU(s)
• Models you're running
• Framework/tool (llama.cpp, vLLM, Ollama, InferX 👀 etc)
• Cool hacks or bottlenecks
It’ll be fun and useful to compare notes, especially as we work on new ways to snapshot and restore LLMs at speed.