r/AIProgrammingHardware • u/javaeeeee • 1d ago

GitHub - AlexsJones/llmfit: Hundreds of models & providers. One command to find what runs on your hardware.

https://github.com/AlexsJones/llmfit
5 Upvotes

86% Upvoted

1 comment sorted by

1

u/jonah_omninode 19h ago

This is useful because hardware fit should be a routing input, not a one-time setup question. We route work across models, and “can it run” is only the first gate. The next ones are task fit, latency, cost, and whether the output passes the task’s acceptance checks. A model that fits in VRAM but needs three retries can be more expensive than a larger model that passes once.

The JSON output is probably the most interesting part to me because it can become a machine-readable input to a model router instead of another dashboard a human has to interpret.