The world of open-source AI is a victim of its own success. HuggingFace, the central hub for models, is a firehose of innovation, with dozens of new language models appearing daily. This creates a paralyzing problem for developers and hobbyists: what should I actually run on my machine? Picking the biggest model your GPU VRAM can hold is a rookie mistake, often leaving you with a slow, outdated, or simply lower-quality model. whichllm is a command-line tool built to solve this exact problem, and it does so with an intelligence and utility that makes it an indispensable part of the modern AI toolkit.
The Smart Recommender You Need
The core conceit of whichllm is simple but powerful: recommendations should be based on evidence, not just size. Where other tools or manual methods might just show you a list of models that fit in your 24GB of VRAM, whichllm goes several steps further. It synthesizes data from a wide array of real-world benchmarks, including LiveBench, Chatbot Arena ELO, and the Open LLM Leaderboard, to create a composite quality score. As the README points out, this is why a newer, benchmark-proven 27B parameter model might rightly be ranked higher than an older 32B model that also fits—a nuance that makes all the difference.
This evidence-based approach is fortified with several layers of intelligence. The system is "recency-aware," meaning it demotes scores from stale leaderboards to prevent a model from 2024 from outranking a current-generation model on an outdated test. It also grades the evidence itself, discounting scores that are self-reported by uploaders or improperly inherited from a larger base model. This thoughtful, almost journalistic approach to data quality is what separates whichllm from a simple hardware filter. It gives you a recommendation you can actually trust.
More Than Just a List
While its primary function is to rank models for your current hardware, whichllm shines as a comprehensive planning tool. The GPU simulation feature is a killer app for anyone budgeting a new PC build or comparing cloud instances. By running whichllm --gpu "RTX 4090", you can see exactly what your experience would be like before spending a dime. The tool even supports multi-GPU setups (--gpu "2x RTX 4090") and a dedicated upgrade command to compare your current hardware against a list of potential candidates.
This forward-looking capability extends to model-first planning. The whichllm plan "llama 3 70b" command flips the script, telling you what kind of hardware you'd need to run a specific model you have in mind. These features elevate whichllm from a neat script to a strategic utility for serious practitioners. It addresses the entire lifecycle of the user's decision-making process, from initial exploration to hardware acquisition.
From Recommendation to Conversation in Seconds
Perhaps the most practical feature is the one that closes the loop between recommendation and execution. The whichllm run and whichllm snippet commands are brilliant additions. After finding the perfect model, you can immediately start a chat session with it by name. whichllm handles the heavy lifting in the background, using uv to create an isolated environment, download the model, and launch an interactive session. There's no manual pip install, no searching for the right GGUF file on HuggingFace—it just works.
For developers, whichllm snippet "qwen 7b" is just as valuable, printing copy-pasteable Python code to load and run the model. This drastically reduces the friction of trying something new, encouraging experimentation and rapid prototyping. It understands that the user's ultimate goal isn't a list of names, but to use a model.
Where It Fits (and Where It Doesn't)
It's important to understand that whichllm is a focused tool, not a kitchen-sink platform. It is not a replacement for a model manager and server like Ollama, nor is it a GUI-based application like LM Studio or Jan. Its command-line nature is a feature for its target audience of developers and power users who value scriptability (--json output is a key feature) and efficiency. Users who want a point-and-click experience will be better served by the alternatives.
However, whichllm is the perfect companion to those tools. You can use whichllm to decide which model is best, then pull it into Ollama to serve it persistently. Its strength lies in its specialization. It does one thing—providing the best possible answer to "what model should I run?"—and it does it better than anyone else. With over 6,000 stars, a release just last month, and a remarkably low 21 open issues, the project is clearly healthy, actively maintained, and resonating with the community. It's a mature, reliable tool you can confidently integrate into your workflow today.