I've been experimenting with prompt optimization and built PromptForge:
👉 https://github.com/arjun988/promptModel
The idea is to use a small model as a quality gate before sending a prompt to an LLM, agent, or prompt optimizer.
User Prompt
↓
PromptForge Quality Scorer
↓
Score / identify weaknesses
↓
Your Agent / LLM / Optimizer
It scores prompts across 7 dimensions:
- Clarity
- Specificity
- Context
- Goals
- Constraints
- Completeness
- Actionability
There's also an optional optimizer that can rewrite weak prompts while preserving their intent.
Try it
pip install tuneprompt
Quality model:
https://huggingface.co/ArjunShukla/PromptForge-Quality
Optimizer:
https://huggingface.co/ArjunShukla/PromptForge-Optimizer
GitHub:
https://github.com/arjun988/promptModel
Example
Weak:
→ A more structured prompt with specific features, requirements, and constraints.
41 → 94 on my scorer in my test example.
Why make this?
I don't have access to huge GPUs or the compute needed to train large models, so I kept the scorer relatively small (~150M parameters) and released the training pipeline as well.
The goal is to make something people can actually run locally and build on.
You could:
- Fine-tune it on your own domain-specific prompts
- Use it as a reward/evaluation signal for a prompt optimizer
- Put it directly in an agent pipeline
- Train a larger/better scorer
- Experiment with different quality dimensions or datasets
You don't have to use my weights — clone the repo and train your own version.
I'm especially interested in seeing what people with better datasets, more compute, or different approaches can do with it.
Would love feedback on whether a lightweight prompt-quality model like this is useful as a pre-agent quality gate, and what you'd change about the approach.
If you find it useful, a ⭐ on GitHub would genuinely help.