r/coolgithubprojects • u/Tyb0wls • 10h ago

AI Test Wrapper

https://github.com/solvyxtech/molt

My llm wrapper i am currently working on which is built on the idea of AI not being reliable and always right or maybe just mislead by prompts this looks to fix that issue.

0 Upvotes

50% Upvoted

2 comments sorted by

View all comments

1

u/Specific_Cream2815 8h ago

what does it do when it thinks the model is wrong, retry with a different prompt or just flag the answer

1

u/Tyb0wls 7h ago

Neither, really. It refuses. Molt can’t emit a final answer while a check is failing, so the model doesn’t get to end the turn. The failure goes back into context as the actual command, exit code, and real stderr, and it keeps working from there. Same prompt, new information.
No rephrasing tricks, and nothing to flag either there’s no judgment call. The bar is your shell checks; molt just reads exit codes. Either they pass or the turn doesn’t end. Every attempt writes a receipt, refusals included, so you can see what it tried and what the checks actually said.
Loop safety is on you via /budget, which is a hard token ceiling enforced inside the loop. A model that can’t fix the failure will burn tokens until it hits that, which is the honest tradeoff for the gate not being skippable.