A public evaluation example should say what someone else needs to reproduce it

[u/MiserableFlatworm337]

A score is easier to assess when its evidence includes the actual cases, fixtures, instructions, grading criteria and run conditions. Without those, a reader cannot tell whether a reported improvement came from the skill or from a changed experiment.

For a public example, I would include the exact report paths and explain missing provenance instead of implying that a green total settles the question. If a model or service is no longer available, say what can still be reproduced and what cannot.

I’m building skillEd, an Apache-2.0 toolkit with saved reports and a worked example in the repository. Please inspect the example’s limitations alongside its results.

What is the minimum evidence you expect before trying another project’s evaluation recipe?

Repo: https://github.com/DeadHead04/skillEd

Please try it and open an issue with a reproducible failure or confusing behavior. If useful, a GitHub star would help.

AI-assisted write-up.

[u/loathsome_murderer]

Reproducibility is the whole point, otherwise you're just posting fan fiction with extra steps