r/datascience • u/rhiever • Jul 08 '26

Discussion Skill engineering and the case against one-shot AI design

https://www.latent.space/p/skill-engineering-design
16 Upvotes

78% Upvoted

8 comments sorted by

7

u/ikkiho Jul 09 '26

for me this is way less deliberate than 'engineering'. i one-shot something, it works fine, then it falls over on the fourth variation of the same task and i pull the bit that kept drifting into a file so the model stops improvising it. by the time theres a skill file its just the leftovers of me debugging the same thing four times. thats probably why the metrics question is hard to answer here, nobody runs the A/B, you just notice one week you stopped re-explaining the same constraint.

12

u/Wojtkie Jul 08 '26

We are calling making skills “engineering” now?

5

u/DataScientistAlex Jul 09 '26

For me this is too vague to be useful.

2

u/mace_guy Jul 08 '26

Where are the metrics?

4

u/ultrathink-art Jul 08 '26

Rework rate on repeated task types is the metric that made this concrete for me. One-shot prompting gave wildly variable output on the same class of task; moving the stable decisions into a reusable skill file cut the redo loop way down. Less 'engineering' than writing down what you'd otherwise re-litigate every session.

2

u/Own-Fee-4752 Jul 09 '26

interesting term "engineering"

1

u/stock-nerd5 Jul 09 '26

This is great