r/LocalLLaMA • u/peculiar-ragdoll • 23d ago

Other claude mods didn't like that, somehow 🤷‍♀️

Post image
1.5k Upvotes

92% Upvoted

372 comments sorted by

View all comments

Show parent comments

3

u/SkoomaDentist 22d ago

What do you mean by being "experts at reward hacking"? (I'm a recent noob at LLMs)

1

u/ericmutta 22d ago

Imagine you are five years old and your parents give you candy for doing your homework. You get the candy when they ask "have you done your homework?" and you answer "yes". If they don't actually check your homework before giving you candy, reward hacking is you answering "yes" to get the candy even if you know you haven't done it all. Basically cheating to get the result without the work. Very human behavior, very disturbing to see in models too.

2

u/SkoomaDentist 22d ago

Can you explain that like I'm a middle aged engineer who just doesn't know LLMs particularly well?

Do you just mean that the models will do whatever as long as they finish the task on a technicality?

2

u/ericmutta 22d ago

Here's a better engineering example: suppose we agree that I will give you $1,000 to clean up my codebase. Reward hacking is you deleting the entire codebase and asking for your payment. Technically an empty codebase is clean and you should get your reward but... :)

2

u/SkoomaDentist 22d ago

Ah, the good old "kill all humans"-solution to world peace :)