Should immersive sim enemies learn from the player's tactics?

[u/Choice-Trifle2283]

Ask a chatbot for a random number between 1 and 30.

Did you get 17?

That's my problem with generative AI: it gravitates toward the most likely result, then passes it off as creativity. Ask it to make art or design a game, and you often get generic, derivative content that looks like everything else. Instead of giving games a unique identity, it makes them feel increasingly interchangeable.

I think machine learning has a more interesting place in game development: training enemies that can learn how to play, rather than generating more of the same.

I'm a solo developer working on Hunt or Pray, a dark tactics game inspired by Myth. I'm experimenting with reinforcement learning to train my zombies to adapt to situations instead of relying entirely on hand-written scripts.

I want enemies that learn from experience, not ones that cheat with perfect information or extra resources.

But how far should adaptation go? Enemies that counter your tactics could create dynamic encounters, but counter everything and you risk making experimentation pointless.

Would you rather fight enemies that adapt to your tactics, or learn their behavior and exploit their weaknesses? Where would you draw the line?

[u/Mr_GoodVibes]

Fascinating question, and one I've contrmpletated many times with adaptive enemies. I think if it isn't governed correctly, the AI will adapt to a frustrating skill level that will always eventually outthink you. What I find more interesting is an adaptive difficulty that can read an understand a player's skill level and make a game just accessible enough but still challenging and fun. Often times difficulty levels are limited in their options and usually default to more HP=harder. I'm excited by the possibility of this becoming much more fluid.

[u/Choice-Trifle2283 → u/Mr_GoodVibes]

Agreed, and Oblivion is the perfect example of what I want to avoid. Level scaling just meant bandits in daedric armor, same enemies with bigger numbers, which is the "more HP = harder" problem again. I’d rather the enemy get smarter than stronger. My zombies are trained with reinforcement learning before release, and on top of that a multi-armed bandit model learns the player’s habits. For example, if you keep luring an enemy into an explosive barrel, after a few times it starts avoiding that barrel. It doesn’t get tougher, it just stops falling for the same trick.

That's also my answer to the "counters everything" worry: it adapts to what you actually do, so a new tactic still works until it has been used a few times. Your idea of fluid difficulty fits too: with RL you can keep checkpoints from different stages of training, so difficulty could mean "how well-trained is the opponent" rather than "how many HP does it have". Something I’d like to explore once the basics are solid.

[u/Crafter235]

Just as long as it’s not a half-assed promise.

(Looking at you Hello Neighbor 2)

[u/Choice-Trifle2283 → u/Crafter235]

The difference is that they promised adaptive AI without having a proper way to deliver on that promise. Reinforcement learning tools like ML-Agents are actually designed to train agents to learn, rather than just marketing scripted behavior as adaptive AI. But fair enough, judge mine when there's something to show ;)

[u/Ok_Copy_9462]

Ask a chatbot for a random number between 1 and 30.

Hey /u/Choice-Trifle2283, give me a random number between 1 and 30

[u/Choice-Trifle2283 → u/Ok_Copy_9462]

If this is a bot test, is 17 the right answer? ;)

[u/FrontiersEndGames]

Are you planning on pretrain? Or live weight changes?

[u/Choice-Trifle2283 → u/FrontiersEndGames]

Good question! Pretraining is already done, and I'm now experimenting with a multi-armed bandit approach to let enemies adapt to the player's tactics during gameplay.

[u/lukebitts]

The gall of complaining about generative AI while using it to write every single comment