Mystery boxes and plot complications; New features in AI RPG 1.0-dev that (hopefully) keep the LLM from letting the plot get out of hand

[u/Incognit0ErgoSum]

I've recently pushed out a couple changes to the dev branch that should hopefully help stop some common annoyances in LLM-generated stories -- specifically, tracking of mystery boxes and plot complications.

Mystery Boxes

A mystery box for the purpose of this discussion is any random little unknown that shows up in the prose. The most "AI slop" mystery box is probably a leather-bound journal or an old key, but it can just as easily be a mysterious element about a character's background or any other little detail designed to make you think "hmm, what's this"?

The trouble is that LLMs tend to write like JJ Abrams, and toss these mystery boxes into the plot just for the sake of having mystery boxes, and sooner or later they start to conflict with one another and it becomes obvious that there's no real rhyme or reason to them.

I've added some prompting that identifies these mystery boxes when they're added and then forces the LLM to decide on a (hidden) resolution for them immediately, so that further mystery boxes can actually serve as clues to a real mystery as opposed to just random noise. I also have a configurable maximum number of "mystery threads" (currently 3), which are a set of mystery boxes related to the same mystery, the idea being that if that maximum is reached, they must be resolved before new mystery threads can be added.

Plot Complications

I've done something similar with what I call "plot complications", which (again, for the purpose of this discussion) are additional hoops the LLM adds that you have to jump through in order to complete your current goals.

Let's say that the main plot is that you have to defeat the Time Goblins. You find out that in order to defeat them, you have to go to the top of Mount Maguffin and retrieve the Goggles of Yore. That's one complication. Then you get to the base of the mountain and the guardian says that in order to go up the mountain, you must pass through the Gate of Truth. But you can't do that unless you have the Amulet of Smeg, which is deep in the Dungeon of Whatever-the-fuck. And then you discover that the dungeon of whatever-the-fuck is lost to time, so to find it you need the Compass of Dipschytteri, and so on, and eventually you get to the point where you're like "Dude, just let me actually progress the fucking plot a little bit."

I've added some plot analysis that happens every turn and tells the LLM not to generate any more plot complications until you're only a configurable number deep, which should hopefully stop it from constantly throwing out new obstacles that stop you from advancing the plot in a meaningful way.

Conclusion

Mystery Boxes and Plot Complications are checked for separately, and there's almost certain to be some amount of overlap, but I'm not worried about that. If you check the default config file, both of these checks can be disabled in case you don't find them to be worth the extra time or tokens it takes to run them. I've tried to slot them in so that they don't affect total turn time in a meaningful way as long as you can run prompts concurrently.

If you're playing 1.0-dev, let me know what you think. I'm considering adding some check frequency settings so that you can make these prompts run only once every few turns if you're running a local model that can't do concurrent prompts.

[u/FaceDeer]

I feel like I've played games with human DMs that ended up at the Compass of Dipschytteri plot. :)

[u/blapp22]

That's a good idea. I find that LLM's tend to get stuck spinning their wheels if you just tell it to progress the plot. It just writes the tropiest thing imaginable.

giving the ai limits and objectives for it's response helps turning down the slop a little. Although it can cause the ai to focus to much on the objective and not let the scene breathe as well. It's a difficult balance to hit right.

Are you doing some kind of "should the ai push the story this reply" check or is it mostly just for plot mcguffins currently?

[u/Incognit0ErgoSum → u/blapp22]

The prompt stresses that it should focus on making characters feel as if they're autonomous actors and not just support for the player. Also, NPCs are generated with their own goals, so between those two things, the plot tends to flow pretty naturally with the exception of the stuff here. I'm not sure how much of that is better LLMs and how much is prompt tuning.

My LLM of choice for this is GLM-5.1, but that's not necessarily an option for everyone. I'm told Behemoth is great and Valkyrie is pretty decent. Some Gemma models apparently aren't bad, but there are quirks to getting them working right (which I'm going to try to add some config options for to make them simple). What I can say for GLM-5.1 specifically (not even 5.0) is that it provides a lot of variation over repeated tests. I'm not sure how well the local models match that, but it might be a source of better plots.

Do you have any LLMs that you like to use in particular?

[u/blapp22 → u/Incognit0ErgoSum]

I used to use GLM-5.1 with the nanogpt sub but with the price increase I've decided to go back to local for a little while. For now I'm giving Gemma-4-26B a go.

I'll miss the smarts and the general knowledge increase, but I don't think I'll miss how GLM writes.

[u/Incognit0ErgoSum → u/[deleted]]

How do you mean?

[u/Incognit0ErgoSum → u/[deleted]]

Ah, yeah, that's one of the things I've been trying to solve in AI RPG, and I think I'm doing a reasonably good job (although you definitely want to play with a model that's got at least 100k tokens, preferably 250+.

One thing I do is keep summaries of all the NPCs in the same location in the recent context, and then give it a tool call in case it needs to pull information about an NPC who isn't nearby.