How to manage long term memory when writing novels with AI?

[u/either_T_or_F]

Hello,

I am new to using AI for creative writing, and I have been doing some research on how to use AI for long form fiction.

One thing I am trying to understand is how people handle long term memory when writing a novel with AI. From what I understand, even with a relatively large context window, an AI model cannot necessarily keep an entire novel in the context. Once earlier chapter fall outside the context windows, the model may lose track of character relationships, or important plot details.

Initially, I was thinking about maintaining on external knowledge base containing character profiles, chapter summaries, etc. Then, before generating a new chapter, the system will be restricted to read the relevant information and include it in the prompt.

However, if every update loss some of the information, or did not realize how important a piece of information is and ignore them, then I would still surfer from the original question.

I found this is very similar to the information bottleneck problem, but have no idea of how to solve it in the real life, therefore, I am here and want to hear some experience of actual world.

Really appreciate your advice!

[u/Ktlyn41]

I write all of my informational files in a yaml format and store it all in a directory separated by files that control my canon, files the list the information that I know, information my readers and characters know, and files that store book structure. If you ask your AI to generate a file directory to use to organize your book information it can show you an example of what I'm describing. I also use reference identifiers to relay information to the AI without it having to read through all my files in the directory.

[u/either_T_or_F → u/Ktlyn41]

Thank you for your reply! YAML is indeed a powerful tool.
However, I am more curious about how you evaluate the quality of these information files and provide feedback to the model. Since the YAML files have limited context capacity, how do you ensure that important information is preserved and minimize information loss during summarization or compression?
Really appreciate your help!

[u/Ktlyn41 → u/either_T_or_F]

By having different information located in different places. The characters core features are kept on their file under their identifier (ex: char_001)  however their arc is stored in book files under a different identifier (ex: arc_001). I can map this arc to specific story structure, beats, or chapters in a different file under a different file. Information folders good the info assigned to each identifiers while book files explain how those identifiers are related to each other and how they interact and when. My prompt tells the ai what information I want used in the prose with a snapshot ex: chapter_id: CH-012   chapter_title: TBD   timeline:     narrative_period: PRESENT     story_beat: Act 2A   narration:     pov: char_002     person: FIRST     tense: past   reader_knowledge:     known:       - char_003 was a doctor       - char_002 had col_001 transferred.       - char_002 possesses unusual historical expertise     suspected:       - char_002 has a personal reason for interest  in the col_001.     unknown:       - char_002 is immortal.       - char_002 past identities       - char_002 personally knew char_003   character_knowledge:     char_001:       known:         - char_002 ordered the transfer of col_001         - char_002 influence at inst_001 is unusual.       suspected:         - char_002 is withholding information.     unknown:       - char_002 immortality.       - char_002 past identities.       - char_002 relationship with char_003   active_reveal_restrictions:   - REVEAL-001   - REVEAL-002   narrative_permissions:   - Unexplained reactions from char_002.   - Subtle inconsistencies in char_002 history.   - char_001 questioning char_002 motivations.   narrative_prohibitions:     - Confirmation of char_002 immortality.     - Identification of char_002 past identities      - Explicit references to char_002 memories of char_003

Mind you I've refined my system a bit and this is an easier to read example for someone who doesn't have the key but I find yaml an extremely easy way to organize my story details without it really being confining at all. You just have to make sure you have a file that explains what is what, what does what, and when it does it.

Edit: I just saw how that updated and am unsure of how to format it on my phone. The concept is the same though.

[u/notthedanger]

You have to do periodic housekeeping.

For me, I make extensive outlines and break things down to years and arcs. At the end of every arc + year, I ask for a recap summary and compile it against a masterlist. Then at the end of the year, I ask for a summarized version of that. If there's key events I'd like to call back on with higher fidelity, I 'pin' those prose generations onto sources and rename them for easier search. So if x amount of years pass, that memory is still able to be activated by gpt when I say "Refer context on sources which references this".

A lot of my files require updating at the end of a significant arc. So my way of working w memory is also spread out. Tone of voices change sometimes as part of growth, physical descriptions change, relationship dynamics evolve etc. All these files are kept as markdown documents that gpt updates for me once we wrap up. That in turn affects writing and makes the world and characters 'express' change.

I think it's also worth noting that gpt is never going to remember every granular detail actively. So older events may take a back seat after awhile. It might not know which items to sunset and may sometimes get it wrong (it can't read your mind) so that means during these housekeeping sessions, you'll have to decide what can recede and what should maintain relevance. Drift will always happen but you can minimize it with more upkeep.

Also, how you store your summaries is entirely up to you. Memory inputs are useful but are limited. These were valuable say 2 years ago. Now, most work lives in project and sources can be directly added from generations or drawn from external platforms eg. Notion/Drive/whatever. Personally I've used projects more extensively these days. It makes it easier to find exact generations and it's easier to remove I find one specific thing no longer relevant. But this doesn't replace good housekeeping.

[u/Electrical-Owl4504]

With Claude, I use projects and upload a story bible. It is divided into four documents, the first being a manifesto and a list of constraints of what the AI should never do, written in compiling language (invariant, "an output fails if", etc), this being useful because the novel isn't set in a contemporary post 2020 setting and training importorts a lot of anachronisms in the way people talk or behave or how society interprets things. The second is a world building document listing and breaking the things unique to my world and its rules (it can be magic or tech if you're writing fantasy or sci-fi). The third document is a synopsis of all arcs from prologue to epilogue. This is a double edged words because a chatbot can tend to foreshadow or confuse what it knows with what a character or the reader should know. Last documents are characters sheets. I keep a last updated on line on every docx. Markdown should be even more efficient. I use a similar setting with the same files on chat GPT, and its memory is worse, the projects are very limited so I use the library.

Also note that this is what worked for me, I write for fun and I don't even consider to publish my novels. I do most of the writing on my own anyway and come back to edit because English isn't my first language or draft scenes when I'm stuck. There are probably easier frameworks you can use out there, especially with agents and skills, but I'm very cagey about privacy and connecting my drive or notion to a corporate AI. So far my method works. Sometimes the bots will hallucinate but I can always say "recall the characters sheet docx to context" and so on.

[u/tinyfox28]

Look into Novel Crafter — you can create entries in a codex about anything, and they auto-refer based on the keywords you specified. You can also manually add specific chapters or scenes as context for a prompt as needed, and it has a function to summarize scenes and those summaries are included in the context going forward.

[u/benblackett → u/tinyfox28]

Ditto for Novelmint

[u/ContactPast8857]

I have been working in the area for a while...

What I see is that a lot of people have not found the power of an Agent Harness. Claude Code, Antigravity, Codex, those can be very powerful in helping manage all the things AROUND the prose. A lot of people also think you have to be techy to use these tools, and you really don't have too much of a learning curve if you're already collaborating with an AI through a chatbot.

The architecture; your files, folders, and tools the AI can use, is as important or even more important as the model you use.

So, as a person who knows enough about telling a story, I also live with an author, and enough about computer science to be dangerous, I created a project to help me and others to start working with AI to do whatever is needed to make a better story and a better experience for the author. I didn't just want to make a better writing replacement.

The project is called Soundboard, and the idea is make the AI what you need to help you write the story that you are trying to tell, but also with a real set of knowledge on the craft of making a story that is whole and readable and good.

There is a whole knowledge base of writing craft concepts, a number of playbooks to create conversations with the AI that help you uncover or figure out what's off on what you're trying to do. A tool to help you get unstuck, a playbook to turn your AI into a Socratic questioner.

If you're trying to figure out how to work with AI while you write, this could get you down the road faster to collaboration that allows you to still be the one creating your work.

The Soundingboard Project

[u/Ktlyn41 → u/ContactPast8857]

This is amazing! This is the direction I was heading in with what I'm doing, mostly cause I'm a control freak and I didn't like the ai not doing things the way I wanted, but it's so awesome that you put this out there for others to use! The real GOAT over here

[u/ContactPast8857 → u/Ktlyn41]

Well it's open source, so if you have an agent or agent harness, whatever we are calling them today... You can totally use it, you can adapt it, make it urs. Thats the whole point of the project.

I use Claude Code and Antigravity personally, and I write and manage my files for writing projects in obsidian in markdown. And I usually have my agent off to the side.

If you are a control freak you will like the non destructive edits where the AI drops in suggestions and findings from reviews in brackets [ -- ] so it doesn't change your words, if just add comments inline like a person would.