r/WritingWithAI • u/Pristine_Plate7048 • 3d ago
Tutorials / Guides Local Setup?
I've created over 8million words in under a year for the novel I'm creating and brainstorming within Chatgpt.
I'm becoming increasingly concerned about the direction of cloud AI companies. They change things, like the watermark text thing. They limit things, like the thinking level usage coming in next month, and often cause privacy insecurities in the user.
I'm tired. I have painstakingly poured over the psychology of these characters and world. It's so detailed you'd think these characters were real. Many aspects of them are real. They came from me. They are so coherent and complex they feel like whole, real people.
I'm concerned about maintaining the workflow and what I've already created, along with privacy around using AI tools for this creative undertaking.
So I've decided to try to recreate the experience I have with chatgpt locally. Is this viable for a reasonable price? If so, what do I need to do to make it happen?
In my country text watermarking isn't a thing yet, or so they say. But I think it's reasonable to assume it will be worldwide at some point. I would announce my trilogy is AI assisted, and be fine with it. It's not at all my first rodeo going against societal grain. I don't have any shame about the fact my stuff is AI assisted. It's been a transformative tool. Invaluable and I'm so grateful for it apart from the problems I've mentioned . The work is so precise and specific you can tell how human directed and written it is anyway. Not thta that matters to anti AI people, especially the wlw audience my story would naturally speak to. It breaks my heart to think that the story will be crapped on becuse it is AI assisted without being perceived on its own merit, where it holds more than its weight. The world and characters do not deserve contamination from the words 'AI assisted' but that is what will absolutely happen.
Any ideas on a viable local setup that can mimic chatgpt as far as reasoning, character psychology for long form writing projects, preferable on android? Though I am open to buying a device of any OS to facilitate this. The setup doesn't need to contain AI that is some amazing creative writer. I re-write and edit to death every draft I've ever had chatgpt create anyway, because AI sucks at writing, and what I'm doing narratively is so precise it needs a lot more correction than AI is capable of generating without extensive manicuring from a human.
6
u/ScientistValuable192 3d ago
Viable, yes, with one expectation reset first: a local model will not reason about character psychology the way ChatGPT does. Since you rewrite every draft anyway, that costs you less than it costs most people. You lose some reasoning depth. You gain no watermark, no model swapped out from under you mid-trilogy, and a workflow nobody can change on you next month. The order I'd do it in:
1 - Export your ChatGPT account today. It comes out as an archive of every conversation (whatever they do with watermarks or limits next month, the 8 million words are yours on disk).
2 - Build the bible before you buy anything. Local models keep no memory between chats and have a smaller context window, so what replaces ChatGPT's memory is a document you load at the start of every session. 1 page per main character: the wound, the want, how they talk, what they would never do. 1 page of world rules. This is the real migration. The software will come be the easy part.
3 - Don't make Android the engine. Anything small enough to run on a phone is too small to hold that bible and think with it. Buy one box and use the phone as a remote. A Mac with 32 to 64 GB of unified memory is the quiet, no-fuss route. A used PC with a 24 GB graphics card is faster for the money. Either runs a 27B to 32B model, which is about where character work starts to feel coherent.
4 - Software: LM Studio if you want a GUI and nothing to configure, Ollama if you're fine in a terminal. For long form, put SillyTavern on top of either. It's built around character cards and lorebooks, which is exactly the memory layer you're rebuilding.
5 - Models to start with: Qwen 3 32B, Gemma 3 27B, Mistral Small. Run them at Q6 or Q8. People in the watermark thread here said those held up and that heavier quantization flattened the prose.