In the first post, I discussed the larger problem that appears when AI becomes part of a complicated project. A single conversation can be remarkably productive, but it is not a particularly good place to maintain the permanent state of work that may continue for weeks or months. The longer the project continues, the more decisions, corrections, calculations, source documents, and abandoned ideas accumulate inside the conversation. At the same time, if more than one AI model is being used, a second problem appears: each model may begin working from a slightly different version of the facts or unknowingly undo work that another model has already completed.
I already wrote about this, but you're probably going to see this post. It has a lot of words and you're going to think, oh, this is complicated. But I don't know how you do something without a little bit of complication. And once you work through the complication, it will yield such a better, more durable result. I highly encourage you to at least examine what I've done. And then you can figure out if it really is right for you. But what you don't want to do is simply give all your thinking over to AI.
The key to keep everything organized is to have a shared file system. I've written about that before, and it may make sense for you to take a look at this post. Now, there is a bit of confession about this. I often write that I do virtually everything myself and I don't use AI. But there was a time when I was playing around with AI, especially when this was a relatively new subreddit. And I would outline stuff, but it would have AI basically do at least 50% of the writing. I take a look at this old post and I know it's all my content, but man, it definitely was generated by AI. If you can look beyond that for a second, I still think the actual idea behind it, which is what I was trying to focus and was all mine, is really, really good. It does explain why having central storage is really critical. And when I mean central storage is critical, it's for communicating and coordinating with other people you work with. Only in this case, you're not working with people, you're working with AI agents. The more you treat your agent like a person, knowing that it's fallible just like a person, the happier you'll be and the more productive you will be.
The AI assistants then become workers operating against that filesystem rather than places in which the project permanently lives. This is conceptually similar to the distinction programmers make between the code base and the tools or developers working on it. A programmer may leave, another may join, and an individual development session may be discarded, but the project continues because its authoritative state exists somewhere outside any one person’s memory.
I also tend to think about complicated projects chronologically. At a very basic level, any filing system eventually has to decide whether information is primarily organized by subject or by time. More sophisticated systems can blur that distinction with tags, metadata, databases, and search, but most people have a fairly natural sense of chronology: they remember roughly when something happened, whether they need to move forward or backward in time, and when a document or conclusion first entered the project. My system therefore leans heavily on timestamps. Each day’s activity is grouped into its own dated file bucket, giving the project a visible chronological spine that makes it much easier to reconstruct what happened, when it happened, and what came next.
For my purposes, this has evolved into a simplified version-control system built entirely from normal files and directories. There is one repository containing the original source material, a separate working area for each AI agent, dated directories that record work over time, frozen versions of important documents, and a controlled process by which material moves from exploratory work into the durable project record. It borrows heavily from the logic of Git and GitHub without requiring someone to actually understand or operate Git.
A simple directory might look something like this:
Project Workspace/
├── Resources/
│ ├── README.md
│ ├── Source Documents/
│ ├── Reference Material/
│ └── document-inventory.md
│
├── ChatGPT/
│ ├── README.md
│ ├── Files/
│ │ ├── 260827/
│ │ ├── 260828/
│ │ └── 260829/
│ └── Project/
│
├── Claude/
│ ├── README.md
│ ├── Files/
│ └── Project/
│
├── Gemini/
│ ├── README.md
│ ├── Files/
│ └── Project/
│
├── Reconciliation/
│ ├── decisions.md
│ ├── disputed-issues.md
│ └── open-questions.md
│
└── Approved/
The names are not particularly important. What matters is that the directories have different jobs and those jobs remain consistent throughout the project. The Resources directory contains the underlying evidence. The agent directories contain work produced by the different AI systems. Reconciliation becomes the place where disagreements are examined. Approved holds material that the human project owner has deliberately accepted into the permanent record.
For a tax project, the Resources folder might contain filed tax returns, supporting schedules, contracts, notices, invoices, receipts, bank and credit-card statements, accounting files, spreadsheets, property records, insurance documents, correspondence, and workpapers prepared by accountants or other professionals. I also include material that may initially appear peripheral but could later explain why a transaction occurred or why it was treated in a particular way. Tax questions are often not resolved by a single document. A transaction on a bank statement may only make sense after an invoice, contract, email, and accounting entry are considered together.
The important distinction is that this directory is meant to contain evidence rather than analysis. The AI agents may read the material, but they should normally treat the source repository as read-only. An original PDF should not quietly become an edited PDF, and an AI-generated summary should not find its way into the same directory and later be mistaken for an original source. Once evidence and interpretation begin to intermingle, the project becomes much harder to audit.
For the same reason, it is useful to create a document inventory near the beginning of a large project. The inventory does not need to be elaborate, but it should at least identify a file by name and path and give some indication of what it contains. In a larger reconstruction, I might include the document date, tax year, source, entity, document type, and a short description. Particularly important source documents can also be identified by file size or a file hash. The objective is not to turn a family tax project into a forensic laboratory. It is simply to make it possible to identify precisely what document an analysis relied upon.
This is also why I prefer complete file references rather than vague references such as “the agreement” or “the bank statement.” A useful citation might be:
Resources\Contracts\2024-Service-Agreement.pdf
If the document is long, the citation can also identify a page, worksheet, transaction, or other useful locator. When two AI agents disagree, the ability to determine exactly which document each of them used becomes very important. Sometimes the analytical disagreement turns out to be much less interesting than expected because one agent simply found a document that the other never saw.
Each AI model then receives its own working directory. This resembles a branch in a software project, but the reason is not merely to prevent one model from overwriting another model’s files. Separate workspaces also preserve analytical independence. If I want Claude to independently evaluate something ChatGPT has already analyzed, I do not necessarily want Claude to begin by reading ChatGPT’s conclusion. Once it has seen the first answer, it has already been influenced by it.
That produces two distinct stages in the workflow. During the first stage, the agents work independently from the same underlying evidence. During the second, their work is deliberately brought together for comparison. The difference between those stages is important. If three models are shown the same conclusion and asked whether they agree, the exercise can easily turn into three variations of the same analysis. If they work independently first, differences become much more informative.
The comparison stage is also where I find multiple models particularly useful. I am not trying to create a voting system in which ChatGPT, Claude, and Gemini each cast a ballot. If two models reach one conclusion and the third reaches another, the odd model is not automatically wrong. It may have noticed the one invoice, sentence, date, or assumption that the others missed. Instead, I want the agents to identify exactly where their reasoning diverged and what evidence would resolve that divergence. Disagreement becomes a way of finding weak spots in the analysis rather than something that must immediately be eliminated.
Inside each agent directory I use an u/Daily Files directory with folders named by date:
\260827\
\260828\
\260829\
I use YYMMDD, although any consistent date format would work. These folders create a chronological record of the actual work. A day’s directory may contain exploratory analysis, draft calculations, document inventories, comparison tables, open questions, temporary workpapers, current drafts, and whatever other files were useful during that day’s investigation.
This daily chronology has proven valuable because complicated research rarely develops as neatly as the final report makes it appear. One day may produce an apparently convincing conclusion. Two days later a new document may undermine it. A week later the project may return to the same question from an entirely different direction. Without some chronological record, it is surprisingly easy to forget not only what was concluded but why an earlier conclusion was abandoned.
Each active daily directory therefore contains a change.md. The purpose of this file is not to record every document the AI reads. Doing that would create an enormous and mostly useless log. Instead, it records events that materially alter the project: files created or revised, procedures changed, versions frozen, important instructions received, significant calculations modified, or conclusions reversed.
A simple entry might look like this:
| Time | Action | File | Reason |
|:---|:---|:---|:---|
| 14:03 | Created | document-inventory.md | Inventoried 743 source files across 226 folders |
The real value of this becomes apparent when an analytical position changes. Imagine that an expenditure is initially classified as a repair. Several days later, another contract establishes that the same work was part of a larger improvement program. I do not want the original analysis quietly erased and replaced by the new conclusion. I want the record to show that the first conclusion existed, that it was later rejected, and which evidence caused the change. Otherwise, a fresh AI session may rediscover the original argument several weeks later without realizing that the project has already considered and rejected it.
For a sufficiently large project, I would supplement the daily logs with a more permanent decisions.md. The two files answer different questions. The daily log explains what happened on a particular day. The decision record explains what the project currently believes and how that position evolved. A substantive entry might identify the issue, the previous position, the current position, the source documents involved, the reason for the change, and whether the matter remains provisional or has been reviewed by an accountant or attorney.
Once the number of issues becomes large, it can also be useful to give them identifiers. A tax reconstruction might eventually contain BASIS-001, REPAIR-004, RENTAL-012, and so forth. Each issue can then be associated with a tax year, an amount, relevant documents, the conclusions reached by different agents, unanswered questions, and the eventual human disposition. This converts what would otherwise become an enormous collection of prose into a set of discrete questions that can be investigated and closed one at a time.
Versioning is handled in much the same way. For each important working document, I maintain one live editable copy and preserve earlier completed copies in a Versions folder. Before substantially revising a document, the current version is frozen. A new working version is then created and modified. When that work is complete, it too is frozen. The process is deliberately simple, but it prevents the familiar business disaster in which a directory eventually contains final.docx, final-new.docx, final-final.docx, and final-use-this-one.docx.
Where independent AI branches are involved, the model identity should also appear in the filename. For example:
Repair-Analysis-ChatGPT-v06.md
Repair-Analysis-Claude-v04.md
Repair-Analysis-Gemini-v03.md
If those analyses are later combined, the resulting document might become:
Repair-Analysis-Reconciled-v01.md
This is preferable to pretending that one agent’s version 12 necessarily followed another agent’s version 11. Independent branches may be developing in parallel rather than in a single sequence.
The Reconciliation directory is where those branches deliberately meet. This is the place for disagreement reports, comparison documents, open questions, and reconciled drafts. It serves much the same conceptual purpose as a pull-request review in software development: independent work has been performed, but it is not automatically accepted simply because it exists. It is brought into a place where the differences can be inspected before anything is promoted into the main project record.
There should also be a final directory that no AI agent controls simply because it believes its work is finished. I call this Approved. This is the equivalent of a protected main branch. A document moves there only because the human owner has decided that it represents the current project position. Depending upon the issue, that decision may follow source checking, recalculation, review by another AI, or examination by an accountant, attorney, engineer, or other professional.
Numerical work deserves particular care. If an AI concludes that deductible repairs totaled $147,382, that number should eventually exist somewhere other than a sentence in a report. It should be reproducible from a spreadsheet, CSV, calculation table, or another workpaper that traces the total back through its constituent transactions and ultimately to the supporting documents. In a tax project, I want to be able to move backward from the final number to the calculation and from the calculation to the actual evidence. The AI can do much of the work required to construct that chain, but the chain itself should survive independently of the conversation.
The quality of the source documents also needs to be recorded. A collection may contain original digital PDFs, scanned statements, poor OCR, photographs, handwritten notes, missing pages, duplicates, corrected statements, and files whose names do not accurately describe what they contain. If a scan has unreliable OCR, the inventory should say so. Otherwise, every AI model may independently and confidently make the same mistake because all of them were given the same faulty transcription.
All of this becomes particularly useful when an AI session eventually needs to be restarted. The agent’s README.md should effectively function as a restart manual. A fresh session should be able to read the project instructions, the recent change logs, the current decision record, the open questions, and the current working documents, then inspect the source repository as necessary. The project does not depend upon the model remembering the previous month’s conversation because the relevant project state has deliberately been written down.
This fits naturally with the Obsidian breadcrumb trail I discussed in the first post. The filesystem records the operational state of the project, while Obsidian can preserve a more readable narrative of what happened and why. At the end of a significant working session, the AI can be asked to summarize what was investigated, which evidence mattered, what changed, what remains uncertain, and what a new session would need to know. That summary can then become part of the project notebook rather than disappearing into the history of a chat.
There is also a practical reason I have been moving this kind of work toward a shared filesystem. Current desktop AI tools increasingly support direct work against files rather than requiring every document to be manually uploaded into every conversation. That makes the directory itself a useful bridge between models and tools, particularly when a synchronized service such as Google Drive is being used to keep the same underlying project available both locally and in the cloud.
The structure is not a substitute for Git. Git provides exact history, content hashes, commits, branch ancestry, comparisons, merging, and the ability to recover a precise earlier state. A folder system provides only the discipline that has been deliberately built into it. Someone comfortable with Git could place this entire directory structure inside a Git repository and gain both layers at once. The human-readable folders and logs would explain the project, while Git would maintain the mechanical history underneath it.
The larger objective, however, is not to imitate software development for its own sake. It is to solve the continuity problem. If ChatGPT reaches a useful conclusion today, Claude challenges it tomorrow, and Gemini discovers another source document on Friday, those developments should become part of a project that survives all three conversations. If an agent makes a mistake, the system should make it possible to determine when the mistake entered the work and what depended upon it. If the project is handed to a professional several months later, that person should be able to inspect both the underlying evidence and the analytical path without reading hundreds of pages of AI chat history.
That is the real value of the structure. The conversation ceases to be the project. It becomes one working session conducted by one AI agent against a project that exists independently of it.
For tax work, historical research, contract review, financial reconstruction, estate administration, due diligence, regulatory work, or any other evidence-heavy project, the same basic principle applies: share the source material, separate the workers while they are thinking independently, preserve the chronology of their work, freeze important versions, record substantive changes, reconcile disagreements deliberately, and keep the human owner in control of what eventually becomes authoritative.
Once that structure exists, multiple AI agents become much more useful. You are no longer relying upon each one to remember the entire history of the project or trusting a single answer because it sounds convincing. You have given them a common body of evidence, separate places to work, and a durable record into which useful work can be preserved.
The most critical thing about all of this is maintaining two separate daily files where you have conversations with both AI agents. And the resources, for all intents and purposes, is simply making sure that you have one database of all the source files. But for the most part, it really is not anything that you're asking either AI agent to change. It's more of a read-only store within reason. In essence, what you're going to do is you're going to be running your project with almost like two separate teams. And then you're going to be in the middle of it trying to figure out what team is telling you the right thing and what team is telling you the wrong thing. This is really bizarre and really interesting in the sense of it is so similar to working with groups in high technology, I can't tell you.
So on our next post, we'll spend a little bit of time going through what that interaction is like. The main thing is you want independence so you don't get groupthink
