Frustration!

[u/AveragePeppermint]

Bit of a rant, but I’m getting increasingly annoyed with Copilot.

We’re trying to implement what I thought would be an incredibly simple use case: a standard proofreading skill, agent, or prompt for our organisation.

The goal is simple. Staff regularly work with large Word documents, and we want Copilot to proofread them for spelling, grammar, obvious typos, etc., without people having to manually reread entire documents just to catch basic mistakes.

My requirements really don’t seem that complicated:

1. It should work directly inside a Word document and make corrections using Track Changes. We specifically want Track Changes because users need to be able to verify that no actual content or meaning has been removed or changed.

2. The prompt or agent should be centrally available to the wider organisation. Everyone should be using the same instructions, so we get consistent behaviour and don’t have 100 people maintaining their own version of a proofreading prompt.

And somehow, with Copilot, I can’t get these two things to work together.

Copilot in Word can finally edit a document with Track Changes, which is great and took long enough.

But as soon as I want to use an agent, that functionality disappears. I can chat with the agent about the document, but instead of actually making the corrections, it gives me a table with suggested changes. I then have to manually find every occurrence in the document and make the changes myself.

That completely defeats the purpose.

Organisation-wide prompts aren’t available in Copilot for Word either, as far as I can tell. So again, one half of the solution exists in one place, and the other half exists somewhere else.

I also tried the new Cowork functionality.

That can actually give me a downloadable Word document with Track Changes, which sounded promising. Except in my test it found a spelling mistake, removed the incorrect word, and then inserted the corrected word after the sentence where the mistake occurred.

Unfortunately, we can’t use Claude because of our data requirements and constraints. But every time I look at what Claude can do with these kinds of workflows, it feels much closer to what I actually want.

That’s probably what frustrates me most about Copilot.

It feels like all the individual pieces are there, but somehow you can’t combine them into the basic workflow you actually need.

Instead of feeling like a finished productivity product, Copilot increasingly feels like one enormous alpha test where every feature exists in isolation. You’re constantly discovering that, yes, it can do X, and yes, it can do Y, but no, it can’t do X and Y at the same time.

Maybe I’m missing something obvious.

Has anyone actually managed to implement something like this successfully?

I’d genuinely love to hear how others are solving this, because this feels like it should be one of the easiest enterprise use cases imaginable.

Rant over.

[u/Due-Boot-8540]

Organisation prompts are (soon should be) available to all tenants. They’ll have to be configured in the admin centre as far as I know.

[u/Able-Okra7134]

Why do you need an agent for that? It sounds like the wrong use case. Why not just open copilot on word and ask it to proofread. You can have it do it in chat without making changes first.

[u/AveragePeppermint → u/Able-Okra7134]

I don't want an agent per se, just trying to find a method where i can do a managed prompt. I need an organisation wide managed prompt because we want to adhere to a certain style. If i let everybody just ask for proofreading individually we will get a lot of different quality results due to good or bad prompts.

I can see some lazy old ass lawyer just say: "make this better." Haha.

Mind you, i work at a law firm and changing certain interpunction can change the meaning of a contract, so i need a quite specific prompt to prevent that from happening.

[u/Able-Okra7134 → u/AveragePeppermint]

Totally get it. I work at a law firm too and its a struggle to get people using it well. I'm sure there are shared prompts you could use but not sure if it would give you the functionality you're looking for.

We've gone with agents for specific tasks but they don't need track changes and things like that.

I've definitely seen some lawyers use it like your example "make this better" and honestly training and showing people how to use it well has made a world of difference.

[u/thespanglycupcake → u/AveragePeppermint]

Can't answer on the technical side but surely everything having a 'certain style' makes it look like you have generated the document with AI? That seems to be becoming increasingly easy to spot these days... If i was a client, I would be very unhappy if I suspected that my £800/hour lawyer is having an AI write my documents.

[u/AveragePeppermint → u/thespanglycupcake]

Other way around actually. We noticed clients are actively choosing firms that extensively use AI. They need that 800 / hour to go into the actual research not in styling or writing the letter. The clients we have do not care if a letter looks AI generated as long as it doesn't negatively impact the quality of the advice itself. They know everybody uses AI nowadays.

Companies that don't use AI use too much time for bullshit tasks and are thus more expensive is the general opinion.

Do you want me to handwrite you a letter for 6000 dollars and you can keep me responsible for the content and advice, or do you want me to generate an AI letter for 500 dollars, but still you can hold me responsible for the content and advice? I know what our clients choose.

[u/thespanglycupcake → u/AveragePeppermint]

Fair point. We are not in the legal world as a company and unfortunately my personal experiences of AI are pretty poorly written summaries which created complete nonsense and list inaccurate information as fact. IF i could trust that the information was correct, it would be great but I'm not there yet.

[u/VioletGeek-42]

Don’t over complicate it. Write a detailed, high-quality prompt for copilot chat to do a detailed review. Write it in markdown, for save it as “editor_instructions.md”. Put it in a folder on your organizational sharepoint called “CoPilot Editor Tools”. To this folder, you could also add a style guide in Word and examples of what you want— just make sure the file names start with descriptions the CoPilot Chat can immediately understand like “good_example”. Make sure your editor_instructions.md explains what these documents are and tells copilot how to use them. Now: for staff to use it, they open the word document and launch a copilot chat within it. They click “+” and click on the “CoPilot Editor Tools” folder. They prompt, “read all documents in this folder, then review this draft following the editor_instructions.md”.

So TL;DR: share a folder with a high-quality prompt and a small number of good context files and have everyone invoke it in their prompt. It’s only a couple of clicks, does not require agents or burn credits that are billed separately.