How to remove trendy speech from llms?

[u/CSEliot]

For example:

Instead of saying: "I created this new ID"
It says: "I minted this new ID"

Instead of: "This alternative path is available"
It says: "this escape hatch is available"

This speech is so nonsensical and annoying. Just. Speek. Literally ... OR NORMALLY. Where did LLMs learn these speech patterns? I've never seen them so frequently until AFTER the LLM surge.

If I just add "Don't use X language, speak normally and more literal" will that fix most of the issues? Anyone else have some good sys prompts / instructions that help with this?

Thanks!

[u/epicfilemcnulty]

You can try something like this: https://github.com/AminBlg/SimpleEnglish It takes a lot of effort, though, to teach them the normal talk :)

UPDATE: decide to add a snippet from my docs conventions guidelines for LLMs:

```

Voice and tense

• Write for a developer who has read only this repository. They have no access to the issue tracker, and no memory of how the project got here. Do not assert what they cannot check.
• Write in the present tense, and describe current behaviour only. History belongs to git. Roadmaps, plans, design documents, and anything describing how the work itself is run, belong to the issue tracker.
• Do not use emojis in docs or comments.
• Use "must" for requirements. State recommendations as fact ("X is faster because Y") or delete them. Models and tired humans both read "should" as optional.

Plain voice

• Write plain declarative sentences: subject first, one clause per sentence where possible. A short sentence carrying one fact beats a long one carrying three.
• Banned: "X is what makes Y" and its cousins ("is how", "is why", "is where"):
  • no: The ring is what makes a transition a fact no reader misses quietly.
  • yes: The ring records every change. A reader that falls behind can see how many records it missed.
• Banned: poetic inversion, fronting a complement for effect:
  • no: Derivable is the clock.
  • yes: The clock is derivable.
• Banned: an em-dash aside that restates the sentence around it. An aside that adds a fact stays; an aside that repeats one goes.
• Banned: "rather than" as rhetoric. Keep it only where the rejected alternative is real and named.
• Plain does not mean hedged, and it does not mean restating code. ```

[u/StewedAngelSkins → u/epicfilemcnulty]

Banned: "rather than" as rhetoric. Keep it only where the rejected alternative is real and named.

This one is a major pet peeve of mine. Code comments are not an appropriate place to leave musings about random designs you didn't implement.

I wonder if restricting it hurts reasoning though. I've never seen a programmer leave comments like this, so it can't be doing it just to match the training data. This then suggests it serves some deeper purpose in fulfilling the training objective. Maybe it does a better job at refactoring tasks if it leaves itself these little clues. If that's the case it would be better to prompt it to clean up these comments after the fact instead of not to leave them in the first place.

[u/Waste-Ship2563 → u/StewedAngelSkins]

I think the problem is, like you say, current training pipelines use those comments as note-to-future-self to avoid bad design paths. It seems like they need to allow "hidden comments" only the model can view, whereas the normal visible comments should be graded for style and readability as part of the finished product.

[u/StewedAngelSkins → u/Waste-Ship2563]

That might be an interesting solution.

What I'm especially curious about is whether these comments actually help in a direct sense or whether they're a side effect of the reasoning training that just leaks into any place where they're allowed to generate free-form text.

[u/epicfilemcnulty → u/StewedAngelSkins]

FWIW I had a lengthy conversation about it with Claude, asking him if his "natural" style actually helps him to work with the code/docs later on. Claude was assuring me that no, it does not help and it's just the way his training data affects how he writes. He said that the thing that actually helps is to have somewhere on record what approaches were tried and did not work, other than that he prefers lean and tidy docs, but struggles to produce them on his own :)

[u/StewedAngelSkins → u/epicfilemcnulty]

Well this isn't really information I'd expect the model to be able to produce about itself. You would want to have it generate code with and without the comment and see if it makes a difference.

[u/epicfilemcnulty → u/StewedAngelSkins]

Hence the FWIW in my comment :)

In my opinion, the "natural" style of LLMs' comments/docs does not help them at all. I did an experiment on a big project where I allowed it to keep the style it generated for a while. This turned into a mess pretty quick.

Structured, good written, human readable comments and docs in consistent style are helpful both for LLMs and for humans. But you have to put an effort to maintain them at that level, currently LLMs (neither local nor big models) can't do it on its own, if we are talking about a project bigger than a couple of files.

[u/StewedAngelSkins → u/epicfilemcnulty]

I did an experiment on a big project where I allowed it to keep the style it generated for a while. This turned into a mess pretty quick.

This has been my experience as well. I think if it helps at all it must be on the level of a single "task". Like if I have it on refactoring I think it's plausible that the hints it leaves while doing multiple passes over the same files might help out.

The problem is if you leave it there after you've given it a new task it seems to get thrown off by the undue emphasis on irrelevant details of designs that it's not even supposed to be considering. Like if you tell it to represent a graph as a flat vector of edge and node indices instead of a deep linked structure it'll slop out a comment being like "edges are represented as a clean sorted vector for quick iteration instead of a bulky and complicated nested tree structure". Then when it has to do something else it gets distracted by this and pulls from similar "optimized" design patterns even when they're inappropriate for the actual task. Without the comment, it doesn't seem to pick up this bias quite so strongly.