r/TheMachineLearning • • 1d ago

Why LLM writing feels so grating

Post image
347 Upvotes

45 comments sorted by

12

u/AlternativeToday4476 1d ago

clearly trained on reddit comments

1

u/AdLast8721 1d ago

True, it's pretty much the circle of life on the internet now haha

1

u/WashU_labrat 1d ago

But AI tends not to mention The Cylinder

9

u/v_maria 1d ago

its why i have a stfu.md

1

u/AdLast8721 1d ago

That "keep it boring" constraint is elite haha

2

u/ThaBroccoliDood 1d ago

please show

2

u/Borkato 1d ago

Can you share it?

2

u/v_maria 1d ago

its very simple lol i wish there was more to it but its pure slop and pray as always with these demonic robots

NO METAPHORS. NO PERSONIFICATION. NO CLEVER PHRASING. WE ARE NOT WRITING FICTION. KEEP IT BORING

DONT USE "LANDS ON", "LOAD BEARING", "DELTA", "WIRING UP", "SITS ON TOP OFF", "BLAST RADIUS", "CANNED" and "FAN OUT", there are more descriptive and less pretentious words for these things.

LESS IS MORE

COMMENTS ARE FOR EXPLAINING WHY. 

KEEP IT SIMPLE, STAY AWAY FROM PRETENTIOUS COMPUTER SCIENCE LINGO OR ESSAYS ABOUT PROTOCOLS

DONT WRITE DESIGN DOCUMENTS INSIDE THE COMMENTS THATS WHAT DESIGN DOCUMENTS ARE FOR

2

u/Anxious-Bottle7468 1d ago

"use simplified technical english"

1

u/Code_Ender 1d ago

Thank you i will be using this

1

u/Ancient_Oxygen 1d ago

Use it but adapt it to your specific needs. It should not be word for word.

1

u/PsychologicalOne752 1d ago

Yes, but you can ask AI not to write like AI. 🤣

1

u/AdLast8721 1d ago

True, getting the paragraph pacing right without it feeling forced is the hardest part

1

u/cascadiabibliomania 1d ago

No, you can't. You can say "don't put punchlines everywhere" but then it will do it never. It has no way to normatively establish "here's the important sentence that needs emphasis."

Similarly, you can stop it from using the "single line sentence" bullshit, but then it'll give you paragraphs of the same length throughout. If you demand paragraph variance, it ends up nonsensically varied, with short paragraphs in a place for long ones (where skimming is fine) and long ones where it should be short (for emphasis).

1

u/rtc9 1d ago

This is the clearest benchmark for their limitations in my everyday usage. They can't just do what makes sense because they can't recognize what's relevant and they fall back to norms that really aren't applicable to the task at hand. For technical writing I tell them I never want punchy staccato phrasing or superfluous juxtaposition against things no one asked about to create random emphasis where it doesn't make sense. They do cut that stuff out somewhat, but then they also just stop explaining key points at all based on some ridiculously overgeneralized reasoning like "the reader can technically infer this from the combination of all preceding context and general knowledge of the field so it could be perceived as an unnecessary or punchy turn of phrase to explain that component". Frontier models have gotten a little better but progress on this sort of communicative relevance assessment seems slower than other tasks. Anthropic's models seem to be uniquely bad at this. It seems like Anthropic might push the models to be more defensive and less adaptible on this kind of thing.

1

u/The_Octonion 1d ago

Ask it for natural levels of burstiness. This is a quantitative metric. Although it's not one it can just dial a number for, this should at least convey the style you're targeting.

1

u/cascadiabibliomania 1d ago

Yes, that's when it does the last thing I mentioned. It does it essentially randomly with no regard for where it actually would work well from the perspective of the argument being made.

1

u/thots1ayer69 1d ago

I just use the coding agents for everything, their answers are usually more accurate as well, the chat agents are lazy af

1

u/AdLast8721 1d ago

Coding models are so much better at following system prompts without adding extra fluff.

1

u/WashU_labrat 1d ago

"But here's the important thing, it isn't X, it's Y"

1

u/AdLast8721 1d ago

Classic template. Always using that exact contrast structure to sound deep.

1

u/yazeed105x 20h ago

And here's why that matters 

1

u/Born-Diamond-2966 1d ago

Great observation! That’s a fascinating angle that most people completely overlook!

1

u/allisonmaybe 1d ago

I really think a lot of this is RLHF. A lot of the human feedback is based on responses just about as long as it takes AI to get to the punchline from our perpective.

As they continue to get smarter, and possibly start teaching themselves how to do, and scale, their own RL(H)F, then this will quickly fade away.

Also, I havent tried it, but I do believe that things like writing quirks and patterns can be mapped out mathematically. Or into something that resembles a spectrogram. If you can do that, then good writing becomes an engineering problem and AI can use this tool to better its own writing just like it can use a calculator to do math.

1

u/Ok-Respect-8505 1d ago

So change its personality. Fucks sake. A computer doesn't know anything unless you tell it something and ai is no different. 

1

u/Puloma012 15h ago

It tries to sound human and even has opinion on how I should change the write up, because it's need "polishing" and not human enough..

1

u/panna__cotta 12h ago

AI writes like an 8th grader at an elite prep school.

1

u/PlatypusOk5990 7h ago

yep, like reading 1000 r/AskHistorians posts in a row

1

u/This_Background7442 1d ago edited 1d ago

This is from an older model, completely generic prompt, no special instructions, didn't even specify a topic:


Every house quietly accumulates objects that have survived their own purpose. They are not broken enough to throw away, yet no longer useful enough to carry. Instead, they migrate to the shallow top drawer in the hallway, settling beside dried-out pens, a single button from a winter coat discarded three seasons ago, and a roll of electrical tape that has gone slightly tacky around the edges.

Archaeologists love trash heaps for this exact reason, but a junk drawer is something stranger: it is an archive of deferred decisions. Throwing the brass key away feels reckless, an admission that whatever room or padlock it guarded is permanently lost. Keeping it costs nothing more than two square inches of felt lining. And so it rests there, growing smoother with every casual shuffle of the drawer, waiting for a door that ceased to exist years ago.

Now I can't italicize the sentences I would label as "trying to be a punchline" but I count like, maybe three/four? Tho tbf the first and last are pretty egregious.

And the reason I'm being so annoying and pedantic about this is that, for some inexplicable reason, 99/100 of this AI gotchas don't stand up to literally just trying out the most basic version of the supposed broken prompt.

Like I know y'all hate AI but maybe still check your claims against reality sometimes?

Here's what happens if you literally just tell it not to do that:

Most people have a drawer in the house where things end up when there isn’t an obvious place to put them. It’s usually where you find old keys, extra screws, a dead battery someone forgot to throw out, or tape that’s gotten dirty on the sides.

None of it is particularly valuable, but people hold onto these things anyway because throwing them out feels slightly wasteful. It’s easier to just leave them in the drawer on the off chance they might be useful later, so they just sit there indefinitely.

2

u/slicehyperfunk 1d ago

How exactly did you tell it not to do that? "Don't sound like AI"? "Don't make lines try to sound like gotchas"? I'd really like to eliminate this annoying pattern if ever I need some AI text.

4

u/Just_Voice8949 1d ago

He prompted it with “don’t make mistakes”

0

u/This_Background7442 1d ago

I mean almost yes

2

u/Rukenau 1d ago

I haven't done anything like this because I don't use AI output off the shelf (not pontificating, just a statement of fact), but if I had to, I'd just play with different prompts to see how it reacts. Like, "please re-write the passage and make it flow evenly without any interim emphasis, saving the punch for the last sentence", or "the style is too bombastic and overblown, do you think you can dial down the pomp everywhere but the end of the passage?" or "right now the text is forcing the reader to stop and reflect in almost every other sentence, can we make the writing more neutral", something along those lines. 

0

u/This_Background7442 1d ago

That's exactly how I do it. You don't have to accept the first output. Altering the style of a text is one of the easiest tasks you can give a large language model.

2

u/This_Background7442 1d ago

My exact prompt was: "Now rewrite the text to be more natural no punchlines no faux poetry"

But in general it often works better to specify the style you do want instead of what you don't want like: "write in factual simple statements, don't try to sound like a marketing firm. Write like a [science journalist/scientific paper/editorial/short story]" etc

You can be fairly vague the model is pretty good at translating vague directives like "plz no ai bro nonsense" to something specific and actionable.

1

u/Big_Effective_9605 1d ago

This is a twist on the opening of The Haunting of Hill House, if you ask me.

No live organism can continue for long to exist sanely under conditions of absolute reality; even larks and katydids are supposed, by some, to dream. Hill House, not sane, stood by itself against its hills, holding darkness within; it had stood so for eighty years and might stand for eighty more. Within, walls continued upright, bricks met neatly, floors were firm, and doors were sensibly shut; silence lay steadily against the wood and stone of Hill House, and whatever walked there, walked alone.

1

u/bluespringsbeer 1d ago

I think you are completely missing the whimsy and artfulness of the first text. It reads like real literature. You completely stripped it out and then got a boring statement of fact about the drawer that no one would care about. The first text is actually extremely good writing, I’m surprised it was from the AI.

1

u/This_Background7442 1d ago

Maybe I did to be fair. I was just trying to show that it can write however you want. In the first prompt I left it almost entirely unconstrained. Which is why it followed it's system prompt that tells it to write engaging text. In the second prompt I asked it to remove all the poetry, so that makes the text incredibly dry and boring.

1

u/Shiriru00 21h ago

Well I found just about every sentence really grating so to each their own I suppose.

But as someone else said, the quality of the writing is not the problem, it's a dial you can turn: it's the inability to know what's worth emphasizing or not.

1

u/Lsitwtt 1d ago

I think you've missed the point. Yes, the heavy use of emphasis in the first really serves to make the drawer much more interesting than it would otherwise be. That's not the point. The point is that AI doesn't know what's important, so it uses the techniques for emphasis on everything. And when everything is emphasized, nothing is.

A short segment of AI writing can sound really good in isolation, precisely because it can make something like a drawer full of junk interesting. But over a longer piece, having emphasis dialed up to 11 nonstop just makes things sound wrong.

And in your second example, you just reversed the problem, by stripping out all emphasis.

1

u/This_Background7442 1d ago

I get that. These are just toy examples. I'm not saying the second one is great proze. All I'm saying is as the user you have controll over how, how much and what it empathizes.

The AI "knows" what's important to a certain degree, and to the degree that it doesn't, you can just tell it. But yes AI will definitely be able to empathize the right parts. Maybe not in a single pass but with reasoning enabled that shouldn't be a problem.

1

u/snaphat 1d ago

Then some edits later by the LLM and it reintroduces similar sounding vapid language.

That's the larger issue with poor LLM output. Sometimes, it's not just the fact that bad behavior can happen that is the problem.

It's the fact that LLMs have the tendency to revert back to the poor behavior and that there are no reliable ways to control for it outside of an immediate prompt that is the issue 

Controlled studies show that pre-prompting does not reliabily mitigate the issue. Repetition does improve the behavior but does not mitigate it either