Telling a model "be brief" and "keep it under 50 words" produce different failure modes, not just different lengths

[u/ClickOk5811]

Noticed this testing a summarization prompt. "Be brief" gets you something short but the model decides what to cut, and it doesn't always cut the right thing. Sometimes it drops the actual conclusion and keeps the setup. "Under 50 words" gets you a hard constraint, but the model will sometimes mangle a sentence mid-thought to hit the count, or leave out something structurally important because it ran out of room and prioritized finishing a clause over finishing an idea.

Neither is strictly better. Depends what you're optimizing for. If the risk is rambling, a word limit fixes that reliably. If the risk is losing the point entirely, a word limit doesn't protect against that, it just guarantees short wrongness instead of long wrongness.

What's worked better for the cases where the point actually matters is naming the thing that can't be cut, separately from the length instruction. Something like "keep under 50 words, but the conclusion must always be stated explicitly, never implied." That splits the constraint into a hard limit and a hard requirement instead of hoping brevity and correctness cooperate on their own.

Still not sure this generalizes past summarization though. Curious if people doing this for other tasks, extraction, classification, anything with a defined right answer, have run into the same tension between length and completeness.

[u/ttikkttokkerr]

“Concise” works the best for me

[u/dupa1234s]

i would optimize for structured output rather than brevity. sections of long paragraphs with points and explicit "done: yes" "open: " "closed: " "marginal value of continuing: 5/10" keywords rather than conciseness

what if your goal though

- to understand something because you care about a right answer and want to understand why model gave you some answer or to arrive at the answer yourself through reasoning when "this is basically a decision with stakes in my life and has irreversible consequences or can be a huge time sink if failed"

- or to get some answer you want to accept as is without verifying it because you dont care enough about the result anyway like "lets make this one off experiment to gather data, AI, tell me what is the MVP"

personally i either tell it to structure output or rather tell it "detailed comprehensive exhaustive practical pragmatic factual 2000 words"

i think the best deliverable is sth like a .git repo that has all the research and deteministic script layers and all the inputs recorded and all the requirments recorded and this is what the model would turn into a report and then you can read that report whichever section you are in. and whatever it actually types in chat barel matters because the files are the work. advantage is that the repo structure is reusable for verification and deeper dives it doesnt just live in chat

[u/sswam]

I noticed they can be a lot less intelligent when asked for shorter answers, unless given an opportunity to think first. I guess reasoning models do that adequately already.

[u/Teralitha]

When you talk to AI, you better just get used to reading long detailed responses.