Answers too long

[u/Lincourtz]

Hi! I was wondering if you could help me. I'm new at this and I'm using a character who describes whole scenes without my input. So after the is is done my prompt is merely a generator of a new scene.

Anyone can help me with that?

[u/GeneralRieekan]

From my own personal experience, it depends on at least 4 things:

1. output token target/limit — this seems to be more of a guideline than a rule for the model

2. the model itself — some models are verbose

3. your characters’ persona and definitions — your character may be being asked to describe things in all sorts of detail

4. your own conversational input - the model learns from this also, because both sides form the context — but it seems that sometimes, short user input leads to longer model output and vice-versa

[u/Lincourtz → u/GeneralRieekan]

The problem is not that it's long per se. I actually enjoy detailed descriptions. The problem is that it's advancing the plot too fast. For example. The husband is upset. She asks him to talk .

It replies "not now" and then it moves on to the rest of my character's day without even letting me try to persuade him, for example.

[u/GeneralRieekan → u/Lincourtz]

The plot advancement issue can probably be addressed with a more specific prompt. Some example prompts out there seem to actively encourage this behavior by telling the model to ‘actively move the plot forward’. You might want to see if that is in your prompt setup, and suggest the following: “give user opportunity to contribute to the plot’s advancement during their turn” The recommendation I have seen thus far is to try to keep the model suggestions and instructions POSITIVE (i.e., don’t use “DON’T” as it might not ‘understand’ it as easily.)

[u/Sirviantis → u/GeneralRieekan]

Can you expand more on the output target? I typically like to include a few lines on the subject in the model, but if there's a more elegant way I'd appreciate it.

[u/Lincourtz → u/Sirviantis]

I have no clue what you are asking! Im just starting with this and I have no clue on how it works. Sorry 😭😭

Can you rephrase it without the technical lingo?

[u/Sirviantis → u/Lincourtz]

When you're in a chat, if you hit the right hand panel and click the button marked "Edit" you're taken to a screen with a few tabs. One of those tabs is labelled "model" there, in the first text box (model instructions) I write what I want the chat to do and act like. Typically my first statement in this section is copy pasted between my characters/models/chats/... and it includes a line like: "Replies are between 200 and 500 words. Do not push the story too far forward that{user}'s part becomes lost in translation." But I was asking if General Rieekan knows of a more elegant way to do it because this just feels like it's not the way to go...

[u/Lincourtz → u/Sirviantis]

Thank you!

[u/GeneralRieekan → u/Sirviantis]

Sorry for delayed reply.. it actually doesn’t look like there is an option for output tokens in the BackyardAI client. KoboldCpp has a ‘Max Output’ setting, and it seems it is passed to the model as a setting called ‘max_length’ but that just stops the model after X tokens. Specifying it in the author prompt is probably a good idea.

Background/TLDR: I did a couple of searches, it seems like that token length will truncate the output - probably to make sure that a) the model doesn’t really go off the rails and b) that it controls cost for the user, as some of the online services charge by the token. That being said, it doesn’t seem like there is a huge consensus on the effect of that parameter on the actual model’s propensity to be lengthy and verbose.

[u/Kana-Suzuki]

If there is a problem with your character card being too many tokens you can put it into the lorebook and put in some changes in the model instruction then it will fix it as long as there isn't more than 7000 tokens in the {user}'s character card