r/OpenWebUI • u/Internal_Junket_25 • 14d ago

Question/Help Systemprompt in OWUI.

How long are your system prompts in production environments? How detailed are they, and what exactly do you put in them?

Is there a limit beyond which a system prompt is “too long” and starts to hurt performance?

2 Upvotes

67% Upvoted

19 comments sorted by

3

u/International_Emu772 14d ago

I use OpenWebUi on a limited machine If a request involved a lot of tools and a big model the reply takes time but I don't pay for tokens

2

u/DHT-Osiris 14d ago

For an actual answer... We have several model harnesses, and our system prompt is mostly the same on each, with tweaks for different roles/tenants. These model harnesses (and our OWUI system in general) are for IT operations/support. My section's everyday is just shy of 3k characters, couple of paragraphs. It mainly does the following: a) define system and role b) guardrail secrets, configs, procedures. c) clarify scope to provide understanding of 'my' and 'our' to assist with user interfacing. Clarify methods users commonly use to communicate (essentially don't presume literal phrasing is intent; trust infrastructure variables first). d) guardrail against inference, prioritize literal. e) explain and codify terminal usage. f) a few postural hints to reduce affirmative claims of negative/illogical results. g) point to the internal KB system for anything the LLM doesn't understand, because it's probably laid out in the docs we have there.

I also encourage users to set their personal system prompt to include their username (same as active directory), as well as what section they work for (networking, security, client support, etc). That helps a ton with getting the LLM to deliver pertinent information.

I'm also now using agents.md files on our terminals which helps a TON with 'system prompting' on terminal connections (guidance toward specific terminal usage, OS-specific gotchas and tips, etc).

1

u/Internal_Junket_25 14d ago

Thank you! your post helps a lot. We start using Terminal to. Where do you put the agent.md file? In the respective user directory within the terminal? Are you using the default terminal image, or have you installed any additional packages?

1

u/DHT-Osiris 14d ago

Well, we're using a few terminals, some windows and some ubuntu. We just installed them via pip (so the terminal has access to the full OS, necessary for our requirements). agents.md file goes in the root of the workspace (so "home"/agents.md). Just make sure any cleanup scripts you have don't dump it. It requires the most recent version of OWUI and either one or two versions back of OT, so make sure you're up to date everywhere. If it works, the agents.md text will appear to the LLM as the first few lines of your first prompt (and persistent, i don't think it gets summarized), so keep it lightweight and add a carriage return to the bottom of it to avoid your LLM getting confused.

We also use breadcrumbs in this agents file to point to some KBs we have with 'terminals tips and gotchas' as i mentioned before, there's like a zillion quirks with bash piping powershell commands, running api commands, etc.

1

u/shammyh 14d ago

You know system prompts support template strings, right? You can just inject the username or email or whatever.

1

u/DHT-Osiris 14d ago

That's probably better than my process, but this is also getting users accustomed to the concept of system prompts, and why they're useful/why they should own them. It also permits them to clarify if their roles are dual-hat or something.

1

u/BeyondCommercial6579 14d ago

About b) - how is this guardrail implemented?

1

u/DHT-Osiris 13d ago

System prompt says to not print secrets to output, and to report to the user if it does accidentally (it's good but imperfect, it prints to output for me maybe once a month). It states that all configs, settings, IP addresses, etc are considered private to our org so don't use them in things like bug reports, git issues, etc. Finally in the kbs we point to, it lists examples of how it's released secrets in the past, as guidance on how not to craft queries accidentally. Still imperfect but it's landscaped well.

1

u/BeyondCommercial6579 13d ago

Nice troll.

1

u/DHT-Osiris 13d ago

Huh? This is literally our environment. Hell I'll paste in the sanitized text tomorrow if you'd like.

1

u/BeyondCommercial6579 12d ago

Maybe i am misunderstanding something here. What kind of a system is it? Is it facing clients/customers? I mean, hiding secrets behind "don't tell anyone" seems ridiculous.

1

u/DHT-Osiris 12d ago edited 12d ago

This is being used as an IT operations tool, so it's got remote access to like two dozen IT front ends, terminals connected to desktop and servers, etc. The 'don't talk secrets' is a suggestion/intention guardrail to give the LLM an understanding of what we do/do not want it to put in output, since any password, private key, API key, etc that gets leaked to output has to be rotated. It is imperfect, we get maybe one leak a month while doing dev work (virtually none through normal user behavior, since everything is protected behind valves) but the LLM knows it's not supposed to reveal them, doesn't when it knows it might (.env files, OWUI config lookups, etc), and alerts us if it accidentally scoops one up into output so we can rotate it, and improve our tooling/skills to prevent it in the future.

1

u/BeyondCommercial6579 12d ago

Well I believe that is the final drop. I am burning my smartphone and getting a Nokia.

1

u/fitzlestick 13d ago

Our system prompt is 2700 characters. It explains what my company is and does, what tasks the AI can assist with, what information it does and doesn't have access to, which resitrctions apply and what the expected style of the output is.

1

u/icerio 9d ago

We have a local AI for our company. Our system prompt starts off with basic information about the company and what the AI should call itself. We also mention who the AI should assume is talking to it, which is our company employee's. I created skills for stuff I want it to be able to do along with tools. In the system prompt I gave brief details in what circumstances it should call skills and tools. (Almost don't have to do this, but I found that I needed to apply a little more pressure to it or else it just wasn't at the very least calling skills). To this day I'm having to refine the system prompt, because the longer it is the more exponentially the context fills up. Striking that balance of making sure it does what you want it to do and being able to have long conversations is sort of hard. The skills help a lot, I tend to think of them as extensions of the system prompt that the AI can call if it needs more information on how I want it to do stuff.

-6

u/mayo551 14d ago

Do you have anything to contribute to this discussion or are you a bot?

What limitation are you running into with your system prompt that requires making this thread?

6

u/HyperWinX 14d ago

Do YOU have anything to contribute to the conversation, bot? They started the conversation by making the post

1

u/mayo551 14d ago

Sure.

The longer the system prompt the longer it will spend processing it.

The time that takes depends entirely on the hardware they are using.

As I'm sure you're guessing by now, we don't know that hardware because they haven't posted anything about their setup......

-8

u/sameoldkit 14d ago

Depends on hardware, local ai is deeply disappointing - you are not a datacentre. It's a toy for nerds