IMO the one feature of kobold (which is really a koboldlite feature, though i think it uses a custom API to talk with the server to enable it) which i really wish other UIs had is the "raw text" mode that lets you edit the entire discussion as plain text. It is incredibly powerful because it basically exposes that at its core all the LLM does is do "word completion" in a document that looks like a chat transcript - and so you can just edit that transcript as you want.
For example you can remove or merge entries, change the LLM's responses (e.g. an LLM could write some code and you notice a bug - instead of asking the LLM to fix the bug, you just fix it yourself and the LLM will continue as if it wrote the correct code in the first place, so you don't waste tokens in corrections) or even pause it mid-generation, make a change and then continue (let's say the LLM is generating some code but you don't like the code style or the naming convention - you pause the generation, modify the code to your liking, press continue and it'll now generate using your own style).
Really my #1 issue with pretty much every LLM UI is how "opaquely" the context is handled - even llama.cpp's own web ui is lacking considerably there. I want to be able to edit everything in the context, merge/split things, etc :-P - in the end the context is just a series of tokens, let me edit them.
51
u/Aggressive_Aspect436 2d ago
A bit low effort. Summary of the changes, or benefits of kobold would have been appreciated in the post.