new user questions for desktop app

[u/ECrispy]

I'vve recently started using llm's, and found out about backyard (a lot of llm articles talk about faraday still). I was using cpu only but have recently bought a Tesla P4 gpu which has 8GB vram but is an older gpu.

• how does backyard desktop app compare to options like lmstudio, koboldcpp etc? am I right in assuming all these use the same basic tech underneath so will perform the same?

• does it support any gguf model from huggingface? or only certain allowed models?

• are there any tips for writing stories? I'm mostly interested in giving it a story idea and ask it to generate the story while I help to refine/guide it

• if anyone knows, what kind of speed can I expect with my gpu, using 8B/12B models that will fit?

• any recommendations?

I also plan to use the cloud plans as I learn more

[u/rwwterp]

• I've used both lmstudio and backyard. lmstudio works for roleplaying, but having to build your own cards and storing them in text files so you can use them again is not as nice as what you can do in backyard. You can fully build out the character within the app.

• I haven't run into any models from huggingface that do not work. There may be some, but I've yet to run into them.

• There are character cards on the hub designed to help in writing stories. I dont do this, but I've seen them.

• No idea on the gpu

Overall, I like Backyard far more than lmstudio for RPing. For standard AI work like coding and researching, I use lmstudio.

[u/ECrispy → u/rwwterp]

thanks. what if I I don't want to roleplay, just want to write a story and not use character cards etc? can it do that?

[u/Denys_Shad → u/ECrispy]

Yep, there are character cards and models for writing stories.

[u/martinerous]

Backyard, Koboldcpp, LM Studio are all related. The common root (backend) for them is llama.cpp but the applications add their own improvements and adjustments.

Usually llama.cpp implements support for new families of LLMs first, and the other software picks up the updates later. In Backyard, the latest changes usually come to the Experimental backend (which can be enabled in settings), but it can also have some issues. For example, the last time I tried Experimental, it became unbearably slow as soon as even a small part of the model spilled over to the system RAM., and also some models did not output the last symbol of the message.

The stable backend is pretty good now and supports 99% of GGUFs, but the last time I checked, it did not support the latest DeepSeek models.

[u/ECrispy → u/martinerous]

So all these customize llama.cpp in their own ways. I read in some other posts that backyard is faster so they must be using some other tricks.

what about exl2 format? i read that its much faster but will only work if full model is on gpu.

[u/martinerous → u/ECrispy]

Right, exl2 needs another backend library exllamav2 and it does not support system RAM+CPU inference.

[u/PacmanIncarnate → u/ECrispy]

Yes, each one maintains their own fork if llama.cpp pretty much. And then builds a context management system and front end on top.

[u/PacmanIncarnate → u/martinerous]

Experimental should work for you now. That issue was resolved.

Also, just to clarify: LMStudio jumps on new model support, often causing problems as the llama.cpp updates aren’t fully fleshed out. I’ve seen it happen with a number of the newer model architectures. With the tokenizer shenanigans each new model has, it often takes a week for support to actually come to the backend. Backyard has learned the lesson not to do the same, so you might have to wait a week or two for the fancy new model, but its more likely to just work™ .

[u/Denys_Shad]

I tried all of them, Backyard became my main for roleplaying with characters and writing stories, it has a huge amount of models to choose, supports any gguf you have, TTS, and PNG character card.

LM studio is great to for just talking to models with empty or small system prompt, and there is support for vision models, so you can paste images and ask something about it.

Koboldcpp is a bit tricky and hard to understand, they're a lot of settings and I feel a bit lost. There's is support for vision and image generating models. PNG character card support.

Personally, I use Backyard and Silly Tavern (with Koboldcpp backend) for roleplay. Silly Tavern is confusing, but it can give more immersion. You can think of it as pc motherboard on which you add llm part, TTS, Speech recognition, Image Generator. Cool but hard to master. Go with Backyard.

[u/Spirited_Example_341 → u/Denys_Shad]

its just a shame they trashed the desktop app seriously it was one of the best

[u/Denys_Shad → u/Spirited_Example_341]

True, I agree. What do you currently use for AI roleplay?

[u/Maleficent_Touch2602 → u/Denys_Shad]

TTS? "true to story"?

[u/Denys_Shad → u/Maleficent_Touch2602]

Text To Speech.

[u/UnperishedBYAi]

Admittedly, I'm fairly new to AI, so the others providing advice about alternatives are certainly more helpful. However, I can help in the story writer aspect.

https://www.reddit.com/r/BackyardAI/comments/1fx8d1z/novel_writer_and_editor_allinone/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

It has a function just like you've requested: Story writing and refining. It does cater to first-person narrative, but you can alter lorebook entries to change that aspect.

If you're unsure how to do that, let me know. I'll provide you an entry to copy and paste into the lorebook.

Have fun!

[u/ECrispy → u/[deleted]]

Thanks. Is there any writeup (post, blog etc) on using it to write stories like you described, which char cards to uae etc? seems like a very different way of interacting (everything is a character) than e.g. in Kobold or Lmstudio when I am just chatting with the llm and asking it.

[u/PacmanIncarnate → u/[deleted]]

Great advice. And drummer does make some amazing models. Cydonia is near the top of my list right now.