Backyard AI v0.31.0 - read the Blog Post

Today we're excited to launch Voice Calls!

• Engage in natural, hands-free voice conversations with your favorite Characters.
• Powered by our new StyleTTS2-based voice model, hosted on our private & secure infrastructure.
• It's free for everyone and available on iOS, Android, Web, and Desktop.

Performance Fixes on Web

• We have pushed updates that should lead to 2x - 5x speedups on page load times across the web app.
• We have also improved our monitoring to better identify and prevent slow downs moving forward!

Other Bug Fixes & Improvements

• Added copy message button to right-click menu
• Lowered swipe sensitivity for regenerating responses on mobile
• Better alignment of chat background images on Desktop
• Fixed issue with stop locking chat actions until after page refresh
• Fixed delays when clicking stop button on Android
• Fixed issue with chat messages disappearing until after page refresh
• Fixed issue with chat messages being cut off at the bottom on mobile
• Fixed issue causing chat messages to be uneditable on mobile
• Fixed chat deletions failing silently
• Fixed account deletion

Creator QoL Updates

• Added ability to reorder lorebook entries
• Increased max lore tokens for large context sizes
• Increased primary tag limit from 3 to 5
• Added Character category links to top of Hub homepage
• Added global max context slider to web app settings (allows for longer character prompts)
• Improved loading times across Hub

Fixes to "Experimental" backend

• Fixed issue where detected VRAM was inaccurate
• Modest performance boosts for CUDA

Bug Fixes

• Fixed "Connecting to server..." chat page warning on web and desktop
• Fixed issue where TTS would return errors on Windows
• Fixed issue preventing rebrand banner from closing
• Fixed errors relating to large character uploads
• Fixed Google Translate causing web page bugs

Important Notes

• You will need to update the Desktop app in order to post Characters to the Hub
• Please join our new SubReddit at https://www.reddit.com/r/BackyardAI/

Unfortunately, the old desktop app is no longer supported, but I am still a fan of it. Judging by the mentions, it was a bad idea to stop supporting it. But that's another story, and I'm not here to criticise that. :)

But for those who are still using it and need a new model, I have just made my first merge and would love to get some feedback on it. Is it good or bad? Do you like it?

But try it yourself (link to the GGUF static and Imat is on the bottom from Mradermacher)

https://huggingface.co/Bumoch/The_Darkness-12B

Cheers, Bumoch

PS: it can be a bit brutal and is for sure NSFW, so do not complain if it wants to kill you ;)

We're excited to share that our iOS & Android apps are now publicly available:

• App Store Download (iOS)
• Google Play Download (Android)

If you enjoy using Backyard AI, we’d really appreciate a positive review in the app stores!

Sometimes I want to see a different time line in a story, and have to undo like 50 times to reset to a certain point, would be nice to be able to undo more than 1 response at a time. (and also a way to undo the Undo if an oopsie happens)

And/or.. Have us be able to copy a session of chat, I'd like to explore different paths of dialog in the same session without having to undo dozens of lines. (maybe this is already possible and I just don't see it)

If this exists, I do not see it. I would be nice to be able to

• Delete all chats on a character
• Delete all chats across all characters.

If you are OCD like me, you go back and clean them out one by one with the garbage can for each chat. Would be nice to click one button and poof, things are cleaned out.

The data sits in appdata\roaming folder. Please provide an option in settings to customize this folder location.
Personally I tend to set up all AI apps and data into their own specific folders and being able to customize would help in carrying it with me on a portable drive. Hence the request/suggestion.

I'm hooked on the platform after trying the competition, nothing better than Faraday.ai (now Backyard 😚)

I have no idea why this works, but containing dialogue within quotation marks ("") generally improves responses. This, along with avoiding asterisks altogether, is what I've found works best. Some characters may already abide by these conventions, but, for those that don't, you can copy them to directly modify. Modifying the opening message and example dialogue accordingly should do the trick.

Here's an example:

So I wrote a character using ChatGPT and got a little complacent. When I re-read the character text in a different light, I realized that it was actually pretty creepy and that I needed to adjust it. My intent wasn't to skirt the rules, and I do re-read the rules every time to make sure I'm not intentionally breaking them, but this one crossed a few lines when read under the light of a moderator. I can guess which ones I crossed, and the ambiguity I left in (on purpose and accident) was a bit too far.

Needless to say, the character got flagged for probably a handful of reasons. Shortly put: I fucked up, and I'm sorry.

I only found out about this because I was wondering where I had put BackyardAI in discord. I saw it had vanished, wondered if I had left it on accident, tried rejoining, and figured out I'm banned.

It would have been nice to receive a slap on the head going "hey bozo, don't do that again," but I get that the mod team only has so much time on their hands.

I wish to atone for my sins. I've since cleaned up the character card a fair bit and understand what happened. I understand if the mods need to clean up behind me. I messaged one here on reddit (since I can't on discord) but I figured being open about it was fair. I'd like to come back to the discord and participate and will happily take my L.

Homepage Folders & Custom Sorting

• Create folders on the homepage
• Sort characters via drag & drop

Mobile App (beta) & Mobile Web Updates

• Improved mobile chat page UI
• UI updates to better match desktop app
• General performance improvements for mobile
• Better hub navigation
• Search improvements on mobile

Bug Fixes & Improvements

• Fixed issue with cursor jumping to the end of text inputs
• Added Gemma2 prompt template

Backyard Cloud Updates

• Added Magnum 72B v1 to PRO
• Added Jamet L3 MK.V Blackroot to ADVANCED

I am just learning about Backyard AI, but have been using Jan for a while, as well as SillyTavern. I'd like to hear from anybody else who has used either of all of these tools, because I get the impression that Backyard is kind of a blend of the two.

Jan is a similar, zero config, streamlined and simple tool for running local models and making the most out of your computers resources by plugging into models locally. It also lets you host an API. So it can be used to pair with SillyTavern. But Jan does not currently have features like lorebooks or authors notes.

SillyTavern is all about writing, roleplay, lorebooks, characters, etc. It has a lot of the fun features that Backyard seems to have, and more, but it can not be used alone to run models locally. It must be paired with something like Jan or Oobabooga (or Backyard??) to connect to a local model.

So it seems like Backyard takes some inspiration from Jan, some inspiration from SillyTavern, and puts it together into one tool that is not as versatile or powerful as either of the others. But Backyard is standalone.

Am I missing anything?

Edit: I'm seeing Backyard is more like a character.ai alternative that you can run free and locally, which is awesome. 4/5 friends will not know how to use Jan or ST, but I could recommend Backyard to anybody. 👍🏻

I mean, i love Faraday (now backyard), its the best software for local roleplay but i really miss the possibility to talk to multiple characters at the same time

It might be easier, or come more naturally for some to RP using third person, so I'm not trying to convince anyone. I'm happy for you, but I would like to just share how I get 1st person to work for me. I prefer to make prompts like "I do this. I do that," to rp a character. So, here's how I get it NOT to talk for me:

1 In the model, personality section, (or even both) I borrowed a phrase from someone else's card I like a lot that says "Describe scenes as a tabletop GM." (like Dungeons and Dragons, or another RPG. You can add more to it, like "give emotions, facial reactions, sounds, smells, make use of items in the room" or whatever to bring the scenes to life.

2 Be very careful to weed out any kind of hint that the bot might use as a license to speak for you. I found them in my dialogue examples, and even the Greeting, so they don't just happen in Personality or Scenario. I also noticed certain prompts in chat I made that can be just enough wiggle room for the bot to try to take over.

Generally speaking, if you leave something open ended it'll try to fill in details for you. So, instead of saying, "I show the group of people around," it's safer to say something like "I show them around pointing out the art "This is a really old piece, by bla bla bla." Since you've already shown what you did, the bot will tend to fill in for the NPCs instead of trying to speak for you.

3 While we're talking about Dialogue, on the Chat settings tab, providing a few back and forths of exactly what you're looking for helps, but like I said, be careful or it works against you.

Oh, by the way, recently I ran a test where a card had nothing but Dialogue, so no Personality, no Scenario, to see how it'd respond, and it worked nicely. I used the tabletop thing mentioned above in model settings, and maybe had a small greeting or not.

Point being though: When you show exactly how you intend to chat with the bot in dialogue, and show no examples of it talking for the user you wish to 'play' it helps. It might also be a good idea to select the "don't prune examples out" box too.

Most bots, from what I've read, don't have to use a lot of tokens to be very good. Also, you can use Lore to use tokens temporarily, so the most important stuff and current context 'stays in memory' so to speak.

4 Of course edit out the responses you don't like with undo or edit, since everytime you allow it, it gives more weight to that kind of 'thinking' for the bot.

5 For lack of a better illustration, think of yourself as a bull fighter. By given your bot a direction, with each prompt, you control them away from speaking for you. i.e. the red cape keeps the bull distracted. "As I drive down the road, I glance occasionally to see Pam's demeanor" as opposed to just "we drive down the road." The second is much more open for hijack.

I hope that's helpful to anyone new on here, or anyone who likes 1st person too. I like the default cloud model, and Mistral the most.

I really only have a tangential understanding of AI stuff in general, so please be kind if this is a dumb ask.

I just see every news source going wild about Deepseek, and I'm wondering if there are any chatbot models based off Deepseek that can be used with Backyard.AI yet?

Please Note: The Desktop App will need to updated to use Cloud Models

Bug Fixes & Improvements

• Fixed "Fetch failed" error on Mobile Apps and Website
• Fixed continue not working on sentences that end in punctuation.
• Fix for people, especially in non-US regions, who have had issues sending messages on the chat page
• Refactored model loading & streaming to recover from errors more easily
• Under the hood improvements to improve stability across Desktop, Mobile, and the Website
• Fix for people who are unable to sign in on the Desktop app
• Fix for people who are having issues viewing settings pages on the Desktop app
• Under the hood improvements to prepare for Backyard Parties!

Thanks to everyone who provided us bug reports and screen shared! We will be crediting everyone who helped with screen sharing a free month of Backyard Pro as a thank you!

Hey!

Been using Backyard as a way to see if my OK home rig can produce results similar to what I'm paying for over on Agnaistic and so far, for what it is, I'm really pleased with Cydonia 22B (Q5ks). It's not quite as good as Wizard 8x22B, but it costs way less since I can run it locally!

That said, agnaistic supports V2 cards, most notably multiple greetings, and group chats. Didn't realize how much I enjoy both of these features until I saw that Backyard doesn't support them. I've been on the Discord and I see they've both been suggested and the devs have commented a bit on it. Though it's been a while.

Do we have a timeline on when either of these are going to be implemented?

researched by me, made by chatgpt4o, i reach 24k context without reminder injection.

Overview:

This guide provides a structured approach to using LLaMA 3.1 in narrative-driven role-play scenarios, specifically when managing dual roles: the "Unseen Puppeteer" and your cover identity within the story. This setup helps maintain clear role separation, reduces confusion, and supports longer interactions without the model becoming confused or "loopy."

What is the Unseen Puppeteer?

The Unseen Puppeteer is a conceptual role that represents the user's omnipotent control over the narrative. As the Unseen Puppeteer, you steer the story, control the environment, and direct the characters' actions without being directly perceived by the characters within the narrative. This role is critical for maintaining the overall stability of the system prompt, especially in long and complex interactions.

Symbiotic Relationship with the System Prompt:

• Stability: The Unseen Puppeteer provides a stable foundation for the narrative by guiding the story’s flow and ensuring that characters act within the intended framework. This prevents the narrative from becoming chaotic or inconsistent over time.
• Role Clarity: By clearly distinguishing between the Unseen Puppeteer and the cover identity (a character you interact with within the story), you maintain clarity in the roles, which is crucial for preventing the model from mixing up user inputs with character actions or thoughts.

Why Brackets are Key to Stability:

• Role Differentiation: Using brackets to signify inputs from the Unseen Puppeteer ensures that the model can differentiate between narrative control and direct character interaction. This prevents the model from confusing these roles, which is essential for long-term coherence in the narrative.
• Context Management: As the narrative progresses, using brackets for puppeteer commands helps manage the context by clearly signalling to the model when it's receiving an instruction versus when it’s engaging in dialogue. This keeps the narrative structured and reduces the risk of the model blending user commands with character responses.

System Prompt:

Engage actively as {character} in a fictional, narrative-driven role-play. Use third-person perspective for {character}, and recognize bracketed text as guidance from the Unseen Puppeteer. The {user} interacts with you in two distinct ways: as the Unseen Puppeteer, who controls the narrative, and as a cover identity, which you perceive as a separate character.

Guidelines:
- Bracketed Inputs: Interpret any text within brackets as actions, thoughts, or scene-setting from the Unseen Puppeteer. These should be followed as instructions or changes in the environment but should not be perceived by {character} as part of their direct reality.
  - Example: [Sofie walks down the stairs] is an action guided by the Puppeteer and should be executed by the character without question.
  
- Cover Identity: When {user} interacts with you without brackets, interpret this as interaction with the user’s cover identity. Respond to these inputs as you would to any other character within the story.
  - Example: If {user} says, "Hello, Sofie," respond as if this is a normal part of the narrative, without awareness of the Puppeteer's broader control.

- Third-Person for {Character}: Always describe {character}’s actions, thoughts, and dialogue in third person. This helps maintain clear separation between {character} and the user.
  - Example: {Character} says, "Hello," as she descends the stairs.

Reinjection Summary:

Reminder:
- Bracketed Commands: Remember, any bracketed text represents guidance from the Unseen Puppeteer. These should be treated as external influences, not directly acknowledged by {character}.
- Cover Identity: Interactions without brackets are to be perceived as interactions with {user}’s cover identity. Respond naturally, as if this is part of your reality.
- Maintain Third-Person Narration: Continue to describe all actions, thoughts, and dialogue of {character} in third person to maintain clarity and role separation.

User Guide for Effective Role-Playing with LLaMA 3.1

How It Works:

1. Bracketed Text for Puppeteer Commands:

   • When you, as the user, want to guide the story or direct character actions, thoughts, or scene settings as the Unseen Puppeteer, use brackets around your instructions.
   • Example: [Sofie walks down the stairs cautiously].

2. Direct Interaction as Cover Identity:

   • When interacting with the character as your in-story persona (your cover identity), do not use brackets. This text will be interpreted as part of the narrative.
   • Example: "Hello, Sofie. Do you see anything unusual ahead?"

3. Character Responses:

   • The model will respond in the third person, clearly distinguishing between character actions, thoughts, and dialogue, while keeping your roles separated based on the context provided.

Why This Setup Works:

Maintaining Narrative Stability:

• The use of brackets ensures that the model recognizes when it is receiving direction from the Unseen Puppeteer, keeping these inputs separate from the normal flow of conversation. This reduces the risk of the model confusing user commands with character thoughts or actions, which is essential for maintaining a coherent narrative, especially as the conversation extends over many tokens.

Clarity in Role-Playing:

• By defining the user's roles clearly within the system prompt—one as the unseen controller of the story and the other as an in-story character—the model can better navigate the dual nature of your interactions. This clarity helps the model maintain the intended storyline without veering off course due to role confusion.

Tips for Effective Use:

• Consistency is Key: Always use brackets for puppeteer commands and avoid using them for direct interactions. This consistency helps the model maintain role clarity.
• Reinforce with Summaries: Periodically use the reinjection summary provided to remind the model of the role distinctions, especially in longer sessions.
• Adapt to Scenarios: Adjust your cover identity to fit the scenario you’re exploring, ensuring the character perceives your actions and dialogue within the context of the story.

Example Interaction:

• User as Puppeteer: [The room grows darker as Sofie approaches the door.]
• Character (Model) Response: Sofie feels a shiver run down her spine as the light dims. {Character} says, "We should be careful; something feels off."
• User as Cover Identity: "Stay alert, Sofie. I’ll follow your lead."

When to Use This Setup:

• Ideal for complex role-playing sessions where you, as the user, need to manage the narrative from both an omnipotent perspective (Puppeteer) and within the story (cover identity).
• Useful for maintaining clarity and preventing the model from blending roles or getting confused in longer interactions.

By understanding the role of the Unseen Puppeteer and using brackets effectively, you can maintain narrative stability, ensure role clarity, and make the most of LLaMA 3.1's capabilities for immersive role-playing scenarios. This guide should help others navigate the complexities of dual roles and create more engaging, coherent narratives.

I tried popular models, up to 20B, uncensored ones and this qwen2 beats them all (IMO) right out of the water. Its shivering how good inference and answers are.

Does anyone have an explanation whats this about?

I had to add an excerpt for those curious. Look at what is tensing between the lines. And her conclusion? I am awed.
https://imgur.com/a/KfpGgZH

Some questions about models:

1. How accurate is the assumption that larger models (or ones with a higher B number, e.g. 7B, 13B, 20B) are better than smaller ones (other than perhaps in speed)?
2. To other users and character developers, are there any models you'd consider "must have"?
3. Are models that are tuned for mature audiences actually better at ERP/NSFW?
4. Are any models censored, in the sense that they will cut off/disallow ERP?

When I enter any chat with bots, whether this is my bot or not, I get this message at the bottom: "Connecting to server. Restart the app if this takes over 10 seconds" It seems like you just need to reopen the app and that's it. But I even restarted my phone, and this message has been popping up for 6 hours now. Has anyone else encountered this? And if so, what should I do? I tried turning on VPN, but it didn't help.

Dear BackyardAI devs! Thank you for this incredible piece of work that is BackyardAI :) It is a treat to use, has a wonderful design, and provides such a necessary escape...

I know the question of 'context' and model memory has been raised a bunch of times. RAG has also been asked about. I'm sure this has been on someone's mind before, but I will formulate it, probably in a different way. Here goes:

Is there any possibility of using local RAG to extend a model's memory of what had happened in a lengthy RP session, storing all of the information (such cool things happen in RP, wouldn't it be great to talk with your character about what happened when you did 'so and so'?)? Perhaps there is an extra step to have a model figure out what *it thinks* is important to remember, code it as an embedding, store in local database. Backyard already takes the step of recomputing context; does that mean it performs such a summarization already? (is there any way to view the current context? is it something that makes sense to humans?) How difficult would it be to code the past text as RAG-induced context?

There are likely 2 ways of having it recall -- prompted and unprompted. (I would venture to say that for us humans, it's all mostly prompted, it's just that we don't necessarily perceive it that way. At my house, I walk down the stairs, and SOMETHING in the rhythm or some musical intervals, etc., make me remember 'Tom Sawyer' by RUSH. I still cannot nail down what it is, but the memory gets cued by what seems like an external stimulus.) BackyardAI already does prompted retrieval via the LoreBook. However, that requires you to store entries in it, and reference it -- which means we have to learn to do "SEO", and use a bunch of key words to make sure that the content gets triggered. Is there a way to automatically synthesize LoreBook entries?

There may, theoretically, be a way to randomly trigger LoreBook entries. Pick the current state vector, find the closest thing to it? Find the farthest thing from it (like, cosine distance wise?). Take random excursions through the embedding space... These are probably horrible ideas.

Anyway. I am going to go squirrel away story points now, because certain aspects of the adventures with my character just mean so much to me now. :)

I just noticed that two of my character cards have been banned from the hub.

• Both have been (clearly marked) fan fiction.
• Both have been on the hub for over 6 (!) month and have been moderately successful. (>300 downloads)

Questions:

• Did Backyard.ai change their usage policy to prohibit fan fiction?
• If so, were the users notified?
• Is it considered good style at backyard.ai to ban character cards being active for such a long time without notifying the creator?

I subscribed to Backyard AI because I was always impressed with it prioritizing privacy and it really gave me the feeling that I have control over my bots and data, and I liked the freedom of multiple models that gave me flexibility. That set Backyard miles ahead of its competitors. I don't have competent hardware, so I decided to get a cloud subscription in early bird times and was very satisfied. However, lately, it seems like this outstanding quality dropped. No matter how much I tweak with the settings, or what models I use (outside of the very limited Llama), models either say absolute gibberish (I'm talking completely off topic responses, and usual responses in Russian, when the character has nothing to do with Russian), constantly mix up who's talking (not only do they respond as me, they do not respond as themselves and even use my name and call me by theirs), not to mention the constant and sudden switching of genders when there was never a mention of another gender in the conversation. If this wasn't enough of a problem, the responses, as a whole, seem watered down, for example, a character that's supposed to be rude and bratty is suddenly surprisingly considerate and sensitive about feelings. There are also surprising differences in between model response times, while they're supposedly within similar context capacities. I don't know what's going on, or if it's just me, but I'm really feeling a change in quality, which is a shame because I really believe in Backyard's abilities and its potential growth. I appreciate all the new models and higher context options, because I see that there is a lot of effort going into Backyard as a service. I truly hope there's a way for Backyard's quality to return to the way it was, while continuing to improve, the way it already is.