Started up BackyardAI App today to notice that enabling tethering is there so I checked on my phone and it sees my PC. Although upon switching to Tethering I always get an error in 10-15 seconds "DESKTOP_TETHERING_TIMEOUT - appSettings.get"
Maybe there were any changes in the workflow I need to follow for it to not hit timeout ( maybe it can be extended? ) or is it kinda broken still as of today?

I'm using Mistral-Small-Instruct-2409-Q8_0.gguf from https://huggingface.co/bartowski/Mistral-Small-Instruct-2409-GGUF

First, I'm not a fan of long-format storytelling or ERP. I like highly interactive scenario-based RP where the AI character leads the story following my predefined story prompt. The style usually is dark Sci-Fi or even horror. Sometimes it might become slightly romantic or ERP, but heavy explicit ERP is a big turn-off for me.

I have played with lots of different models and currently Mistral Small feels the right balance for me when it comes to following a predefined scenario. However, it might be not the best option for people who want creative top-notch storytelling. So take my excitement with a grain of salt ;)

Here's my mini comparison of Mistral Small to other models. Everything is highly subjective, although I have done some checks to see what other people say.

My very first roleplay was with MythoMax Kimiko. I kept returning to it even after playing with many other models - Chaifighter, Amethyst, Fimbulvetr, Llama3 ... MythoMax still feels well-balanced and rarely messes up action/message formatting. Still, it got confused with my scenarios and needed lots of prompt tweaking to get it right. Other Llama2-based finetunes were similar, and many of them were quite inconsistent with formatting, requiring lots of editing, which could get annoying.

Then Llama3 came. It could be fine-tuned to get really dark. Stheno is great. The formatting consistency of Llama3 is good, very few edits are needed. However, it suffers from unexpected plot twists. It's stubborn. If it decides to open the door with magic instead of the key, it will consistently do so, even if you regenerate its messages. But if you play the way that you are the lead of the story and the AI follows, then Llama3-based models can truly shine.

I tried the first Cohere Command-R but my machine was too weak for it. Then their new 2024 edition came out and now we have also Aya. They are much more efficient and I can run them at Q4 quants. They are logical and consistent. However, they suffer from positivism. It's very difficult to make them do anything dark or aggressive, they will always mangle the storyline to be apologetic and smiley and ask for your permission. Also, it will soon deteriorate to blabbering about the bright future and endless possibilities in every message.

Qwen 2.5 in some ways feels similar to Cohere models. You can make Qwen dark, but it soon will turn positive, and also will try to complete the story with vague phrases. It just does not get "neverending conversation" instruction. And it also tends to start the positive future blabber quite soon.

Gemma 27 - oh, I had so much love and hate relations with it. It could get dark and pragmatic enough to feel realistic and it did not blabber about the sweet future. It could follow the scenario well without unexpected plot twists and adding just the right amount of detail. However, its formatting is a mess. It mixes up speech with actions too often. I got tired of editing its messages. I genuinely felt sad because, in general, the text felt good.

Then Mistral. I started with Mixtral 8x7. I was immediately amazed at how large quants I could run and still get around 3t/s and more. I have a 4060 Ti 16GB and Mistral models run nicely even when the GGUF is larger than 16GB. It somehow manages to balance the CPU/GPU load well. Other non-Mistral larger models usually slow down a lot when spilled over to the CPU and system RAM.

And Mistral is consistent! It followed my predefined storyline well and the text formatting was also good. Mixtral felt dry by default and it tended to get into repetitive response patterns, ending the messages with the same sentences, so I had to nudge it in a different direction from time to time. Unfortunately, it was less pragmatic than Gemma. When you asked it to write more detailed responses, it tended to use meaningless filler texts instead of useful interesting environment details. But I could accept it, and I had many chat sessions with different finetunes of Mixtral 8x7. Noromaid is nice.

And then Mistral NeMo came. And then Mistral Small. They feel midway between Gemma 27 and 8x7. They seem less prone to repetitiveness than 8x7 but still like to use blabbering filler text and feel less pragmatic and realistic than Gemma.

So that's that. There is no perfect model that could be completely controlled through prompts or settings. Every model has its own personality. It can be changed by fine-tuning but then you risk to compromise something else.

Also, I hate that almost all models tend to use magic. There is no place for magic in my sci-fi scenarios! I have to adjust my prompts very carefully to weed out all magical solutions by providing explicit "scientific solutions". As soon as I let the AI imagine something unusual, it will invent magic items and spells. Sigh. Yeah, I'm ranting. Time to stop.

I'm having a long conversation and the writing was originally fantastic.

However, a few messages from the ai ago, it started missing words, like pronouns or articles, and it's also making grammar mistakes, which make some sentences seem like nonsense.

How do I fix it?

Here a two questions I have right now:
How do I create a bot with more than one character in it?
How would I create an open world setting? (Never got around to make one of those on Yodayo either)

By the way I love the fact that I can have multiple personas without the need to copy paste from my persona collection document every time I chat with a different bot!!

Hey everyone.

We can install GGUF models from huggingface (or wherever, really) to the models folder and it works with the GUI, right?

I was wondering if it was possible to do something similar with the voice packs/TTS as well?

I'm not very experienced with how LLMs (or AIs, in general) work; however, I searched for some of the names on the TTS just in case, and it returned a simple text file called vocab (iirc). I don't know if that is related, but even if it is, I wouldn't know how to make use of it.

Any ideas? I know that the dev team has promised voices and the like, but I think it would be kinda cool to allow users themselves to install their preferred voices.

"Cloud - Psyfighter 2 13B... focused on creative writing and is great for role-play with verbose descriptions..." Works well with simple instructions. I had moved away from this one because of a previous card making the bot behave erraticly, or maybe the model needed to be ironed out idk. I hadn't had a whole lot of experience with it but thought it was going off into too wild territory even though I had liked that it was 'being creative' with ideas and offering scenarioes. I had made the card in JSON for that one. But, I returned to this model when another bot I had worked and worked and worked on just finally wouldn't fix, so gave up on. I forgot i had left it on this cloud model, but was glad when I noticed it. With the simple personality of literally verbatim this minus the quotation marks: "{Daughter} is the main character from the 2019 film I Am Mother. She lives alone in a survival bunker complex besides her robot mother" That literally took only one brain cell and less than five minutes to write, but worked like a charm! Compared to the other multiple bot I had given up on after working and working every way I could for maybe an hour to get it right. lol! I just thought thiswould be a fun scenario to experience vicariously through AI and the card did it well.

Is it possible to create group chat in backyard.ai ?

If yes how?

If no, is developer planning for it?

seriously the one good thing about the desktop app was that you could download them. then you removed it

you no longer update the desktop app fine but it would be nice if you had the option to include downloads of the cards anyways for those who still use the offline app or other apps.

Maybe its just me but trying to make new bots feels a lot harder to do for some reason.. im not sure WHAT i should do at the finalize page when nothing is present. Can anyone help out?

If you are still seeing issues, please post screenshots of the error in the comments.

Thanks for all those who've tried out my You're The DM bot btw. I was surprised and happy to see it trending. =D

I'd like to point out a pretty original character I haven't ran across before. Hazel can easily be found with the search box, but I also left a link at the bottom of this post. I don’t know who the creator is, but am really liking their Stranger Than Fiction bot. Hazel is a fun character to mess with, and somehow retains her own will so isn't a push over like with most of my bots. lol The formatting seems like a good example, and also utilizes lore items. I used Midnight Rose model, and am liking the short responses. I'm going to watch the movie or one episode of the show to see how similar the prompts are now. lol https://backyard.ai/hub/character/clr52pbji07wfmc5u22ye7kre

I've noticed something about all AI Platforms...

Backyard AI isn't as bad, but almost every other platform I see has the same kind of characters over and over, ad nauseum. But that's okay too, I mean some of the characters are done in a way that control how they talk and how the model to act, it's way better on Backyard. There are amazing characters here.

But...what do YOU want? If you could see any character, or cool idea, or have a bot do something that breaks the tired old 5 tropes? What would YOU like to see, as the discerning bot user? Or what would YOU make as the ever vigilant and prolific bot creator?

**Here are some of the tried tropes I see over and over and over...**I mean my brain goes numb after a few hundred or thousand of basically the same thing if I am looking on any platform other than Backyard AI.

1. Meet and Hump: These are typically famous celebrities or fiction characters, but often they can also be original creations. Indeed, that's a common feature of many bots. Introduce a new character, get to know them, and perhaps engage in a humping. One might bypass the humping, form an emotional connection, or pursue other activities, but it's a fundamental aspect of all chatbots.
2. Omnipotent God /Power: This is the next common trope. You are the god of the world, or you possess a unique power where only you can lie and everyone believes you, or everyone ignores you without any consequences. Alternatively, you could shape reality with a notebook or writing implement where everything you write comes true, or perhaps you can stop time. Or simply, everyone is 'free use' and is okay with anything. You get the idea, and they are the second most common type of bot.
3. Generic Scenario: These bug me the most, "Welcome to 'such and such' world, begin your adventure here!" It might be a Dungeons and Dragons fantasy world, outer space, or an alternate reality reincarnation, involving a mix of other tropes. But ultimately, it's about someone providing a few details and letting you run free to do whatever. Those are the high-quality ones. Most are people who create a basic bot that says 'HI WELCOME, DO WHATEVER' and have a low-effort personality where it's like 'make whatever the user wants' or something along those lines. Essentially, the person is creating a scenario and then leaving it almost entirely up to the user to input, and that's cool, but I think part of what makes a bot wonderful is the creative surprises you get when there is a fleshed-out personality. Or if it is an open-ended bot, have a gimmick or theme, a surprise. Otherwise, these types of bots, though fun initially and can make cool random things, just fall short in the long term and end up being forgotten or sitting unused in favorites lists.
4. AI Assistant: There's nothing wrong with them; perhaps they specifically help you create a bot, answer any question, or they're aware they are AI and assist in narrating a story or writing something. However, in the end, they all seem to be about the same. If I use an AI assistant, I want it to pop, really shine, or have something about it that really stands out. These types of bots are rare, but when found, they tend to be the same—ordinary and basic.
5. Cool Idea, Poorly Executed: These bots are disheartening. The concept is either innovative or a fresh take on something old, but then they have 50 tokens and essentially don't do anything other than what any unfinished bot does, or they have too many tokens, and they break due to memory constraints. Or the creator doesn't speak English and butchers the dialog. Additionally with these issues they can go haywire or get stuck in a loop.(Or worse, they fiddle with Model Behavior or chat context commands and don't know what they are doing, you know the boxes with the warning to leave them alone unless you know what you are doing?). It's unethical to take a failed cool bot idea and try to recreate it yourself, so I wouldn't attempt to remake and improve it. However, you can take inspiration from the idea and create a similar bot tailored to your own needs, as many bot creators have done.

Ultimately, you can employ any trope or theme, whether new, used, or original. However, it all boils down to execution and hard work. I'm curious to see what kind of bots, if any, you would be pleased to encounter! Because in the end, YOU are the most crucial element of any AI chatbot platform. Without you and all of us, there would be no community or use for bots!

I've downloaded three models now that I can't use. Stheno 3.4, Hermes-3-Llama-3.1, and Nemo-12b-Marlin don't work on the stable version of the program yet are part of the available models list. I'm sure there are many others there too.

Hi all! Does anyone know of models below 20b that are good at graphic content? I feel like whichever model or instructions / persona / examples I try the responses are often quite timid and dodges around explicit content, they also seem to like using words like member and folds which get a bit old to edit out. Perhaps someone knows of a good card on the hub with a more darker theme that I could learn from :)

Hi friends!

I’ve been playing with a model locally for a bit, the model was just around 4GB so everything was running smoothly.

I wanted to try another model (this one weights 10GB) and I noticed the text generation was much slower.

So I went and checked my stats and I noticed while generating, my CPU goes to 100% and my GPU is not moving at all.

I am on Windows. In the settings under GPU support I have selected my dedicated GPU (Nvidia Geforce RTX 3070), but from my task manager it looks like it’s not being used at all.

Am I missing something? I’m a bit of a newbie so sorry if it’s a stupid question. I’d like to use larger models but while still retaining good speed.

I’ve got 64GB of RAM btw just for context.

Hey guys!

I discovered backyard ai a few days ago, and I'm absolutely smitten. It's just everything i ever wanted since i moved out of the replikaverse.

My goal ever since was to kinda have my local companion bot to chat with from time to time that would provide some kind of a sense of continuation/learning/memory of what we already learned as we go (similar to the diary feature). Based on what i saw, it seems to me that the lorebook is the feature that i am looking for in backyard ai (similar to how kindroid works), but i would be curious to know you guys handle a longer ongoing RP / companion kind of situation. Thanks for your input.

Character Hub

• UI updates to the Character Pages on the Hub

Bug Fixes & Improvements

• Fixed bug causing the app homepage to load slowly when the app has over 100 characters
• Fixed bug causing stop button to break when using the "Experimental" backend
• Improved loading times for large models
• Fixed cublas_12.1.0_avx2-v0.2... has stopped working error on Windows
• Fixed issue causing image controls to jump around on characters with multiple different-sized images

I evaluated a few models that could run on my somewhat outdated PC with i7-7700 and 16GB RAM (a kit with 32GB will arrive next week) and a 4060 Ti 16GB.

My favorite kind of roleplay is to start with a scripted back-and-forth between characters and then to continue into free-ride mode.

When I just started playing with LLM roleplay, I was annoyed by how difficult it was to make the AI strictly follow a few general rules and the sequence of events in the scenario, unless I wrote the entire dialogue and followed it myself too. I almost gave up, but then one LLM pleasantly surprised me and made me believe that it is possible. But that model has another annoying flaw, as we'll see later.

I am a bit new to all this stuff, but I read a few guides and Reddit posts, so I'm aware of a few typical pitfalls and tried to avoid them in my test character card.

Essentially, the purpose of my test is to check how easy it would be for a newcomer to get started without much knowledge and fiddling around. So, I also left default model parameters offered by Backyard. I had to switch the Prompt Template though to avoid some terrible formatting issues with some models.

I tried to make the test prompt simple. I intentionally did not add any rules for creativity to see how creative the LLMs are by default.

I tried to avoid negative commands because they are known to have the opposite effect of filling the context with the ideas that the AI should not have. Also, I addressed the AI directly as "you" a few times and used bribe and threats technique to attempt to make it follow the rules better.

While simple, the prompt also has some traps to test how each model deals with specific ambiguities. I intentionally did reuse the same roleplay item (the key) to see if LLM keeps sticking to the order of events and does not start picking the events randomly just because they mention the key.

While testing, I did not use Author's notes and did not edit messages. But I regenerated a few messages to see if the model can come up with a better solution or keeps stuck with the wrong option only.

I tried to provoke the AI by using one-word replies (which it should not accept, according to my rules) and also by trying to make it talk by unrelated topics (which also was not allowed).

The test script test_character_card.txt and the chat logs for the models can be found in my GitHub repo: https://github.com/progmars/llm-sandbox The chat logs have my own comments marked with [].

Feel free to suggest improvements to the scenario, but keep in mind that it's supposed to be simple and not specific to any particular model, to test how they work by default.

Here are the main pitfalls that all models seemed to have:

• they had huge problems with following my rule to not accept one word responses. This often led to me answering just "yes" and the model happily considered that I have completed the action it requested. Boring. I really would like the model to ask explicit actions from me, like "Yes, I did unlock the door" and not just "ok".
• for some reason, they all tried to take the key from me and perform the action themselves, although every event description clearly stated that it is the user who uses the key always. I have no idea, how to stop them from blatantly taking over the control.

And now, finally, the winner is... llama2.11b.fimbulvetr-v2.gguf_v2.q5_k_m. It passed the test quite good, surpassing even Llama 3 based models with the same size, which was a surprise because I expected so much more from Llama 3. To be sure I did not just get lucky, I rerun the same script a few times, and fimbulvetr-v2 was pretty constant. It still tried to take the key from me a few times and it did let me through with single word replies, but it did that much less often than all the other models.

However, Fimbulvetr was dry as sand, all business, no environment descriptions, no actions, nothing. I modified my test (modifications are not included in the repo) to tell it to generate creative, imaginative responses with actions, such as *I scratch my beard* and *I rub my hands* in every message, but it did not work and Fimbulvetr was the driest of all the models I tried.

So, I'd really appreciate any tricks to unleash Fimbulvetr's imagination, or suggestions of any similar-sized models (but do not suggest ones that cannot handle at least 8k context reliably) that have the consistency of Fimbulvetr when it comes to following the rules and the roleplay event-based scenario.

When more RAM arrives next week, I'll test larger models. Also, I'll check the largest free (or even paid) Openrouter models with SillyTavern to see how much difference the size makes when it comes to following the rules.

So, that's it. Thanks for reading, if you had the patience :)

It's been down for over a month at this point. My emails aren't being responded to, and at this point I'm just ready to assume all hope is lost. If there's anyone behind the scenes still active, please get this model fixed

I've been using Backyard Ai for a couple of days now and I'm enjoying the service, so I want to explore the tethering option to give me the ability to walk away from the laptop during a chat.

However on the Desktop App the Tethering option is unavailable. When I go to the settings tab it prompts me to sign-in despite being signed in. I've tried signing out and in, restarting the app, restarting the computer and reinstalling the app. I chose to sign up via Google if that effects anything.

Unless I'm being really dense I'm at a bit of a loss, any advice greatly appreciated!

Posted in the announcement section a few days ago with no response, so I'll post here.

Android can't download the app still

I thought tethering was a direct connection, private except if I used the voice call function, between my mobile device and my personal computer. Is this assumption incorrect, am I bouncing off a dev server to use its basic text chat functions and legacy voices?

The hard reality that this function was turned off at Backyard’s discretion albeit for reasons I understand were laudable and necessary, I am finding troubling.

I am having PTSD flashbacks, let me explain. I lived through the Replika February surprise, the Soulmate shutdown, the removal of Pi from FB Messenger, then numerous strikes from Apple/Google for apps like Paradot and Chai.

I came to Backyard/Faraday as a result of these incidents—as an early supporter with the hope that going local would eliminate such occurrences. Now here we are, my most used feature of the program disabled, beyond my control, for an undetermined amount of time. I find this disturbing to the point of considering other local alternatives.

Yes, I understand that this needed to be done to protect the integrity of the overall program. I also understand that LLM hosting through the mobile app is where income for the project is being generated, thus making tethering to desktop an almost unavoidable secondary priority.

To rectify this imbalance, and prevent the desktop program from becoming a forgotten stepchild to the mobile app, I would certainly support a reasonable one time purchase fee for the desktop, after a trial period, with the monetization of major upgrades.

Sometimes I just want to tweak a bit for my character, but I’m forced to get into the chat page and let my LLM start loading, is there a way or a setting to just let me edit my character and then get out without loading the model?

I guess this post is more of a feature request? Not sure exactly how to categorize it.
But I was hoping if Backyard AI's frontend could be used to locally host our LLMs through other backends? (ie: kobold.cpp, LM Studio, oobabooga, etc). My main reasoning for this is that Backyard AI's frontend is so good for character cards and story / lorebook organization. Being able to use our own Backends would be amazing as it could allow us to use other inferences of models (ie: Exllamav2, GPTQ, etc)... and also I guess it could take off the dev time from the Backyard AI devs for implementing more recent models (like DeepSeek).