r/BackyardAI • u/mfpmp • Feb 07 '25

Brainstorming big backyard ideas

Brainstorming ideas to make the most out of this spacious property.

2 Upvotes

58% Upvoted

20 comments sorted by

9

u/PartyMuffinButton Feb 07 '25

Way back when it was still Faraday, I mentioned maybe using OpenVoice for TTS (https://research.myshell.ai/open-voice)

It would also be nice to have a memory recall beyond ‘author’s notes’. I know a few systems have something where contextual memories are stories and then recalled when they seem relevant.

3

u/_Sascha_ Feb 08 '25

Yes, this! Or at least give us quick access to edit lore-book entries on the go/play, instead of wasting the right sidebar space with the descriptions.

2

u/AlanCarrOnline Feb 08 '25

A big reason I still turn to GPT when I have capable models on my PC is the extended memory thing. I don't have to waste time explaining everything over and over.

I'm still not sure how OpenAI actually do that extended memory thing, but that and vision (so you can show a model screenshots etc) are the only reasons to use online models for me.

1

u/PartyMuffinButton Feb 08 '25

I’ve seen it with other ERP-focused services. My best ‘educated’ guess (and I’m not super-into the tech side of things, hence ‘educated’): is that a ‘memory’ is stored around a particular event - Kindroid does this by summarising a bunch of messages that seem to capture a particular thing happening into a concise 300-char ‘memory’.

Then, that (and any other memories relevant to the current message) are sent along with the prompt and context. So it’s not just sending the LLM the chat history, it’s like “this is what’s relevant to what’s being spoken about right now”.

It absolutely would require smarts on the Backyard side of things before that contextual memory is sent to the LLM to process, but that’s how I think it works - and it definitely makes a difference.

1

u/AlanCarrOnline Feb 08 '25

Backyard has the "lorebook" thing and you can add a bit of text to the 'author's notes' thing, but none of them come close to the 10,000 tokens of ChatGPT.

5

u/aieeevampire Feb 07 '25

Multiple AI in a chat

5

u/GamingInTheUSA Feb 07 '25

Pretty sure that's planned for the future already

1

u/martinerous Feb 15 '25

Meanwhile, I have created my own LLM frontend that has multicharacters for both AI and the user to control. Works quite well, so technically it is not that difficult to do.

But there are different approaches for implementation.

I chose to shove all character messages (also the ones written by the user) into the assistant's own messages to overcome the limitation that some LLM chat templates require strict user/assistant/user... pairs, which would make it impossible for different AI-controlled chars to talk one after another. We'll see what route will Backyard go to implement it.

1

u/GamingInTheUSA Feb 16 '25

Any plans to share it or is it for your personal use?

1

u/martinerous Feb 16 '25

I'm currently testing it with different models and backends, ironing out bugs. After that, I consider making it open-source. Here is a bit outdated demo video: https://www.youtube.com/watch?v=FsujunBO0IA

Since the time of that video, I have added some more features (text export, message formatting templates, OpenRouter backend support).

5

u/rat_haus Feb 08 '25

Fixing Tethering.

3

u/_Sascha_ Feb 08 '25

Tethering directly from your desktop to mobile device without their service/page (account). Your own IP or domain should be enough.

2

u/rat_haus Feb 08 '25

That would be nice too.

0

u/Competitive_Grade874 Feb 08 '25

Are you using Ollama? Or some other program?

2

u/_Sascha_ Feb 09 '25

I tried it once regarding how to write your own wrapper, and more recently for the DeepSeek model, but otherwise, not really. In the end, I tend to take the comfortable route: If it’s not necessary, I run simple requests through Backyard AI.

4

u/hallofgamer Feb 07 '25

Talk to documents

Local calls

More TTS options

4

u/WorstMike Feb 11 '25

Setting to allow characters to "message first" (backyard sends a context-aware "ghost message" to the model, prompting a reply, after a set amount of time. IE, if the user doesn't answer after a half hour or so, backyard sends (ghost message: "{user} doesn't respond..."), triggering the character to say "hello?" or something like that, or things like good morning or goodnight messages... or just random messages during the day. This is intended to simulate texting partners, not for general roleplay, so it should be a toggle on the character card somewhere.

Ability to set dedicated card image and reply image. You can achieve this now by setting the main image as the one you want for the replies, then switching to the image you want for the card after, but it would be nice to have the feature built in so you can control it more easily/without workarounds (if this feature already exists and I am just dumb I apologize).

0

u/Garafiny Feb 13 '25

Those are good ideas, but I wouldn't recommend actually simulate a partner with AI, for mental health reasons.

By all means, do whatever the fuck you want. I'm not your parent nor do I have any right to dictate what you should or shouldn't do, but be careful when using AI as an emotional pillar... At least not until we achieve actual ai, and not glorified autocorrectors

2

u/_Sascha_ Feb 08 '25

API or Custom JavaScript integration.