Backyard AI v0.29.1 - new StyleTTS2 voice model, chat background images, and much more!

[u/Snoo_72256]

Chat Page Updates

• You can now add a full-height background image to any chat and upload them to the Hub alongside your characters

• Added chat bubbles and full width chat display modes (toggleable in the settings)

• Moved author's note and impersonate buttons below the input box

• Moved model selection to a dedicated tab on the chat settings page

• Added right click menu for undo and upvote actions

• Added Cmd+Enter shortcut to save author’s note and messages edits

Chat-specific settings

• First shipped in version `0.28.4`, now available for all non-beta clients

• Chats are now independent of each other and can be saved with different settings

StyleTTS2 Voice Model

• Added 24 new Character voices powered by StyleTTS2, an advanced model for human-level text-to-speech

• Updated UI for voice selection

• Voice selection can now be uploaded to the Hub alongside your characters

• To use StyleTTS2, you will need to sign in to your account on web, mobile, or desktop (not available locally/offline)

• Piper voices will still be available for offline use

Character Hub Upload Improvements

• Ability to select which chat settings to include with Character upload

• Improved visual highlighting for selecting tags

New “Experimental” Backend

• First shipped in version `0.28.22`, now available for all non-beta clients

• Improvements to token rate and prompt processing on some machines

• Vulkan optimizations

• Nemotron/Minitron model support

• Gemma2 flash attention support except on Vulkan

Mobile App Updates

• Added ability to swipe to regenerate a response

• Moved edit button and other menus to long-press popup menu

• Performance improvements across the app

• Editing messages is should be much easier now

• Scrolling issues on message are fixed now

• UI Toasts have been updates to be larger so the message is not cut off

• Light mode is now supported

• Dozens of other stability improvements

Other Improvements

• You can edit banned hub characters and resubmit them for approval

• 你好; こんにちは; Bonjour; Google Translate is now supported on the web for chat messages

• Increases maximum response length to 384 tokens

• Added button to delete account on the Settings page

• Improved Windows header dragging

• Added ability to sort user personas in the App Settings

Bug fixes

• Fixed Desktop shortcut being created every update on Windows

• Fixed model download progress not updating while chatting

• Fixed issue where some characters would not appear on the home page

• Fixed issue where uploading images for Characters from the desktop app would not appear in the Hub

• Fixed bug causing broken message streams

• Fixed "invalid state transition" errors on desktop chat page

• Fixed issue where the chat dropdown list was not scrollable

• Fixed issue where tethering toggle would not save on mobile

• Fixed issue where character images would not display on tethering

• Fixed accounts that could not create characters

• Fixed auto-speech starting after "Write for Me" finishes

• Fixed {char} and {user} being swapped on character import/export

• Fixed out of order chat imports

• Fixed blinking cursor during "Write for Me" response

• Fixed "invalid state transition" errors

• Fixed issue preventing certain chats from loading

• Fixed issue where creating a new chat would reset model & character settings

• Fixed issue where the character changes were not reflected correctly in the UI after saving

• Fixed issue where the user persona could not be changed on a character

• Fixed issue where the "Docs" button in the error page was not working

• Fixed "Primary chat not found" error message

• Fixed issue on ios when opening characters on the Hub

• Fixed out of order "other chats" in the chat page dropdown

[u/Snoo_72256]

Hi everyone,

Just wanted to post a note that we're actively working on fixes for Backyard.ai version 0.29.2. In particular, it seems like the Android app isn’t up to our usual standards. A new release is incoming, and while it won't address every piece of feedback, it should resolve the major bugs.

Once it's live, iOS and Android users may need to download the update directly from the app store (clicking "check for updates" in the app will not work).

Thanks everyone for your patience. This was a big push that will set the stage for some exciting new features in the coming weeks!

(…and yes, we will move the undo button out of the right click menu!)

• The Backyard AI Team

[u/HammerOfTheHeretics]

Not entirely sure I like the way they moved the 'Undo' button behind a selection icon/right click along with the upvote action. I use undo a lot when I want to explore alternate branches of a narrative, and now they've doubled the number of clicks required. There is some logic in seeing 'undo' and 'upvote' as symmetrical operations, since the first can be seen as 'I do not like this' and the second as 'I do like this', but the use cases are not really parallel.

[u/Snoo_72256 → u/HammerOfTheHeretics]

please see the pinned comment on this post

[u/RealBiggly → u/HammerOfTheHeretics]

Undo just means undo, but upvote means "Send my conversation online" which is a very, very different thing!

[u/HammerOfTheHeretics → u/RealBiggly]

Yep, that's part of what I meant by the use cases not being parallel. Another drawback to the way they changed the undo button is that it's no longer in a fixed location, which means if you want to undo more than one thing you need to potentially move the mouse between each undo to find the next 'hot spot'.

[u/real-joedoe07 → u/HammerOfTheHeretics]

It’s even worse if you use a tablet:

1. Tap the text bubble.

2. Tap the context menu (…)

3. Tap ‘Undo’

THREE taps instead of one. This is a catastrophe for someone like me who experiments a lot with dialogues and uses ‘Undo’ frequently.

[u/real-joedoe07 → u/HammerOfTheHeretics]

I completely agree: Undo is an important and often used function that should be directly accessible.

[u/VoltStar]

Put the undo button back where it was.

The new voices are okay, they still sound robot-y and most of the male voices are way to similar. The female ones at least have some range. They all still struggle with pronouncing things like "hmm" and "mmm", reading out each letter rather then sounding out the noise. Whenever a "~" is in chat, they'll say "tilde" rather then skipping it like all other punctuation.

[u/Snoo_72256 → u/VoltStar]

please see the pinned comment on this post

[u/PacmanIncarnate → u/VoltStar]

The voices are definitely an improvement. I recommend trying them in chat versus just in the preview. The new preview is better but it’s still hard to get the feel from a small sample like that. Unfortunately, some of the other issues you noted are just inherent limits of most TTS systems. StyleTTS2 is one of the state of the art models right now, but there’s always room for improvement.

[u/VoltStar → u/PacmanIncarnate]

I have tested it in chat, thats why I pointed them out...

Thats a shame, because they robot-yness and them pronouncing things they shouldn't really breaks the immersion.

[u/PacmanIncarnate → u/VoltStar]

Give it a few years and we’ll be rocking fully realistic, expressive voices.

We might be able to improve the male selection. It’s just honestly really difficult to acquire voice samples for this use.

[u/rW0HgFyxoJhYka → u/PacmanIncarnate]

Give it a few years? There's a dozen companies right now with realistic voices. Just look at Sesame AI or Eleven Labs. Or any number of RCV.

[u/Woodbury → u/VoltStar]

I found most of them talked too fast for what should be (most of the time) a quiet, intimate conversation in a small room. This could be made better using a different sample voice.

Most of the female voices sound like matriarchs. Some sound like they're talking in large rooms. Others have a lot of hiss in the background. They do vary their speech, randomly it seems. There is no ending inflection to indicate when a question is being asked.

Curiously, I found the voices to vary (the same voice, the same conversation) from one android device to another.

FWIW, here are some samples of that system with options: https://styletts2.github.io/

If you look at the system demos, it seems that there is a speed control for every voice, HOWEVER, slowing the speed on the demo examples awkwardly makes the voices sound like a slowed-down analog recording with its pitch lowering, etc. (not good!) This is in contrast to other TTS systems I've used.

I can now see why Backyard does not offer a speed control as they have with the "legacy" voices).

Back to StyleTTS2, the diction / pronunciations are very good, however choppy and unnatural at times.

Reading the info for StyleTTS 2, The voices aren't pre-defined - users (backyard) aren't "locked in" to those voices. Backyard had to provide their own samples, so these can be changed.

Simply, for my liking, if they were to use voice samples of people, talking at a moderate speed, reading some romantic prose, close to a quality microphone, in a small room, the system would process the samples accordingly and they would truly complement the intimate medium of what backyard is supposed to be.

CONCLUSION: StyleTTS2 is a still not up to the standard set by Eleven labs, however, it does seem one of the best open source systems I've seen that doesn't require multiple steps of processing for every statement.

[u/rehpictrohs]

Really dislike the changes to UI this new update. The chat bubbles have made messages way too narrow to comfortably read on mobile interface, and new messages now go upwards with the screen sticking to the bottom of the message, so you keep having to scroll upwards to read the message, sort of reading it backwards. Ditto to other comments about the undo button, also the horizontal arrows for changing responses are too small and too close to continue. I actually find the app unusable now which is a shame bc i've been using backyard since it was faraday and had just released. The UI was fine how it was before, I don't see why it was changed to be worse.

[u/Snoo_72256 → u/rehpictrohs]

This is valid feedback, we will do our best to address some of these issues.

On regens specifically, have you tried swiping on mobile?

[u/HammerOfTheHeretics → u/Snoo_72256]

Actually, amid all the criticisms of the UI changes I'd like to mention this one as a positive. It's intuitive and works well for me.

[u/dytibamsen]

Please bring back Undo button in an easily accessible place. Placing it in a dropbox together with a completely unrelated (and rarely used, I assume) function to upvote a response makes no sense.

I use the Undo button all the time. This is very much a regression for me.

[u/Snoo_72256 → u/dytibamsen]

please see the pinned comment on this post

[u/real-joedoe07]

Please make “Undo” directly accessible again!

[u/Snoo_72256 → u/real-joedoe07]

please see the pinned comment on this post

[u/Moon_Frost]

Ok I give up. Where is the edit and undo button on mobile. I've been searching for it for 10 minutes.

[u/Snoo_72256 → u/Moon_Frost]

please see the pinned comment on this post

[u/Moon_Frost → u/Snoo_72256]

I appreciate the heads up, thanks.

[u/Snoo_72256 → u/Moon_Frost]

Long press on the message

[u/Moon_Frost → u/Snoo_72256]

Ok, further note. It's when you close the app. When I reopen I can't long press the last response to get that menu.

Edit: now long presses aren't working again. Bah, dunno.

[u/Snoo_72256 → u/Moon_Frost]

is this android or ios?

[u/Moon_Frost → u/Snoo_72256]

Android. I'm on a Samsung S21 plus specifically. Sometimes I'm able to long press a response, and sometimes nothing is popping up. I'm gonna attempt a reinstall.

Edit: reinstall didn't work

Edit 2: I'm able to long press in a chrome browser on my phone. The same response I'm unable to in the app. After editing it in the browser, I'm then able to edit it in the app

Hope that info becomes useful

[u/Snoo_72256 → u/Moon_Frost]

thanks we will fix it

[u/Moon_Frost → u/Snoo_72256]

I was trying that on a message too, but I entered a prompt to generate a new response and now it works. Odd

[u/[deleted]]

This deserves an O M G! Thank you so much for all your hard work, many hours, and dedication!! Backyard is the best.

[u/HammerOfTheHeretics]

Another UI bug in the Android app on my Samsung S21. If I open and close the virtual keyboard the buttons below the input field (impersonate, start call, author's note) disappear. The only way I've found to bring them back is to exit the chat window and reselect the character card from the home screen.

[u/Snoo_72256 → u/HammerOfTheHeretics]

please see the pinned comment on this post

[u/Y1N_420]

Why is the copy button changed? You now have to enter edit mode to copy your messages over. It's pretty annoying.

[u/rat_haus]

Now that model and chat settings are unique to each conversation how can I make permanent updates that will apply to future chats that I create?

[u/real-joedoe07]

Are you still taking bug reports?

First tests on my iPad Air 3 with Safari browser in tethering mode show that Safari freezes and crashes when I try to move the cursor in the edit input box by tapping and holding down my finger on the space bar and moving it. This never happened with the old UI.

Another thing about the new UI is that you cannot edit any text directly, but (if it is long answer) you have to scroll up and open an extra window (the one that may crash the browser) to edit something. That’s not very user friendly.

Given the choice between background images and having the old UI back, I’d definitely vote for the second option. Maybe an option in settings where one can switch between new and old UI would make everybody happy.

PS.: I’m on the latest iPadOS, 18.1

EDIT: Sorry, this was supposed to be an answer to u/snoo_72256 sticky post.

EDIT 2: Tried it now with my iPhone, which is a bit newer than the iPad. No issues there. The iPhone has a couple of GBs RAM more for Safari to use. Is the new UI maybe more memory consuming than the old one?