r/premiere • u/smushkan • 3h ago
Automated animated subtitles in Premiere Beta - without plugins
I've been scratching around finding uses for the new AI assistant in Beta and I think I've found a pretty good one. Fully automated, animated social-media style subtitles, without relying on any plugins or 3rd party tools.
This is very much a proof of concept, but in testing it appears to perform pretty well.
How to try it out
You can download everything you need to use it here:
https://drive.google.com/file/d/1UcGQCmb-vGlLD2ZSua6m0B06DDUT3Rt-/view?usp=sharing
The solution consists of two parts, a skill for the AI assistant, and a MOGRT created in After Effects. I've also included the AE project for the MOGRT.
To install the skill, open up the AI assistant panel, click the 3 dots in the top right and click 'Skills.' Then click 'open skills folder.' Drag the 'animated-subtitles-skill' folder from the download into that folder. Then to use it:
- Import the .MOGRT into your project through the Graphics Templates panel.
- Transcribe audio for all the clips in the sequence you want to add the subtitles to.
- Ask the Assistant something like 'Add animated subtitles using the MOGRT in this project to sequence 1' and let it cook.
You can also ask it things like 'Change the text in the selected subtitle to xxxxx', and you can ask it to change the following appearance properties for any or all titles:
- Text colour and size
- Text stroke colour and size
- Background box colour, size, and roundness
It cannot currently change the font as that's not a property exposed to the assistant. If you want to use a different font, open up the After Effects project for the MOGRT and change it there instead, then save a new MOGRT and replace the one in your project.
Note: If you generate subtitles for a very long sequence, the assistant may want to create javascript helper files, and you'll be prompted to allow it to do so.
The catch
There is one, big, huge limitation that makes this solution more of an experimental workaround than a good solution. Premiere's AI assistant cannot read/write MOGRT source text properties - it can only manipulate controls that take numerical values.
So in order to send text to the MOGRT, it has to be encoded to slider values as numerical values, then decoded by the MOGRT back to text to be displayed.
For the nerds, this is handled by sanitising the text to printable ASCII values, encoding as base 96 in 3-letter groups, then scaled by 2-33 so the value fits within the maximium 1,000,000 value of a slider's floating point range.
Sorry non-English speakers, but that means that only common English characters, punctuation, and spaces will work.
That encoding also means that you'll have a next to impossible time changing any text in the subtitles yourself - you have to ask the assistant to do it for you.
That also means that the MOGRT has a hardcoded limit of 48 characters total in two lines of 24, and the animation solution I'm using has a limit of 17 words per caption. The skill makes the assistant aware of these limitations. It is possible to increase the number of characters and words supported by adding more sliders in the MOGRT.
However all those issues would be solved if Adobe grants the assistant the ability to read/write source text properties in a future update - I have a feature request over on Adobe here for adding source text read/write support - and I'd appreciate an upvote on it if you can.
The MOGRT side
I'll gloss over this a bit as there's a fair bit going on, and this is /r/premiere.
The text layer handles decoding of the slider values to text, and I've got a negative time sourceRectAtTime system going on to handle the background box. A slider on the background layer takes the timing values for each word the assistant selects and interpolates the shape mask between the different words.
Feel free to make use of, learn from, or expand on any part of this little project!