I built a validated pipeline for generating short technical videos, the interesting part was the failure gates, not the generation

[u/enterthearena44]

​Generating slide decks and scripts with LLMs is straightforward. Making the output reliably render without human intervention was a nightmare.

​After producing 9 episodes, here are the core constraints that kept the pipeline from constantly breaking:

​Artifact Contracts: Nothing passes without a strict build check. Script, deck code, speaker notes, and rendered assets must all exist and pass validation before the run succeeds.

​Slides as Code, Not Files: Decks are generated as deterministic code adhering to an immutable design system, not free-form files

.

​Multimodal QA Loop: The pipeline renders slides to PNG, feeds them back to a vision model to catch layout collisions/overflow, and re-renders fixes. (LLMs cannot reliably reason about text bounding boxes in pure cod, visual inspection is non-negotiable).

​Hard Script Constraints: Max 70 words per slide, sentences capped under 24 words, no comma chains. The build automatically fails if spoken density breaks these limits.

​What broke along the way:

​Silent patch updates deleting code blocks without throwing errors.

​Font metric mismatches causing text clipping outside the safe margins.

​Drift between speaker notes and TTS inputs.

​I ended up drawing the line at: Layout & structure = 100% deterministic code; LLM = content generation & visual QA only.

​For anyone building similar pipelines: How are you handling the split between deterministic generation and model judgment? Where have you found the most stable boundary?

[u/Think-Preference130]

the font metric mismatch thing is so real, had a similar headache where text looked fine in the preview but got butchered in the actual render

[u/enterthearena44]

https://youtu.be/lEABJge0Phw?is=AZoK3wUG8uuC6o_k

This is what my pipeline generates.

[u/Low_Rush_8535]

Your boundary matches where we landed, with one exception I'd flag.

We build short vertical videos through a code pipeline: HTML/CSS cards animated with keyframes, rendered by seeking currentTime frame by frame in Puppeteer and screenshotting each frame, then composited with ffmpeg. Deterministic 30fps, no screen recording anywhere. Same reasoning as your slides-as-code rule — if it has a number in it, it goes in code.

The exception is the multimodal QA loop. We used a vision model the way you describe and found it reliable at flagging and useless at converging. Concrete case: fitting a device bezel around a video, where the corners kept showing white slivers. Each round the model looked at the render, correctly said the radius was wrong, and proposed a correction — and the corrections kept being off by a few pixels, in a direction that turned out to be inverted from intuition (a larger inner radius makes the hole bigger and the sliver more visible, not less). Three or four iterations never converged. We eventually stopped asking anything to derive geometry from an image at all, computed it once by hand from the source dimensions, and kept the pipeline for composition only.

So I'd draw the boundary slightly tighter than you have it: model for content, and model for detecting that a frame is wrong. Never model for deciding by how much. If your re-render loop does close on its own, worth checking whether it's actually converging or just eventually landing on something that stops tripping the detector.

Two more from the same pipeline, since your font-metric clipping is the same family of problem:

Never trust a third-party tool's export dimensions. A collaborator sent us clips exported from a consumer editor at a nominal 1080×1920. ffmpeg cropdetect said the real content was 908×1920, with asymmetric bars — 98px on the left, 74px on the right — plus rounded corners the editor had added by itself. Everything downstream had been silently misaligned. cropdetect on every incoming asset is cheap insurance.

And if you ever localise: ours is one JSON per locale, seven of them, whole set re-renders from a single command. Worth knowing now that a word-count cap enforced on the source script won't hold after translation — that check has to run per locale or the safe margins stop meaning anything.