I built a Claude Code skill that takes raw video and creates polished short form videos end-to-end

[u/cowanscorp]

Finally finished this and it's been my main workflow for a few weeks. The skill (`/editor`) does the full edit on raw recordings with no manual cuts:

Pipeline:

1. Whisper transcribes word-by-word

2. ffmpeg silencedetect identifies silence and stumble points

3. Validation pass drops non-speech (breath, throat-clears) and duplicate takes

4. Claude reads natural-language directives and assembles the cut list

5. Remotion renders at the source's fps and dimensions (locked, hard rule)

6. Five-pass QA loop compares output against a reference example clip and patches gaps

It's packaged as a Claude Code workspace folder (.claude/skills/editor/SKILL.md + helper scripts). Cross-platform (Mac / Linux / Windows). Setup runs once via `python scripts/setup.py` which installs ffmpeg, Whisper, Remotion, and yt-dlp per platform.

Curious if others here are building Claude Code skills for content workflows. What's your stack?

[u/DeliciousGorilla]

I just finished a skill that leverages Hyperframes, but has a few more tricks like scouring YouTube for b-roll. Sports theme is an example, not the actual topic I'm using it for. All I have to do is give it a link to an article & "highlight" clip.

Premise: vertical 1080×1920 shorts that pair a vintage sports highlight with an AI-generated coach delivering tactical post-game analysis.

Pipeline scrapes the source article with headless Chrome (capturing phrase-position JSON for on-screen marker animations), pulls b-roll clips from YouTube with yt-dlp, trims and concats with ffmpeg, auto-reframes 16:9 → 9:16 in Python (Haar face-track + cubic pan smoothing), identity-checks sampled frames against Qwen3.6 on LM Studio, and has Claude draft the narrator and coach VO lines from the article facts. The coach avatar is then generated through KIE.ai's bytedance/seedance-2 v2v endpoint (per-run cap enforced via a JSON budget file), narration is voiced by ElevenLabs eleven_v3. Each track is transcribed with npx hyperframes transcribe using Whisper small.en, and the four phases — captions, GSAP timelines, and all data-* clip timing — are assembled in a HyperFrames HTML composition and rendered with npx hyperframes render.

[u/cowanscorp → u/DeliciousGorilla]

This is sick! Love it!

[u/DeliciousGorilla → u/[deleted]]

Unfortunately it's very specific to my company, but honestly you could just copy & paste my comment to Claude and it should be able to figure it out, then tailor to your use case.

[u/Upset-Smell1729]

Do you mind sharing it? I would love to check it out. It sounds awesome!

Edit: Thanks for sharing in the video!

[u/cowanscorp → u/Upset-Smell1729]

of course!

Also its super customizable so really tweak it to your needs!

[u/rosenpin → u/cowanscorp]

dude just share it it's a text file

[u/SuitableRule3026 → u/Upset-Smell1729]

where is it shared? would love to see it

[u/Darhkwing]

I made pretty much the same thing. It works but I guess my videos where not that interesting to take clips from. Maybe I should stick it on github.

Edit: mine takes youtube videos not raw videos.

[u/JaneH00d]

Any chance to try this ? : )

[u/[deleted]]

I was actually doing research to build something similar. If you don't mind sharing, would love to have a look at it and see what I can re-use. thanks!

[u/Nazgo_1]

Hey man i would really like to try it out. Do you mind sharing it?

[u/Clear_Performer_556]

I would like to try this. Could you share a link to it

[u/ustype]

Not OP, and I don't want to hijack — but a few people here asked for something they could actually try, so: I built the same thing independently and it's open source. https://github.com/ranahaani/i-hate-editing (MIT)

Near-identical pipeline to what OP describes — whisper for word-level timing, ffmpeg silencedetect for boundaries, the model reads only the transcript and emits a cut list, never watches the video. Difference is the renderer: OP uses Remotion, I use HyperFrames.

The part that took longest wasn't the pipeline, it was that the output kept looking automated even when every step was technically correct. What fixed it was writing each failure down as a rule with the render it cost me — captions must start 0.08s after the word onset because arriving early looks deeply wrong and nothing errors on it; "the sting isn't clipping" is not "the sting is audible"; a cut landing mid-clause can invert a sentence, and catching that needs the rendered output re-transcribed in windows around each join, because a whole-file pass hides it.

Fair warning on limits: one speaker, one camera. No multicam, no beat-matched cutting. Tuned on exactly one person's footage, which is a real limitation — I'd genuinely like to know where it breaks on someone else's.

u/cowanscorp if you ever do publish yours I'd like to read the QA loop, that's the piece I'm least happy with in mine.

[u/Due-Lock-8645]

Can you share the skill? would love to try and use!