An Automated Video Editing Pipeline: Captions, Voiceover and Motion

How to read the labels: first-party = official pages, repositories and registries I fetched myself; inference = my own judgment; not verified = explicitly not done. This article is a design proposal, not a test report — I did not run the whole pipeline on this machine (reasons at the end). The tool comparison extends the previous post, and the two pieces reference each other.

HyperFrames vs. Remotion: the licence gap is an order of magnitude

Read this section first. It decides whether you can turn a product into a business.

I verified both projects first-hand (first-party, 2026-09-16):

• HyperFrames: repository heygen-com/hyperframes, Apache-2.0, 50,496 stars / 4,607 forks, created 2026-03-10 and still receiving pushes that day; its self-description is “Write HTML. Render video. Built for agents.”
• Remotion: the official pricing page (marked as updated 2026-09-15) spells out the free tier, the Automators tier at $0.01/render + $100/mo, the Creators tier at $25/mo/seat, and Enterprise from $500/mo.

Four dependencies that bite

This part is independent of your stack choice. Anyone can step on it.

Incidentally, the previous post's conclusion holds here too: Hypit itself is modified Apache-2.0 that forbids multi-tenant services and commercial redistribution, while the HyperFrames underneath it is clean Apache-2.0. If you are building a product, go upstream.

Seven stages, each with a verifiable exit

I designed this chain as seven stages, each with explicit inputs, artifacts, and acceptance assertions. The point of the assertions is that when something breaks you can localise it to a stage instead of re-running the whole thing.

Caption readability: use first-party standards, not community folklore

Every number in this section was verified against first-party documentation, with the original wording kept.

BBC Subtitle Guidelines (portrait values are official)

First-party — I pulled the official BBC page and checked the raw HTML line by line. The changelog for that version explicitly notes newly added "size and position guidance for 9:16 portrait".

Two key sentences from the BBC, quoted verbatim so my paraphrase cannot distort them:

"As a guide, the equivalent to 37 characters in a 75% width region of a
16:9 (landscape) video is 25 characters in a 90% width region of a
9:16 (vertical) video."

"For 9:16 video in portrait or vertical mode, this is reversed: subtitles
should not be placed outside the central 75% vertically and the central
90% horizontally."

On reading speed, the BBC gives 160–180 words per minute (that is 0.33–0.375 seconds per word).

Engineering consequence: converted for 1080×1920, a 90% horizontal safe area gives x ∈ [54, 1026] and a 75% vertical safe area gives y ∈ [240, 1680]. Those two pixel values go straight into your QC assertions (inference, derived from the BBC percentages).

Netflix Timed Text Style Guide (Simplified Chinese has hard constraints)

First-party — I fetched the Simplified Chinese specification itself. The rules most relevant to Chinese speech:

• 16 characters per line, at most 2 lines (SDH may stretch to 18 characters).
• Reading speed capped at 9 characters/second (7 for children; 11 for adult SDH).
• "Do not use commas or periods. Use one single space instead." — the original wording, and it runs against Chinese writing instincts, so it is easy to trip over.
• Ellipses use U+2026; U+22EF is not supported. No italics. Numerals half-width, and spell 1–10 as Chinese characters where possible. Weekdays must not use Arabic numerals: 星期2 is wrong, 星期二 is correct.
• Line shape prefers an inverted pyramid — short on top, long below — avoiding a top line with only one or two words.

A boundary note: Netflix's rules are written for delivery to Netflix, not as a universal platform standard. Using them as thresholds is "borrowing a stricter industry standard" — safe, but be clear it was not designed for short-form video. inference

Word-level alignment: text first, then synthesize the speech

Of the seven stages, this is the one most often underestimated and most capable of ruining how the finished video feels.

The core design decision: make alignment happen against known text, instead of asking ASR to guess the text.

Usable forced-alignment tools: WhisperX (wav2vec2 phoneme alignment on top of Whisper; the community default), Montreal Forced Aligner (the academic standard; accurate, but you must supply a pronunciation dictionary and acoustic model), and ctc-forced-aligner (lightweight). Third-party sources — the positioning above comes from their official repository descriptions, and I did not run an accuracy comparison.

For the output contract, pin down a structure with few but strict fields:

{
  "words": [
    { "w": "rewrite", "start": 1.240, "end": 1.605, "conf": 0.94 },
    { "w": "the",     "start": 1.605, "end": 1.720, "conf": 0.88 }
  ],
  "audio": { "path": "vo.wav", "sr": 48000, "lufs": -14.2, "truePeak": -1.4 },
  "scriptHash": "sha256:..."
}

Motion must be driven by frame number, not by wall-clock time

This is the line between "can you make a video" and "can you make a reproducible video".

Video rendering is, at heart: evaluate frame n once and get one image. So the animation function must be a pure function: frame → picture.

// Correct: driven by frame number, same frame always gives the same picture
const t = frame / fps;
const y = interpolate(t, [0, 0.6], [40, 0], { easing: easeOut });

// Wrong: depends on wall-clock time, so the same frame differs on fast and slow machines
const t = (Date.now() - startTime) / 1000;
const y = spring(t);   // not reproducible, and it can drop frames

Why requestAnimationFrame is wrong in video rendering: it is bound to the real clock, so if one frame takes too long during rendering the animation state jumps ahead, and the same source produces different pictures on different machines. That makes visual regression testing impossible and makes the diff from "change one line of dialogue" untrustworthy. inference (this is the general engineering consensus on render determinism)

The two animation paradigms each have their place:

• Timeline-driven: right for title cards, transitions, decorative motion — it only cares how much time has passed.
• Event-driven (word anchoring): right for caption highlighting, emphasis, B-roll cuts — it cares which word is being spoken. This is the highest-yield category for talking-head video, because it means "change the line" no longer requires "re-time everything".

The Hypit teardown in the previous post turns exactly that second category into a language-level primitive (the attachment polarity of @name / @name!). You can borrow the idea without its whole framework: define anchors over your word-level JSON, and have motion subscribe to anchors rather than to seconds.

Cache keys: what decides your monthly bill

This section is the economics of the whole pipeline. There is one goal: when you change a line of dialogue, re-run only the stages that must re-run.

The method is content addressing plus per-stage keys. Each stage's cache key is made of every effective input to that stage:

Key design rule: put the cheap stages before the expensive ones, and keep the expensive stage's inputs as few as possible. Stages 4 and 5 are pure local computation, so changing them a hundred times costs nothing; stage 2 is where the money goes. So:

• Do not put style parameters into stage 2's key — changing a caption color should not regenerate the voiceover. This is exactly the value of separating .svs (appearance) from .svml (content) in the previous post.
• The model version must go into the key — a vendor silently upgrading its model makes identical input produce a different voice, and that is the most insidious source of non-reproducibility.
• Rendering must support frame ranges (like Hypit's start-frame / end-frame-exclusive), so a review pass can render only seconds 8–12.

Six failure modes you will actually hit

One caution on loudness targets: different platforms use different values, and I did not verify each platform's official wording one by one, so no specific numbers are given here. Go by the official documentation of the platform you actually publish to, and make it a configurable setting rather than a hard-coded constant. not verified

The smallest viable stack, and what it costs

Cost structure (inference, dependent on your volume): software licensing can be $0 (an all Apache-2.0/LGPL/BSD combination). The real costs are ① metered TTS, ② your own machine or cloud compute, and ③ if you pick Remotion and your team is ≥4 people, a continuing monthly or per-render fee.

I did not test this pipeline

• This machine has no ffmpeg, no TTS and no alignment library. I tried to install them but the sandbox does not permit writing to system directories, so I ran none of the stages. This article is a design, not a test report. not verified
• I did not benchmark WhisperX / MFA / ctc-forced-aligner against each other; I only describe where each sits.
• Loudness targets were not verified per platform, and I deliberately omit specific numbers.
• Remotion's multi-tenant details are quoted from the v5.0 Terms that the vendor marks as "Upcoming"; I could not fetch the text of the version currently in force, so confirm your use-case boundary with the vendor in writing. not verified
• Coqui XTTS's licence page returned 404 when tested, so the terms cannot be verified, which is why I do not recommend depending on it.
• All prices in this article are a 2026-09-16 snapshot. Licence policies change; check the official pages again before you commit.

The method here is "first-party evidence where it can be verified + clearly labelled inference + an honest list of gaps". Treat this design as a starting point, not a conclusion.