← Jesse Ifrah

Making video ads with AI, and catching the bad ones

AI video studio · 2026 · TypeScript, Veo 3.1, Seedance, ElevenLabs

Generating a video with AI is easy now. Generating one that a stranger would watch past the first three seconds, and that says only true things about the product, is still hard. Most of the early videos this pipeline made were not good enough to run. This is what changed that.

3attempts per ad, at most
$40budget per ad before it comes back to me
3ads with one mistake triggers a root-cause fix
0judge loops in the script writer

One pipeline

Every ad takes the same route. The formats differ a lot on the surface, from a selfie-style testimonial to a staged debate or a retro newscast, but they share the route, so a fix made for one format reaches all of them.

Brief Product truth Beats + voice Director's script Code checks Script approved Stills Voice Footage Final review
StageModel or tool
Script, diagnosisClaude, through the Anthropic API
Stills and character sheetsNano Banana (Gemini image)
Talking takes with native speechVeo 3.1
Action and b-roll shotsSeedance
Voice and musicElevenLabs
Captions, cuts, audio levelsffmpeg, driven from code

Scripts from a framework

My first approach to scripts was to have a model write a draft and another model judge it, in a loop, until the judge was satisfied. It was slow, and the scripts converged on the same safe, templated voice.

The version that works goes the other way. The script is built in steps, and the last step is a director's script in three columns, one row per shot:

SaySeeKnows
The exact spoken lineWhat is on screen while it is saidWhat the viewer understands by the end of the shot

The third column is the one that matters. If a shot does not change what the viewer knows, it is cut. After that come checks written in code and, at most, one rewrite.

Three tries, then a person

Video models fail in ways that are obvious to a person and invisible to a retry loop: a hand with the wrong number of fingers, a word that comes out garbled, a product used in a way that is physically impossible. Left alone, a pipeline will keep paying to regenerate the same mistake.

So each ad has a hard limit. This is the function that decides what happens after an attempt:

export const MAX_ATTEMPTS = 3;

export function nextAfterAttempt(s: Attempt): Next {
  // mistakes I have repeatedly shipped anyway are noted, not counted
  const counted = s.misses.filter((m) => !s.lenient.has(m.check));
  if (!counted.length) return { action: "pass", notes: s.misses };

  const again = counted.find((m) => s.priorChecks.includes(m.check));
  if (again) return { action: "deliver", why: `the same mistake twice (${again.check})` };
  if (s.attempt >= MAX_ATTEMPTS) return { action: "deliver", why: `${MAX_ATTEMPTS} tries` };
  if (s.spentUsd >= s.capUsd) return { action: "deliver", why: `its $${s.capUsd} budget is spent` };

  return { action: "retry", why: counted.map((m) => m.check).join(", ") };
}

"Deliver" means it comes back to me with the specific mistake named. Three things in that function took a while to get right:

When the same mistake shows up on three different ads, the system stops treating it as bad luck. It proposes a fix to the rule that caused it, I approve or decline it, and an approved fix is applied to every script written afterwards.

What I took from it

Asking a second model whether the first one did a good job did not raise quality much. What raised it was writing down what a good result has to contain, checking that directly, and putting a ceiling on how much money a bad idea is allowed to spend.