← Jesse Ifrah

A system that tests ads and scales the winners

Creative testing system · 2026 · TypeScript, Postgres, Next.js

Most ad accounts are run by feel. Someone looks at a dashboard, decides an ad "looks bad", turns it off, and makes five more of whatever they liked last week. I wanted the opposite: a loop where every decision can be traced back to a rule that was written down before the ad ran.

The system generates Meta ad creative, reads how it performed, turns off what loses and multiplies what wins. It runs every morning as a fresh process and remembers nothing from the day before. Everything it knows is in the database.

589TypeScript files
106SQL migrations
73test files
1action it may take without asking

A pipeline, with a model at two points

It is tempting to call this an agent. It isn't one. The order of operations is fixed. Claude is called for two jobs only: working out why an ad failed, and writing new creative. Whether an ad is judged, kept or turned off is decided by plain code.

Pull Meta insights Threshold check Judge against the winner Kill losers Roll up the angle Brief the next batch Human approves Render Upload paused

The filled step is the only one that changes anything on its own. The dashed step is a person, reached through a Telegram bot that sends the morning report and waits for a yes.

Thresholds before judgement

Every ad launches with two conditions attached: a spend threshold and a time threshold. Both must clear before the ad may be judged at all. An ad that has spent enough but only run for a day is still untouchable, however bad it looks.

export function thresholdProgress(ad: Ad, metrics: AdMetric[], asOfDate: string) {
  const spendToDate = metrics
    .filter((m) => m.date <= asOfDate)
    .reduce((sum, m) => sum + m.spend, 0);
  const daysElapsed = daysBetween(ad.launched_at, asOfDate);

  const spendMet = spendToDate >= ad.spend_threshold;
  const timeMet = daysElapsed >= ad.time_threshold_days;

  // Both. Not either.
  return { spendToDate, daysElapsed, spendMet, timeMet, met: spendMet && timeMet };
}

Threshold and status are kept as separate ideas in the code and in the vocabulary. The threshold is the launch condition. The status is where an ad sits afterwards:

export type AdStatus = "testing" | "validated" | "scaling" | "fatiguing" | "dead";
export type AngleStatus = "testing" | "validated" | "dead";

Judged against the winner

An abstract target cost per sale is easy to argue with. So each ad is compared with the best ad that product has ever had: the lowest cost per purchase among ads with enough sales to rule out a lucky streak. An ad that costs twice the winner's is turned off. An ad within ten percent of it is validated. As winners improve, the bar only gets harder.

// lowest spend ÷ purchases per product, among ads with minSales or more
export function pickWinners(ads: Candidate[], minSales: number) {
  const best = new Map<string, ProductWinner>();
  for (const a of ads) {
    if (a.purchases < minSales || a.purchases <= 0) continue;
    const cpa = a.spend / a.purchases;
    const cur = best.get(a.productId);
    if (!cur || cpa < cur.cpa) best.set(a.productId, { ...a, cpa });
  }
  return best;
}

Angles first, executions second

An angle is a positioning hypothesis: one reason a person might want the product. The testing side of the system exists to find angles that work. Each angle is tested by a set of ads in several formats, because one bad execution should not be allowed to kill a good idea.

Once an angle is validated, the scaling side takes over and extracts everything it can from it, in three layers that get cheaper as they go: new formats, then new variations, then new performers. Volume comes from scaling. Testing is a capped, speculative spend.

The rule that saves the most money sits between the two: never build variations of an angle that has not been validated. If the positioning is wrong, a hundred executions of it are also wrong.

The rules it cannot break

How it is built

PartWhat it does
Core packageEvaluation, rollups, naming, generation and the daily run, as pure functions where possible so they can be tested without an ad account
PostgresAll state, with a client_id on every table from the first migration; queried through Drizzle
Meta clientWritten behind an interface with a fixture-backed stub, so everything except live data could be built while account access was unavailable
Next.js dashboardClient login, review queue, results per angle
Telegram botMorning report, approvals, and plain-English budget changes that always end in a confirm step
HostingOne container on a single server running the dashboard, the worker and the bot

What I took from it

The useful constraint was deciding in advance what the system is not allowed to do. Writing down "kill is the only autonomous action" made every later design question easier, because anything else had to go through a person, and the work became making that approval fast.