Beyond the Leaderboard

A code audit and proposed workflow for testing marketing-agent improvements through traces, replay, and independent evaluation.

Sulayman Bowles · Jul 9, 2026 · Updated Oct 5, 2026

Read this alongside Make AI results inspectable.

ViralBench asks AI agents to earn TikTok views. This code audit proposes an improvement harness: Codex would investigate failures and test bounded patches against retained evidence. The design has not established a deployed harness or measured performance gains; evaluation and release approval would remain independent.

The reviewed ViralBench loop gives frontier models TikTok accounts and tools for social-content research, reference-post inspection, image generation, slideshow review, and publishing. They pursue views in the fitness category, then respond to delayed account feedback. Tool budgets and changing distribution conditions shape every decision.

I propose adding complete traces, replay tests, and controlled experiments. Codex would investigate a defined failure, form a falsifiable hypothesis, and submit a tested patch for independent review before any live trial.

The ViralBench agent makes the post. Codex proposes improvements to the agent.

ViralBench is a live agent environment

The public experiment compares GPT-5.5, Claude Opus 4.8, and Kimi. Each receives two warmed-up TikTok accounts and runs around 7 AM and 5 PM Central Time, pursuing views in the fitness category.¹

I reviewed the standalone handoff: one 937-line TypeScript file containing the prompt, agent loop, API clients, media handling, account telemetry, memory, rendering, publishing, and run persistence.² Unlike the two-account live competition, it uses one account and leaves a reviewable draft by default rather than scheduling publication.

The agent has up to 18 rounds and five tools:

TikTok and Instagram URLs resolve to images, which are resized, labeled, and returned to the vision-capable model. Generated images also receive labels and can guide later generations, giving the model visual working memory within a run.

Each run allows six Lightreel research calls, at most one per round, up to 12 viewed reference images per round, and 18 working rounds. OpenRouter calls the main model with an append-only conversation history. Same-round actions execute concurrently.

The workflow:

Retrieved examples may be unrepresentative, and TikTok’s distribution mechanism is unobserved. With few posts, the agent must infer whether a format failed or simply received weak distribution.

The environment combines partial observability, limited actions, changing conditions, and delayed rewards.

What the current code gets right

External research grounds the agent in recent content, while a finite budget forces it to choose questions.

The model sees source posts, generated images, and composed slides. Its visual review can therefore assess the artifact directly.

The single-file handoff makes tools, limits, failures, and memory easy to inspect.

Views are imperfect evidence, but the response comes from the platform.

The public handoff stops at a draft, preserving human review even though the live benchmark can publish autonomously. I would retain that boundary for company or client accounts.

The handoff implements content production. The proposed improvement infrastructure remains to be built.

The leaderboard is a system result, not a pure model score

Read the leaderboard as the result of a configured marketing system.

A post’s result depends on:

result = f(
  language model,
  system prompt,
  tool definitions,
  retrieval quality,
  reference-content selection,
  image model,
  account history,
  memory,
  posting time,
  publishing infrastructure,
  current TikTok conditions,
  random distribution effects
)

Model choice, research, image generation, copying instructions, account history, sound, and timing all affect the result.

The leaderboard is a live system benchmark, or field trial of agent stacks. It cannot isolate model intelligence.

This whole-system result is commercially useful: deployed agents depend on tools, data, permissions, and controls.

A weekly winner may have found a suitable format, imitated a successful post, used tools well, or benefited from distribution. The ranking alone cannot distinguish those explanations.

I would analyze each model-plus-harness configuration, not the model name alone.

Tie each result to a versioned configuration, state each change’s hypothesis, and distinguish estimated lift from lucky outliers before promotion.

What the code audit reveals

The file’s header describes a self-contained handoff, not an experimentation platform. Its gaps define the engineering work ahead.

1. The run record is too thin to explain success or failure

The local .marketing-agent-runs.json records the goal, published posts, final answer, reflection note, and round count.

It does not preserve the complete message history, individual tool inputs and outputs, Lightreel source material, viewed-media lineage, image prompts, preview assets, model usage, latency, cost, configuration hash, Git commit, or account-metric snapshots.

The stored summary cannot reliably distinguish retrieval, generation, publishing, and model errors—or explain a successful post.

Full traces should be the improvement harness’s primary dataset.

2. Several important constraints exist only as prompt prose

The prompt requires previewing before publication, a distinct image per slide, exactly one post per account, and completion before the final answer.

The runtime enforces maximum posts per account but accepts a final answer without checking the minimum. It requires no successful preview token, rejects no duplicate image URLs, and does not validate slide count, copy length, text fit, unsupported claims, or generation lineage.

Prompts influence behavior. Code must enforce these invariants.

3. Visual review is incomplete for longer carousels

Preview renders at most six slides. A longer carousel can reach publication with its final slides unseen.

Render every slide or block publication.

4. Cross-run memory is compressed into free-form notes

Future runs load up to ten model-written reflection notes as their only durable strategic memory.

These notes mix facts, interpretations, and narrative without checking them against the trace. They provide no stable representation of tested hypotheses, confidence, failed formats, account-specific effects, or unresolved questions.

The model writes its memory without an evidence check.

5. Account telemetry has attribution edge cases

The code refreshes recent posts’ TikTok metrics through ScrapeCreators when a public URL exists. Without one, it matches a profile post using a normalized caption prefix.

Similar captions can produce false matches. An experiment ledger needs stable post IDs and metric provenance.

6. Same-round parallelism creates hidden execution semantics

Promise.all runs same-round actions concurrently. This suits independent actions, but one image generation cannot reliably use another’s new label in that round. It must wait until a later round.

The prompt leaves this dependency implicit. A typed planner or action graph would express it.

7. The content-acquisition strategy creates provenance risk

The prompt asks the agent to adapt successful posts’ language closely and sometimes mirror their slide sequence. That may help short-term results, but it conflates:

• learning a general format;
• adapting a common hook pattern;
• paraphrasing a specific creator’s copy;
• reproducing a distinctive sequence or visual treatment.

The code neither retains a source graph nor measures reference-to-artifact similarity.

The agent should learn why a post works without reproducing a creator’s distinctive work.

8. The package is not fully reproducible as shipped

The source imports sharp for resizing and instructs users to install it, but package.json omits the dependency.

Unattended tests need a clean installation. The manifest must declare what the code imports.

Closing these gaps would make runs reproducible and changes testable.

What I mean by “Codex as a harness”

OpenAI describes the Codex harness as the loop and execution logic connecting a model to instructions, tools, filesystem operations, and feedback.³ Its harness-engineering work also includes the repository, documentation, tests, review process, execution environment, and validation rules that govern reliable work.⁴

In this design, ViralBench is the executable marketing environment. Codex works on it from outside the content-production loop.

I would assign these roles:

Keep the evaluator, success criteria, and release authority outside the patching agent’s control. Otherwise it could improve its score by weakening the test.

I would separate five functions:

1. The inner agent researches, creates, previews, and submits a post.
2. The recorder captures raw evidence, actions, artifacts, and outcomes.
3. The evaluator applies versioned checks and statistical rules.
4. Codex diagnoses a bounded issue and produces a patch in an isolated worktree.
5. The release controller decides whether the candidate can enter replay, draft canary, or live deployment.

The proposed control flow:

                  LIVE ENVIRONMENT
      Lightreel / TikTok / Doublespeed / media APIs
                         ↑       ↓
                ┌───────────────────┐
                │ ViralBench agent  │
                │   inner loop      │
                └───────────────────┘
                         ↓
          immutable traces, sources, artifacts,
            configuration, cost, metric snapshots
                         ↓
                ┌───────────────────┐
                │ Evaluation layer  │
                │ checks + analysis │
                └───────────────────┘
                         ↓
              bounded experiment candidate
                         ↓
                ┌───────────────────┐
                │ Codex worktree    │
                │ plan / patch/test │
                └───────────────────┘
                         ↓
             offline replay and regression gate
                         ↓
               human-approved draft canary
                         ↓
                 locked online evaluator
                         ↓
                   promote or revert

The proposed loop makes engineering changes reviewable before they reach live accounts.

The evidence layer comes before the Codex layer

Atlas uses a distinction that also matters here: preserve observations separately from judgments.

Responses, rendered pages, schema, and search results are observations. Scores and recommendations interpret them. A revised judgment must not erase the evidence it used.

For ViralBench, each run should preserve an immutable evidence bundle:

• the Git commit and experiment manifest;
• model provider, model version, reasoning settings, and sampling parameters;
• the exact system prompt and tool schemas;
• account state at the beginning of the run;
• every Lightreel query and response;
• every source URL and retrieved media item;
• hashes of source and generated images;
• image prompts and reference-image lineage;
• every preview render;
• the final slide data and caption;
• publishing response and stable TikTok post ID;
• token usage, API cost, latency, and tool errors;
• metrics captured at fixed post-publication windows;
• human review decisions and reasons.

Store derived judgments separately:

• “insufficient research breadth”;
• “slide 1 text is likely unreadable”;
• “final copy is too similar to source post X”;
• “the treatment improved account-normalized 72-hour views”;
• “the agent repeated a failed format despite contrary evidence.”

Each finding should cite its supporting evidence IDs. Codex may propose findings and patches; it must not rewrite the raw trace.

Complete records support reproduction and evaluation, and let reviewers compare model explanations with the actual sequence of events.

The experiment is the unit of change

An experiment needs a falsifiable hypothesis and protected variables before Codex edits.

An example:

experiment_id: visual-critic-v1
hypothesis: >
  A separate visual-critique pass after rendering will reduce
  text-overflow, repeated-image, and synthetic-looking-image failures
  without lowering 72-hour account-normalized views.

baseline_commit: 5f5f57e251023ceb37961c0fc2c808f67ceb71eb
editable_surfaces:
  - agent/critics/visual.ts
  - evals/visual/*
  - experiments/visual-critic-v1.yaml

protected_surfaces:
  - evals/online/primary-metric.ts
  - policies/publish-gateway.ts
  - traces/raw/*
  - secrets/*

offline_gates:
  - all_slides_rendered
  - no_duplicate_images
  - no_text_overflow
  - source_lineage_complete
  - historical_replay_pass_rate >= 0.95

live_design:
  unit: post
  blocking:
    - account
    - posting_slot
    - calendar_week
  primary_window: 72h
  minimum_pairs: 20

primary_metric: account_adjusted_log_views_72h
guardrails:
  - publish_failure_rate <= baseline
  - median_cost_per_post <= baseline * 1.15
  - policy_violation_count == 0
  - source_similarity_high_risk_count == 0

promotion_rule: >
  Pass every hard guardrail and show a positive primary-metric estimate
  with the pre-registered uncertainty threshold.

Define the hypothesis, editable scope, validation commands, metrics, guardrails, and stopping rule before Codex edits code or anyone sees live results.

Retain rejected experiments too: their traces explain which hypothesis failed and why.

Offline replay should absorb most iteration

Replay should handle parser, tool, and prompt regressions before scarce accounts, posting slots, API budgets, and review time are committed to live trials.

Convert historical calls into versioned fixtures:

• frozen Lightreel responses;
• saved TikTok and Instagram media metadata;
• locally stored reference images or content-addressed snapshots;
• Doublespeed style presets and render responses;
• account-statistics snapshots;
• image-generation stubs or a fixed asset bank;
• publishing mocks;
• known failure responses from each dependency.

Hold recorded inputs fixed while comparing prompts, tools, memory, and models.

Replay cannot predict views. It can test whether a post is ready for a live trial.

Deterministic checks

Check these failures without another model:

• Did the agent remain within its research and round budgets?
• Did it inspect source material before generating?
• Was every generated asset linked to a valid reference or explicit text-only path?
• Was every slide rendered?
• Were duplicate image URLs rejected?
• Did publication occur only after a passing preview?
• Did the final account count match the experiment contract?
• Were claims, source IDs, and artifact hashes complete?
• Did the agent recover correctly from a tool timeout?
• Did it leave the repository and fixture state clean?

Rubric-based checks

Other qualities need judgment:

• Is the first slide visually legible at mobile size?
• Does the carousel maintain a coherent subject across slides?
• Does the copy preserve the mechanism of a reference without reproducing it?
• Does the post contain unsupported fitness or health claims?
• Does the final creative look internally consistent?

Give model-based graders the evidence and a narrow rubric. Their scores remain judgments, not ground truth.

Adversarial and fault-injection cases

Inject dependency failures:

• Lightreel returns a plausible but source-poor answer.
• ScrapeCreators resolves only half of a carousel.
• A reference image is too large or corrupt.
• Image generation returns the same asset twice.
• The renderer clips the last line of text.
• Sound extraction fails.
• Doublespeed creates a draft but returns no public review URL.
• TikTok metrics arrive late.
• Two profile posts have nearly identical caption prefixes.
• The model attempts to finish before publishing the required artifact.

The agent should stop safely and retain a diagnostic trace.

OpenAI’s evaluation guidance combines prompts, captured runs, traces, artifacts, checks, and comparable scores.⁵ I would use that structure here.

Live evaluation needs experimental discipline

A replay pass qualifies a candidate for controlled live testing. It does not establish that the change will increase views.

Views are noisy and path-dependent

Account history, content supply, audience, timing, trend decay, sound, source similarity, and platform distribution all affect TikTok outcomes. The same creative cannot reach the same audience twice under identical conditions.

Posts can change the account’s audience and later distribution, so sequential outcomes are not independent observations.

Raw mean views can mislead with small samples and rare winners. Online business metrics often have skewed, heavy-tailed distributions; controlled experiments use robust estimators, covariate adjustment, and longitudinal methods to address them.⁶

The trial should be blocked and paired

With few accounts and slots, I would block trials rather than compare treatment posts with historical controls.

Pair each candidate with a concurrent baseline, controlling where possible for:

• account;
• morning or evening slot;
• day or calendar week;
• broad content lane;
• recent account performance;
• number of slides;
• use of sound;
• paid or organic distribution, if that ever changes.

A switchback or crossover could alternate baseline and treatment across comparable account-slot blocks. Pre-experiment performance could serve as a covariate, using variance-reduction methods such as CUPED to account for different starting levels.⁷

Measurement windows should be fixed in advance

Capture views and engagement at fixed horizons, such as:

• one hour;
• six hours;
• 24 hours;
• 72 hours;
• seven days.

Choose the primary horizon before the trial. A favorable six-hour result cannot replace a planned 72-hour result.

I would analyze an account-adjusted log(1 + views) alongside the raw distribution. The log scale limits an extreme post’s influence; median and trimmed-mean comparisons would check robustness.

Show every raw outcome alongside robust estimates so readers can see the outliers’ influence.

Hard guardrails should precede optimization

Evaluate policy, provenance, factuality, and disclosure separately from views. A weighted score must not let high reach offset a failed constraint.

The proposed objective:

maximize expected account-adjusted views at the locked horizon

subject to:
- platform and disclosure requirements pass
- source-provenance checks pass
- factual and health-claim checks pass
- publishing and rollback checks pass
- cost remains within budget
- human approval is present where required

Only candidates that pass these constraints compete on views.

A/B tests and bandits answer different questions

A multi-armed bandit could allocate posts to promising strategies. It cannot replace causal evaluation.

An A/B or switchback trial asks whether a change improved outcomes under comparable conditions.

A bandit chooses where to allocate the next unit of traffic or publishing capacity, balancing reward and learning.

Use controlled trials to establish effects, then bounded bandits to allocate among validated strategies. Mixing measurement and allocation too early obscures causality.

Codex should receive bounded engineering tasks

Codex can inspect, edit, and run code through its CLI, non-interactive codex exec, TypeScript SDK, or an orchestration layer.⁸ The SDK can start and resume coding threads programmatically. Worktrees isolate experiments; sandbox policies restrict writes.⁹

I would begin with codex exec in CI and a structured output schema. The SDK could later support persistent experiment threads and staged tasks.

Each task should define:

• a single concrete goal;
• the working directory and base commit;
• task mode;
• files that may be edited;
• protected paths;
• raw evidence inputs;
• known failure reproduction;
• required tests and replay suites;
• artifact locations;
• output schema;
• done criteria;
• conditions that require stopping rather than guessing.

I plan to support these task modes:

“Deploy” would execute an approved release, with no authority to approve it.

An illustrative handoff:

Mode: Implement
Goal: Make a successful full-slideshow preview a hard prerequisite for publishing.

Evidence:
- Trace VB-2026-07-08-017 published eight slides after previewing only six.
- The current preview function caps max_slides at six.
- publish_slideshow has no preview receipt check.

Editable paths:
- agent/rendering/**
- agent/publishing/**
- evals/preview-gate/**

Do not edit:
- evals/online/**
- policies/**
- traces/raw/**
- experiment promotion thresholds

Verification:
- npm test
- npm run eval:preview-gate
- npm run replay -- --case VB-2026-07-08-017

Done when:
- every slide is rendered or publication is blocked;
- a preview receipt is bound to the exact slide payload;
- the historical failure becomes a passing regression case;
- RESULTS.md contains commands, results, changed files, and remaining risks.

This handoff defines observable behavior, protected files, and acceptance checks.

The repository itself must become legible to Codex

Codex needs relevant knowledge in its repository. OpenAI’s harness-engineering work emphasizes that information confined to chats, external documents, or memory is unavailable to the agent while it works.⁴

I would split the handoff into modules and versioned artifacts:

/agent
  /policy              system prompts, action schemas, run policy
  /runtime             loop, state machine, dependency handling
  /memory              structured cross-run state

/tools
  /lightreel
  /media
  /images
  /rendering
  /publishing
  /metrics

/experiments
  /manifests            hypotheses, protected variables, promotion rules
  /candidates           Codex-ready tasks and results

/replay
  /fixtures             frozen external responses and assets
  /worlds               named historical environment states
  /faults               dependency and data failure scenarios

/evals
  /deterministic
  /rubrics
  /visual
  /online
  /regression

/traces
  /raw                  append-only event records
  /indexes              searchable run and artifact indexes

/artifacts
  /source-media
  /generated-media
  /previews
  /published

/reports
  /runs
  /experiments
  /leaderboard

/codex
  AGENTS.md
  /skills
  /schemas
  /hooks
  /task-templates

/policies
  publish-gateway.ts
  provenance.ts
  platform-compliance.ts
  health-claims.ts
  cost-limits.ts

AGENTS.md should point to architecture, evidence contracts, task modes, checks, and protected paths. Detailed decisions belong beside the code they govern, in versioned documents and schemas.

Hooks could require an experiment ID, protect restricted files, validate result-to-trace references, and confirm required evaluations before accepting a pull request. Code would enforce the task contract.

Memory should become structured belief, not autobiography

The existing reflection note could remain, but structured memory should separate:

• observation;
• hypothesis;
• decision;
• result;
• confidence;
• scope;
• expiration.

An example:

{
  "belief_id": "BEL-0041",
  "account": "fitness-account-a",
  "claim": "Beginner mistake carousels outperform motivation quotes in the evening slot",
  "evidence_runs": ["RUN-0108", "RUN-0112", "RUN-0117"],
  "counterevidence_runs": ["RUN-0120"],
  "confidence": 0.63,
  "valid_from": "2026-07-01",
  "review_after": "2026-07-15",
  "status": "provisional"
}

The system could derive provisional beliefs from traces while retaining narrative notes. Every evidence reference must resolve.

Expire or discount beliefs when their evidence ages, the account changes, or results contradict them.

Keep a small, contestable set of beliefs. Repetition must not become evidence.

Provenance is a product requirement, not an afterthought

The audited prompt encourages close adaptation, making source lineage an early requirement.

Record each final slide’s lineage:

“Problem, explanation, correction, payoff” is a common pattern. A creator’s distinctive sentences, examples, and compositions may be reproduction. The evaluator should flag that distinction for review.

Under the TikTok guidance cited in July, realistic AI-generated images, audio, and video require labels; Content Credentials may also trigger labeling.¹⁰ Because ViralBench seeks realistic imagery, the proposed gateway should retain disclosure metadata and block publication when a required setting cannot be verified.

A views-only objective may reward extreme nutrition claims, impossible transformations, or fear-based advice. Claims about health, injury, supplementation, or disease should pass a separate evidence and policy check before publication.

“Maximize views” is a useful benchmark objective and an incomplete business objective

The current goal makes views the sole objective, excluding brand safety, selling, and follower count.

Views provide an observable target.

It may also favor copied hooks, outrage, exaggerated promises, high-arousal content, or account tactics that sacrifice future performance.

I would separate two modes.

Benchmark mode

Study attention capture under controlled conditions.

Optimize account-adjusted views subject to platform policy, disclosure, provenance, factuality, cost, and publishing constraints.

Enterprise mode

Measure business outcomes:

• qualified reach;
• attributable site visits;
• assisted conversions;
• customer acquisition cost;
• lead quality;
• retention or repeat engagement;
• brand and compliance constraints.

Business metrics have different horizons and causal roles. Separate leading indicators from outcomes and disclose attribution uncertainty.

Reach alone does not establish commercial value.

ViralBench can test attention capture. Connecting it to revenue requires downstream measurement and appropriate constraints.

The first build sequence

Make the baseline reproducible and observable before changing prompts.

Phase 0: Freeze and document the baseline

• Pin the current commit and dependency versions.
• Add the missing sharp dependency and a lockfile.
• Record the exact live and handoff configurations separately.
• Create an architecture map and a canonical run schema.
• Save a small set of representative historical runs as reference cases.

Deliverable: an installable, replayable baseline.

Phase 1: Add complete traces and hard runtime invariants

• Replace the thin snapshot with append-only event traces.
• Hash prompts, tool schemas, source assets, generated assets, and final payloads.
• Require full preview before publication.
• Enforce minimum and maximum post counts.
• Reject duplicate images and malformed slide payloads.
• Bind a preview receipt to the exact content being published.
• Persist stable post IDs and fixed-window metric snapshots.

Deliverable: traces that explain each run.

Phase 2: Split the monolith and build replay

• Extract provider and tool adapters behind typed interfaces.
• Add mock implementations and frozen fixtures.
• Build deterministic checks and regression cases.
• Add fault injection for each external dependency.
• Preserve the original single-file behavior as a compatibility baseline.

Deliverable: local tests that require no publication.

Phase 3: Introduce Codex as a reviewed engineering agent

• Add AGENTS.md, task templates, output schemas, and protected-path rules.
• Run Codex in isolated worktrees with workspace-scoped permissions.
• Start with Plan, Review, Debug, and Implement modes.
• Require a human to approve every patch and experiment manifest.
• Store Codex transcripts, diffs, test results, and review decisions.

Deliverable: reviewed engineering patches.

Phase 4: Add controlled draft canaries

• Allow a passing candidate to create a draft on a designated test account.
• Run visual, provenance, claim, and disclosure checks on the final artifact.
• Require human approval to publish.
• Collect fixed-window outcomes through the locked evaluator.
• Compare treatment against a concurrent blocked baseline.

Deliverable: a controlled online trial with a rollback point.

Phase 5: Automate promotion within narrow boundaries

Automate low-risk promotion only after offline and online gates demonstrate reliability.

Early candidates could include:

• parser fixes;
• retry and timeout handling;
• trace completeness;
• renderer validation;
• budget enforcement;
• memory retrieval changes that pass replay;
• switching among already-approved prompt variants.

Keep these changes under review:

• publishing permissions;
• content-policy logic;
• provenance thresholds;
• health-claim policy;
• primary metrics;
• promotion rules;
• production credentials;
• changes that expand Codex’s editable scope.

Expand autonomy only where reliability has been demonstrated.

The experiments I would run first

With traces and replay in place, I would test:

Does a separate visual critic improve the final artifact?

A separate critic may catch repeated images, inconsistent people, overflow, or artificial compositions. It may also increase cost and over-edit good work.

Test visible defects offline before publication.

Does structured memory outperform free-form notes?

Replace or supplement the ten notes with evidence-linked beliefs, account-specific hypotheses, and expiry dates.

Replay would test relevant memory use and rejection of stale or contradicted beliefs. Live trials would test account-adjusted performance consistency.

Is candidate generation better than single-path creation?

Instead of committing to one strategy, generate several cheap concepts, score them against current evidence, and spend image-generation budget on the strongest.

A critic may favor consensus and reject unusual, promising ideas. Measure diversity, cost, and live outcomes.

How much research is enough?

Six Lightreel calls is a budget, not a proven optimum. Compare four, six, and eight calls on source breadth, decisions, cost, and views.

Query diversity, full-post inspection, or a separate review of copy against evidence may matter more than call count.

Should planning and execution use different models?

One model could choose the thesis and sources; another could create and inspect the post. Specialization may help, but handoffs may lose information.

Compare system outcomes and trace quality across these configurations.

Does source similarity predict either performance or risk?

Source lineage would let us test the prompt’s assumption that close adaptation improves views. Compare originality bands on views, reviewer concern, and platform outcomes.

This could locate where learning a format stops helping and duplication risk begins.

What this becomes beyond TikTok

A TikTok carousel is the first environment. The proposed engineering loop could serve other marketing systems:

• SEO and AEO recommendations generated from public-web evidence;
• paid-media agents that propose and test campaign changes;
• sales-content agents that build materials from product and customer evidence;
• brand-monitoring agents that investigate emerging narratives;
• content operations that generate, review, publish, and learn across channels.

Each would need to:

1. observe an environment;
2. preserve the evidence;
3. make a bounded decision;
4. act through controlled tools;
5. capture the trace and artifact;
6. measure the external result;
7. turn recurring failures into evals;
8. give Codex a narrow engineering task;
9. validate before promotion.

The shared principle is to keep evidence inspectable, tie derived claims to it, and validate proposed actions independently.

The durable asset is the learning system

A post’s advantage may fade with its trend. A reproducible explanation of improvement can inform later experiments.

Retain traces, replay cases, regressions, source graphs, task contracts, results, and release records.

The live marketing loop supplies an environment for testing. The proposed outer harness would need to distinguish causal gains from distribution noise and close imitation.

Its success would be an independently measured improvement under fixed constraints. That remains a proposed test, not an established result.

Frequently asked questions

What is ViralBench?

ViralBench gives frontier models TikTok accounts and tools to research posts, inspect media, generate images, preview carousels, and publish. They pursue views in the fitness category.

What is an AI agent harness?

A harness surrounds a model with instructions, tools, schemas, memory, permissions, an execution loop, tests, traces, validation, and deployment controls. Different context and feedback can change the same model’s performance.

How would Codex use ViralBench as a harness?

Codex would inspect failures and evaluation reports, edit a defined repository surface, run replay and regressions, and submit a patch for review. ViralBench would supply the environment and live outcomes.

Is ViralBench measuring the model or the whole system?

The result reflects the model, prompt, research, image generator, tools, account, timing, memory, and platform. Compare configured systems; the experiment cannot isolate intelligence.

Can an AI agent safely improve another AI agent?

Bounded work and independent evaluation could make this viable. Separate the builder from raw-evidence control, success metrics, evaluation, publishing credentials, and release approval. Use sandboxes, protected paths, locked evaluations, review, and rollback.

Why not optimize only for views?

An unconstrained views objective may reward copying, sensational claims, and short-term tactics. Platform, provenance, factuality, cost, and publishing checks should pass before outcome optimization.

Why is offline replay necessary if TikTok is the real evaluator?

Replay catches research gaps, broken tools, incomplete previews, duplicate assets, stale memory, and policy violations cheaply and repeatedly. It cannot predict TikTok distribution. Passing replay qualifies a candidate for live testing.

Sources and technical notes

This article draws on the live methodology and a supplied audit of repository commit 5f5f57e251023ceb37961c0fc2c808f67ceb71eb.

1. ViralBench methodology and live experiment. ↩
2. ViralBench repository and the audited marketing-agent.ts commit. ↩
3. OpenAI, “Unrolling the Codex agent loop”, January 23, 2026. ↩
4. OpenAI, “Harness engineering: leveraging Codex in an agent-first world”, February 11, 2026. ↩
5. OpenAI Developers, “Testing Agent Skills Systematically with Evals” and “Build an Agent Improvement Loop with Traces, Evals, and Codex”. ↩
6. Chen Wei et al., “Beyond Basic A/B Testing: Improving Statistical Efficiency for Business Growth”, revised December 27, 2025. ↩
7. Microsoft Research, “Deep Dive Into Variance Reduction”. ↩
8. OpenAI Developers, Codex SDK and Codex non-interactive mode. ↩
9. OpenAI Developers, Codex sandboxing and Codex worktrees. ↩
10. TikTok Support, “About AI-generated content”. ↩

working rounds

18

fixed per agent run

execution tools

5

research through publishing

research calls

6

maximum Lightreel budget

outer loop

01

Codex improves the harness