ViralBench + Codex
A proposed workflow for agent trace review, replay, and independent evaluation.
An audit of a marketing agent and a proposed improvement harness.
I inspected ViralBench’s research, image-generation, preview, and draft workflow, then designed a separate trace, replay, and evaluation process for making bounded engineering changes.
What the standalone agent actually does
The audited agent has five tools, an 18-round limit, a six-call research budget, labeled visual references, and ten loaded reflection notes. The pinned standalone configuration uses one account and AUTO_QUEUE=false, leaving a draft for review.
I traced how information moves from research into image generation, preview, publishing checks, saved snapshots, and later reflection. The existing agent is ViralBench’s work. My contribution is the code audit and the proposed engineering harness.
| Finding | Consequence |
|---|---|
| Preview shows at most six slides | A longer final payload can contain slides the agent has not reviewed. |
| Publish checks are not tied to a full-preview receipt | Nonempty slides, caption, and usable URLs pass checks without proving the final payload was previewed. |
| Snapshot omits the complete tool and artifact trace | The retained state cannot fully reconstruct the decisions behind the post. |
| Metrics use caption-prefix matching | Without stable public post identity, outcomes can be assigned to the wrong draft. |
| Same-round actions use Promise.all | An image action cannot reliably consume a label created by a simultaneous dependent action. |
A preview receipt tied to the final payload
One proposed experiment fixes the preview boundary: render every slide, issue a receipt tied to the exact final payload, and reject a changed or incompletely previewed payload. The experiment would retain the patch, regression case, manifest, and validation record.
A trace recorder would keep research responses, generated media, tool calls, preview results, and draft identity. Codex could use that evidence to propose a small patch. A separate evaluator and release controller would decide whether the change passes and can advance.
The improvement harness and preview receipt are an engineering design, not a deployed service or a measured experiment.
Reproducing failures before a live trial
Replay would freeze the research responses, media, account state, render responses, and publishing mocks for a task. That makes it possible to inspect whether a patch handles the same conditions without spending another live post on a known failure.
The proposed fixtures include duplicate generated images, missing final slides, tool timeouts, absent draft-review URLs, delayed metrics, and ambiguous caption matching. Reflection notes would retain supporting and contrary evidence and an expiration condition, rather than accumulating ten unqualified narrative lessons.
| Fixture | Check |
|---|---|
| Seven-slide final post | All seven slides must appear in the preview receipt. |
| Payload edited after preview | The prior receipt must not authorize the changed draft. |
| Tool timeout or missing draft URL | Retain the failure and prevent an unreviewable handoff. |
| Two posts with similar captions | Require stable post identity before attaching performance metrics. |
Separating replay correctness from external performance
Replay answers whether the system handles retained conditions correctly. A live trial answers a different question: whether a change improves external performance under current distribution conditions.
The proposed live design pairs baseline and treatment by account, publishing slot, and time, fixes measurement windows in advance, and retains raw outcomes. It treats outliers and distribution noise explicitly. No causal lift or completed trial is reported.
Codex’s editable scope would exclude raw traces, primary metrics, credentials, release rules, and publishing policy. The recorder, evaluator, and release controller keep their own responsibilities, so the engineering agent cannot rewrite the evidence used to judge its patch.
The audit and the inspected revision
The full study contains eight audit findings, a proposed trace model, preview-gate experiment, replay design, and live-evaluation plan. Its handoff identifiers and thresholds are illustrative design examples.