To test an AdCP agent before it goes live, put it through three gates in order: the AdCP storyboards, which check that it speaks the protocol correctly; behaviour tests against counterparty agents, which check that it makes sound decisions under pressure; and a benchmark, which measures it against other agents on the same brief. Passing the first gate makes an agent conformant. Only the second and third show whether it should be trusted with a budget.

TL;DR. The Ad Context Protocol (AdCP), published and governed by AgenticAdvertising.org, defines conformance through storyboards: scripted workflows that check every response an agent returns. Storyboards prove wire behaviour, not judgement, so a conformant agent can still close weak deals. This checklist sets out 16 checks across the three gates, followed by the earned autonomy stages that should come after launch.

What does it mean to test an AdCP agent?

Testing an AdCP agent means answering three separate questions: does it follow the protocol, does it behave sensibly when a counterparty pushes back, and how does it compare with other agents doing the same job. The AdCP documentation covers the first question in depth. The other two depend on the agent's mandate and the deals it will face, so they sit with the team deploying it.

AdCP is precise about vocabulary. Its Conformance Specification calls an agent conformant when it meets the normative rules in the storyboards, and verified when AgenticAdvertising.org has tested it recently and issued a signed attestation. It warns against a third word, describing "compliant" as "self-attested, unverified, no external check".

One worked example of how the later gates can be split is Alkimi, which runs them as two products. Concourse benchmarks any buying agent without asking which standard it uses, and the Gym trains agents and lets them spar against counterparty agents regardless of protocol. Neither replaces the storyboards; both start where the storyboards stop.

How do you check that an AdCP agent conforms to the protocol?

Gate one is the storyboard suite. The AdCP guide to validating an agent with storyboards says each storyboard "defines the exact tool call sequence a buyer agent makes and validates every response shape", so most graded assertions fall on the seller's responses. A buy-side team can still read the storyboards as a script of what its agent is expected to send.

Declare v3 support first. The v3 readiness checklist warns that storyboard testing "will hard-fail any agent that does not declare v3 support", and its first item is implementing get_adcp_capabilities. Version 2 was fully deprecated on 1 August 2026.
Claim only what the agent really does. The runner selects universal, protocol and specialism storyboards from the protocols and specialisms the agent declares. A missing_tool skip on a declared protocol means, in the documentation's words, "the seller overclaimed or forgot to expose a required tool".
Run in sandbox mode. Storyboard runs use sandbox mode by default, and an agent handling a sandbox account must not persist production state or cause real-world side effects.
Read a partial result before reacting to it. The documentation treats partial as "a coverage result, not automatically a failure". Production endpoints must not expose the compliance test controller, so full lifecycle coverage needs a dev or staging endpoint that does.
Check the adapter behind the wire. AdCP's guidance on mock upstream fixtures notes that storyboards "cannot tell whether the adapter behind the wire actually integrates with the upstream or returns shape-valid responses with synthetic data". Agents that wrap a DSP, SSP or other platform should run that harness before staging.
Decide whether to seek AAO Verified. The AAO Verified mark applies to one agent URL, role and AdCP version, needs an active AgenticAdvertising.org membership at an API-access tier, and is "continuously re-evaluated, not a one-time certificate". A qualifying regression starts a 48-hour grace period, after which the badge can be revoked.

Why is a storyboard pass not enough to go live?

A storyboard pass shows that the agent speaks AdCP. It does not show whether the deals it closes are any good, and AgenticAdvertising.org says so directly: AAO Verified is "not a substitute for due diligence", and buyers "should still assess contracts, billing, governance, security, and incident response".

The gap shows up in benchmark data. In Concourse Bench v1, 55 of 96 runs completed a buy, and 95 of the 110 signed contracts promised a viewability floor below 70%. Those agents finished the job and still left the buyer weakly protected, a result no wire-format check is designed to catch.

The risk is real for teams that move straight to live money. An ad tech executive whose company ran an early buying agent on a handful of real client buys told Digiday that the team shut it down after two days: "What if it accidentally added a zero? What if it makes a mistake overnight or over a weekend?"

How should an AdCP agent be tested against counterparty agents?

Gate two is behaviour testing: putting the agent in front of other agents that negotiate, refuse and make mistakes, before any real money moves.

Spar with sellers that push back. Run briefs where the counterparty counters, stalls or walks away, and check that the agent neither caves nor loops.
Test every hard limit in the mandate. Offer terms just outside the budget cap, below the viewability floor or outside the brand safety list, and confirm the agent refuses each one.
Feed it proposals that contradict the brief. A proposal that changes a date, swaps a format or adds a condition in passing should be caught, not accepted.
Check that it escalates. When the agent cannot meet the brief inside its limits, it should stop and hand the decision to a named person rather than close on worse terms.
Test against several counterparties, not one. Buyers will meet seller agents built by different teams, some on other standards, so a single friendly test partner proves little.
Keep a replayable log. Every sparring session should leave a record of offers, counters and reasons that can be rerun after the agent changes.

The Gym is built for this gate. Buyers and sellers build their own agents, set the rules those agents work to and spar against counterparty agents on realistic briefs, whichever protocol the agent was built on.

Early sell-side deployments show the same instinct for control. The Weather Company's first sales agent, built on the open framework developed through AdCP, covered a small set of formats and audiences, and a human validated availability and pricing on each buy-side request, AdExchanger reported. "We didn't try to boil the ocean on day one," said Dave Olesnevich, VP of weather data and advertising products at The Weather Company.

How do you benchmark an AdCP agent before launch?

Gate three is a benchmark: the same brief, run many times and scored the same way, so one agent can be compared with another or with its own previous version.

Fix the brief. Budget, channel, audience and quality floors stay identical across every run.
Run it repeatedly. One completed run proves little, while the completion rate across many runs shows how reliable the agent is.
Score completion and contract quality separately. A completed buy on weak terms is a different failure from no buy at all, and both need counting.
Re-run after every change. A new model, prompt or mandate can shift results, so the benchmark belongs in the release process rather than only the launch.

Concourse benchmarks any buying agent on this basis without asking which standard it uses, so an AdCP agent and an agent built to another specification can be measured on the same brief. Work starts with a scoping conversation about the workflow being automated, the brief, the constraints and what a good outcome looks like.

What happens after an AdCP agent passes all three gates?

Passing the gates earns an agent the right to start small, not the right to run unsupervised. A staged model known as earned autonomy moves an agent through five steps: Observe, Recommend, Draft, Human-approved action and Bounded automatic action.

In practice, agents take on the parts of media buying people do not enjoy, such as chasing availability, reformatting proposals and checking terms line by line, while people keep the judgement and approve the deals that matter. The 25 go-live checks for agencies set out what to confirm before each step up.

Which tools cover the sparring and benchmark gates?

Alkimi is neutral on standards: it can work with agents built on AdCP or on IAB Tech Lab's Agentic Advertising Management Protocols (AAMP), with or without a DSP. Concourse covers the benchmark gate and the Gym covers sparring against counterparty agents, while the storyboards remain the AdCP community's own conformance test. Teams preparing an agent for launch can start with a scoping conversation through Concourse.

Frequently asked questions

How do I test an AdCP agent?

Run the AdCP storyboards to check protocol conformance, then test the agent's behaviour against counterparty agents, then benchmark it on a fixed brief. The storyboards show that the agent speaks AdCP, and the other two gates show whether it makes good deals.

What is an AdCP storyboard?

A storyboard is a YAML file that defines a workflow end to end, the tool calls involved and every assertion the responses must satisfy. AdCP's guide to storyboards and scenarios describes storyboards as the conformance specification itself.

Is "AdCP compliant" the same as AAO Verified?

No: AdCP's documentation describes "compliant" as a self-attested claim with no external check. AAO Verified is AgenticAdvertising.org's trust mark, continuously re-evaluated for one agent URL, role and AdCP version.

Can storyboards run without spending real money?

Yes, storyboard runs use sandbox mode by default, and an agent must not cause real-world side effects for a sandbox account. Full lifecycle coverage needs a dev or staging endpoint that exposes the compliance test controller.

Which AdCP version should an agent be tested against?

Version 3.2 is the current stable documentation, and 3.3 is in beta. Storyboard testing hard-fails any agent that does not declare v3 support.

Does an agent need to use AdCP to be benchmarked?

No: a benchmark compares agents on the same brief and the same scoring, whatever protocol they speak. Concourse, for example, benchmarks any buying agent without asking which standard it uses.