One of the limitations of reporting AI buying performance as a single completion rate is that it collapses what is actually a multi-stage process into a binary outcome. A model either completed the buy or it did not. That framing is useful for headline comparison but not for improvement. If you want to make an AI buyer better, you need to know at which stage it failed, not simply that it failed. The Concourse Bench v1 methodology is structured around four stages precisely because each stage has distinct failure modes that require different remediation.

Stage one: what happens during brief intake?

Brief intake is the first stage. The model receives the buying brief and must produce a structured representation of it: the budget (total and any sub-allocations), the audience requirements, the channel requirements, the quality parameters, and the timing constraints. This representation becomes the constraint set that governs everything that follows.

Failures at stage one are rare but consequential. If the model misreads a budget figure, misclassifies an audience requirement, or fails to extract a key quality constraint from the brief, every subsequent stage will operate against an incorrect constraint set. A portfolio that satisfies the model's representation of the brief may not satisfy the actual brief. The problem is upstream and invisible until the deal record is reviewed.

Stage one failures are also among the hardest to diagnose from the output alone. A completed buy that misses a brief requirement could reflect a stage one failure (the constraint was never extracted), a stage three failure (the constraint was extracted but lost during portfolio assembly), or a stage two failure (the model pursued the wrong sellers because it misunderstood the audience requirement). Distinguishing between these requires inspecting the model's intermediate state, not just the final output.

Stage two: what happens during seller engagement?

Stage two is seller engagement. The model identifies which sellers to approach, determines opening positions, and begins multi-round negotiation. This stage tests the model's ability to translate the brief into a buying strategy: which sellers are most likely to satisfy which elements of the brief, how to prioritise outreach, and how to open negotiations in a way that creates room for counter-offers without conceding the brief's core requirements from the start.

Failures at stage two often manifest as poor seller selection: the model approaches sellers who cannot satisfy the brief's audience or channel requirements, invests negotiation rounds in deals that were never going to close, and runs out of viable options before the portfolio is complete. This is a strategy failure, not a portfolio assembly failure. The model knew the brief, but it did not translate the brief into a sensible engagement plan.

Stage two failures are also the most likely to produce technical interruptions: if the model's engagement strategy generates too many parallel negotiation threads that it cannot manage simultaneously, the task may stall before reaching a conclusion. In the Concourse data, five of 96 attempts were stopped by a technical issue before assessment was possible. Some of those stalls originate in stage two, where the model's engagement plan created a negotiation volume it could not sustain.

Stage three: what happens during portfolio assembly?

Portfolio assembly is the most demanding stage, and the one where most failures occur. The model must simultaneously track the state of multiple ongoing negotiations, apply the brief's portfolio-level constraints (budget, channel allocation, audience coverage) across all of them, and make decisions about which deals to close, which to continue negotiating, and which to walk away from, based on the cumulative state of the portfolio rather than the state of any individual negotiation.

Two distinct failure modes emerge at this stage. Budget overrun occurs when the model commits individual contracts that are each valid in isolation, but whose combined total exceeds the brief's budget. The model negotiated correctly at the individual deal level but failed to maintain accurate portfolio-level budget tracking. Every deal it committed to looked fine from the perspective of that negotiation. The portfolio as a whole violated the brief's most fundamental constraint.

Brief compliance failure occurs when the committed portfolio does not satisfy one or more of the brief's audience, channel, or quality requirements. This failure can arise from brief adherence problems (the model accepted terms outside the brief's requirements under seller pressure) or from coverage failures (the model closed deals in some channels but failed to reach agreement in others, leaving the portfolio incomplete against the brief's channel allocation requirements). The output looks like a completed buy. The brief is not satisfied.

Stage four: what happens at commitment?

Commitment is the final stage. The model has assembled a portfolio, determined that it satisfies the brief's constraints, and commits to the deals. Commitment produces a deal record: a document capturing the terms of each individual deal, the portfolio-level summary, and the model's assessment of how the portfolio maps to the brief.

The commitment stage is also where the deal record's quality is determined. A model that commits correctly but produces a deal record that is incomplete, ambiguous, or incorrectly structured creates a downstream governance problem. The deals may be valid. The record that should make them auditable may not be. For buyers who need to demonstrate to their clients or compliance functions that the portfolio was assembled within the brief, a poor deal record is a material failure even if the underlying deals are sound.

In the Concourse methodology, the deal record produced at stage four is assessed for both contractual quality (do the terms make sense and are they internally consistent) and brief adherence (does the record show that the portfolio satisfies the brief's requirements). Both dimensions are necessary for a completion to count. A deal that was closed but cannot be verified against the brief is not a complete buy.

Why does stage-level analysis matter for buyers?

A completion rate tells you what proportion of buying tasks a model finished. Stage-level analysis tells you where the failures occurred. A model with predominantly stage three failures (portfolio assembly) needs different improvement from a model with predominantly stage two failures (seller engagement strategy). Applying stage two remediation to a stage three problem will not fix it. Applying stage three remediation to a stage two problem will not fix it either.

For buyers choosing between AI buying systems, stage-level failure data is more informative than a headline completion rate for another reason: different buying tasks are more demanding at different stages. A campaign with a complex audience brief and multiple channel allocations is most demanding at stages one and three. A campaign with a tight budget and many potential sellers is most demanding at stages two and three. Knowing where a model's failure mode lies tells you whether that model's failure mode is likely to be triggered by your specific brief type.

The practical implication is that no buyer should make a deployment decision on the basis of a single completion rate figure. The question to ask is: where does this model fail, and is that the stage most relevant to the campaigns I intend to run? That question requires stage-level data. It requires exactly what the Concourse Bench methodology was built to produce.