When should you stop an A/B test in a DOOH campaign?

Short answer

Stop when your pre-defined confidence level, minimum sample size, and minimum duration are all met simultaneously. For major budget decisions, target 95% statistical confidence and at least 1,000 to 2,000 attention-adjusted impressions per variant, run for a minimum of 2 to 4 weeks.

Statistical significance: choosing the right threshold

The confidence threshold you set before launch determines when results are reliable enough to act on.

Target 95% confidence for decisions with significant budget or strategic implications, such as long-term media plan shifts. For rapid creative or message testing where iterations are frequent and learnings will be validated across multiple rounds, 90% confidence can be acceptable.

The key principle is to set these rules before the test begins. Pre-defined stopping rules prevent "peeking," where early results trigger ad hoc decisions that inflate false-positive rates. A rule might read: "Stop when variant B achieves 95% statistical significance AND the minimum sample size is reached." Without this, DOOH's noisy, time-variant audiences make it easy to misread transient spikes as real effects.

If performance stabilizes but never clears 95%, consider the size and consistency of the effect. A large, consistent lift across locations and time periods may justify stopping at 90% confidence. A small effect below 10% with weak statistical support should be treated as exploratory only, and major budget reallocations should wait.

Sample size: what counts as enough data

Raw spot plays are not an adequate sample size metric for DOOH. Use attention-adjusted impressions or unique audience reach, which reflect actual human engagement rather than mere exposure.

For creative A/B tests focused on proportion metrics such as attention rate or interaction rate, aim for at least 1,000 to 2,000 attention-adjusted impressions per variant for directional insights. For incrementality tests measuring downstream outcomes like store visits or app opens, aim for hundreds of outcome events per variant to detect meaningful lift reliably.

Low conversion rates, common in brand-building DOOH campaigns, require either longer durations or aggregated data from multiple locations to reach sufficient statistical power. Use sample size calculators calibrated to your expected baseline rates and minimum detectable effect before the campaign launches, not after results come in.

Audience segment tests follow the same logic as creative tests. Each segment must meet the same sample size and duration thresholds independently. Fragmenting audiences across too many segments reduces learning reliability for all of them.

Test duration: minimum runtimes by objective

Never stop a test within the first 3 to 5 days, even if one variant appears dominant. Early data is frequently skewed by transient externalities such as events, weather anomalies, or traffic disruptions that do not reflect steady-state performance.

For lower-funnel metrics such as store visit lift or QR code scans, run tests for at least 2 to 4 weeks. This window covers weekday and weekend cycles, multiple dayparts, and weather-triggered conditions that affect audience composition and behavior.

For brand-lift or consideration metrics, expect multi-month test durations. Matched-market or synthetic-control designs also require at least 6 months of pre-campaign baseline data for stable estimates. These longer windows are not optional: the measurement method depends on having a reliable pre-period to compare against.

Context-triggered campaigns, where ads fire based on weather or other real-time conditions, should be measured by triggered exposure counts rather than calendar duration. A two-week calendar window that includes little triggering activity does not constitute an adequate test.

DOOH-specific factors that affect stopping decisions

Several characteristics of DOOH environments require adjustments to standard A/B testing practice.

Audience variability: DOOH audiences fluctuate daily and weekly due to weather, local events, and traffic patterns. Tests must span enough time to average across these fluctuations rather than capturing an unrepresentative slice.

Frozen variants: Avoid changing creatives, audience definitions, or targeting triggers during the test. Any mid-test modification invalidates the comparison for the period after the change.

Practical significance: Statistical significance alone is not sufficient justification for stopping. Confirm that the observed difference is large enough to have a tangible business impact before reallocating budget or deploying a winner.

Alternative approaches: Classical fixed A/B tests are not the only option. Bayesian testing or multi-armed bandit approaches can be used for iterative or adaptive optimization, where the goal is to shift budget toward better-performing variants in real time rather than waiting for a fixed endpoint.

How AdQuick handles DOOH campaign testing

Planning and executing a rigorous DOOH A/B test requires clean data at every stage, from impression counting through outcome measurement. DOOH advertising through AdQuick gives buyers access to inventory alongside the planning and measurement infrastructure needed to define stopping rules before launch, track attention-adjusted impressions across locations, and evaluate lift against pre-campaign baselines. Rather than assembling data from disconnected sources, campaigns managed through a unified marketplace produce the consolidated reporting needed to apply the significance, sample size, and duration criteria described above with confidence.