CTV Incrementality Testing with Geo Holdouts, DACH and US

Sizing the test before the spend: To detect a five percent lift on a base of about 4,000 weekly conversions, you need roughly eight to ten weeks with a quarter to a third of geography held out. To detect two percent you need most of a year, by which time the budget decision has already been taken without you. So the first output here is not a test. It is a number: the smallest effect your own data can prove inside a window you can actually wait through. If that number turns out to be larger than the effect worth having, we say so and you keep the money. Geography in Germany: The workable unit is the two-digit postal area. There are 95 of them, IP-based geo on connected TV resolves to that level dependably enough to build on, and below it, at five-digit postal level, targeting leaks badly enough to contaminate a control. Matched pairs get built on pre-period conversion series rather than population or income, because two regions with similar demographics and different seasonality make a worse pair than two mismatched regions whose conversion curves move together. Design: Matched-market holdout, read with Bayesian structural time series against a synthetic control built from the untreated regions. Everything is pre-registered before the first impression serves: hypothesis, minimum detectable effect, power, the stopping rule, and the exact query that will produce the number. That document is the point. Deciding what counts as success after seeing the data is how every campaign in this industry finds a lift. Where ghost bids and PSA controls fit: Ghost bids are the cleaner design where your DSP exposes them, because the control group is defined by the auction rather than by geography. On connected TV we usually decline PSA controls: you pay full rate for control impressions, and server-side insertion makes the control group's frequency unknowable, so you have bought an expensive approximation of a holdout. Why we do not use the seller's study: A lift study run by the company selling the media is that company marking its own homework, and it is almost always built on a thirty-day post-view window that credits the channel with purchases already in motion. We will read a seller's study and we will tell you what it does and does not establish. We will not put its number in your board deck as the answer. Triangulation: Where a marketing mix model already exists, the holdout result becomes its calibration point rather than its competitor. Two methods that disagree are more informative than one method that agrees with itself, and reconciling them is part of the work rather than an awkward footnote at the end of it. When we tell you not to run it: Below roughly 1,500 weekly conversions, no design in this document will separate signal from noise in a sensible window. If your media cannot be split geographically, the holdout is unavailable and we should be talking about a different design. If a single retail partner controls regional promotion, their calendar will overwhelm your treatment effect and the test will measure them instead of you. What we hand over: The pre-registration document, signed and dated before spend. The counterfactual series with credible intervals. The notebook that produced it, in R or in your warehouse, runnable by your own analysts without us. And one decision stated plainly: spend more, spend less, or the channel did not move.