Channel Measurement Score: Incrementality Confidence | MMA Global
  2. Channel Measurement Score: Incrementality Confidence
  2. Channel Measurement Score: Incrementality Confidence

Channel Measurement Score: Incrementality Confidence

Assess Your Channel Measurement

v1.1.4-249f35d1

The Ladder of Incrementality

More Incremental  
Least Incremental  

You can think of incrementality as a ladder of options that get closer to measuring true business value as you climb

  • •

    Randomized Experiments

    Trials to measure the precise difference between being exposed and not being exposed to an ad campaign.

    Method Pros Cons
    Randomized controlled trials Purest form of experimentation which gives the best read on incrementality Impossible to implement for certain tactics (e.g. linear TV) and hard for others. Not often offered.
    Ghost ads Purest form of randomized control trial (RCT) testing Proprietary method that is limited to walled garden offerings
    PSA placebo experiments Purest form of randomized control trial (RCT) testing Proprietary method that is limited to walled garden offerings
    Intent to treat or per protocol Close to purest form of randomized control trial (RCT) testing Unclear if those not reached with ads are somehow different, introducing noise into the test
    A/B Tests Advantages depend on how the test is implemented Term is used haphazardly and could refer to any form of RCT testing

     

  • •

    Quasi-Experiments and Incrementality Models

    Techniques that estimate (but don’t measure precisely) the incremental effect of being exposed to an ad campaign.

    Method Pros Cons
    Twinning A typical process to make unexposed consumers as well matched as possible to those exposed. Twinning is done on factors (like demos) that are presumed to be important. These factors can vary.
    Weighting A typical process to make unexposed consumers as well matched as possible to those exposed Weighting is done on factors (like demos) that are presumed to be important. These factors can vary.
    Counterfactual Modeling Closely matches the intent of measuring incrementality: what would have happened had the consumers not been exposed to the treatment variable? Model structure and parameter estimation need to be set up in valid ways
    Doubly robust estimators Estimators combine two modeling approaches to reduce bias and improve the accuracy of causal effect estimation. By using both models, doubly robust estimators provide more consistent estimates of the causal effect even if either one of the models is mis-specified. No downside to adding doubly robust estimators to other weighting or matching protocols.
    Targeted maximum likelihood Statistical method used for estimating treatment effects in observational studies or complex data settings. It can also incorporate prior knowledge Complicated and subject to model misspecification.
    Pre/post designs Eliminates the issue of matching a test and control set of IDs Requires persistent IDs which are not always available. There can also be exogenous factors that affect conversion patterns from the pre to post period that are not adjusted for.
    Spike modeling Less data burden as this method can be run off of more readily available aggregated data. Aggregated data can be divorced from the underlying process and miss the true effect of the causal variables.
    Geo-based testing Test marketing is easy to understand and a traditional method for measuring treatment effects Factors can vary across markets, such as retailers being different, that make the results non-comparable

     

  • •

    Non-Incremental Models

    Systems that don’t make an explicit estimate for an ad campaign’s effect above a baseline of behavior (i.e., what a person would have done anyways without seeing an ad campaign).

    Method Pros Cons
    Counting Simple and widely available Rarely reflects incrementality
    Expert Opinion There are always experts who have seasoned insights Opinions made be wrong in specific instances

     

Download the Research

The series introduces the concept of incrementality, describes various popular methodologies commonly used to estimate advertising ROI, and reveals a set of actionable steps marketers can take to dramatically improve the accuracy of performance measurement.

  • The What and Why of Incrementality:
    An introduction of the importance of measuring incremental impact

  • The Ladder of Incrementality:
    A description of various popular methodologies for estimating advertising ROI

  • Climbing the Ladder of Incrementality:
    A set of actionable steps advertisers can take to dramatically improve accuracy of performance measurement and optimization

Watch the webinar: Measuring Incrementality to Untangle Marketing Contribution

Recorded September 13, 2023 as part of MMA Global's Marketing Measurement Great Debates.

Join our mailing list