Beyond ROAS: A Small-Brand Guide to Ad Incrementality Testing

Navigating the current digital landscape as a Shopify brand founder means ensuring every advertising dollar stretches further than ever. Customer acquisition costs have skyrocketed, climbing approximately 60% since 2020 (Causality Engine). Plus, Apple's privacy updates have obscured up to 75–85% of iOS activity, leaving advertisers grappling with diminished visibility (Causality Engine). As a result, your Facebook or Google dashboard might boast a stellar return on ad spend (ROAS), but the pretty numbers could be misleading.

ROAS Optimization: The Complete Guide for E-commerce BrandsNov 06, 2025

Comprehensive guide to roas optimization: the complete guide for e-commerce brands with real examples and actionable strategies.

Causality EngineCausality Engine

Let’s get real: many founders are chasing vanity metrics. One DTC marketer confessed,

“We were swimming in positive ROAS reports, but our profit margins weren’t budging. It highlighted a major disconnect between the marketing dashboard and the P&L” (Advids).

The ROAS Illusion: Measuring True Incrementality in Video Retargeting with the paROAS Blueprint.

Dismantle the ROAS illusion. Measure true video retargeting incrementality with our blueprint for predictable, profitable growth.

AdvidsOrko Roy

The solution? Shift your focus from surface-level ROAS to incrementality—in simple terms: identify which sales wouldn’t exist without the ad. That's why 71% of advertisers now prioritize incrementality as their top KPI (Dataslayer). Let’s explore how lean teams can test incrementality, pinpoint what’s truly driving growth, and optimize for genuine success.

Incrementality: Top Retail Media KPI for 2025

71% of advertisers now rank incrementality as their #1 retail media KPI. Learn why ROAS isn’t enough and how to measure true campaign impact across RMNs.

Julia MorenoNovember 4, 2025

Why ROAS Can Be a Mirage

ROAS looks enticing—everyone loves seeing “4x” on their dashboards. But if those conversions would have occurred regardless, your real return might be substantially less. The issue: ad platforms often inflate ROAS by 40–60% due to double-counting and overlapping attribution (Causality Engine).

Consider a DTC brand that saw Meta report a 4.2× ROAS, but incrementality testing showed the real lift was only 2.1× (Causality Engine). TikTok flaunted a 3.5×, but the incremental ROAS was a mere 1.8×. That’s your ad budget vanishing into thin air.

How does this happen? Platforms often take credit for conversions that would have occurred anyway—think branded searches. If a loyal customer searches your brand and clicks your ad, Google counts it as a win. But you probably would have captured that sale organically. Data indicates 70–90% of branded search ad conversions aren’t truly incremental; retargeting performs slightly better at 40–60%, while prospecting can reach 80–95% (Causality Engine).

Optimizing solely for ROAS results in overfunding strategies that inflate your metrics but don’t attract new customers. Without caution, you end up paying for sales you would’ve secured anyway.

What Incrementality Really Means (and Why Founders Care)

Incrementality examines the real cause-and-effect: Which sales wouldn’t have happened without our marketing efforts? It’s the difference between being busy and being effective.

Incrementality testing isolates your ads’ impact by creating holdout groups—segments that don’t see a specific campaign—and then measures the lift between them and the exposed groups. This is the “gold standard” for assessing ad effectiveness (Causality Engine).

Take Newton Baby, a DTC mattress brand: incrementality tests revealed their branded search ads contributed zero incremental sales. None. All that spend, all those Google-reported conversions—none of it was net new (AdExchanger).

How Incrementality Tests Helped Newton Baby Ditch Branded Search | AdExchanger

In the past year, Baby product and mattress brand Newton Baby has put all its media channels through a new testing regime for incrementality.

AdExchangerJames Hercher

That’s the type of clarity founders seek. It enables budget reallocation with confidence, reduces waste, and amplifies what’s truly effective. It’s also why even retail media networks are pressured to report incremental lift, not just ROAS (Digiday).

ROAS vs. incrementality: Scrutiny around ad spend grows in booming retail media space

In the fast-growing channel that is retail media, marketers are prioritizing incrementality over ROAS to justify ad spend.

DigidayKimeko McCoy

Incrementality Testing—Lean, Not Just for Big Brands

You don’t need a statistics degree or a Fortune 500 budget to conduct incrementality tests. Here’s how agile DTC teams can get actionable insights quickly:

Geo Holdout Experiments

Divide your advertising by geography. Run ads in Region A, hold them in Region B, then compare sales. If California outperforms Texas after running TikTok ads there (all else being equal), you’ve identified your incremental lift. Pair similar cities or ZIPs for accuracy, and avoid testing during major promo periods (DTC Fashion Decoded).

Stop Getting Scammed! Incrementality 101

Practical DTC growth advice for fashion brands

PSA or “Placebo” Ads

No clear geo split? Use PSA (Public Service Announcement) ads. Display a neutral, non-sales ad (like a hydration tip or charity message) to a portion of your audience, while the rest view your real campaign. The incremental lift is the difference in purchase rates (Demand Local). Platforms like Meta can automate this via their “Conversion Lift” tools.

On/Off “Pause” Tests

Sometimes, simplicity is key: pause a channel and observe the effect. Halt Google Search for a week—if sales remain steady, those dollars weren’t incremental (Digiday). Use this cautiously, maintaining steady variables during the test.

Built-in Platform Lift Tests

Meta, Google, and now TikTok offer platform-native lift testing tools. These tools randomly split your audience behind the scenes, then report the true incremental lift and ROAS. The challenge often lies in minimum spend, but if you qualify, it’s the quickest way to a definitive answer (Causality Engine).

Turning Insights into Action (and Revenue)

Running tests is just the beginning. The real victory lies in leveraging what you discover. Found a channel with low incremental lift? Slash or reduce spend there—even if the platform ROAS appears “high.” Reallocate those funds to campaigns that genuinely drive new customers.

Consider a mid-sized beauty brand: incrementality testing exposed that 40% of their “conversions” were non-incremental. They cut branded search spend by 60%, reinvested in prospecting and retargeting, and emphasized conversion rate optimization. The result? Blended ROAS surged from 2.1× to 3.8×, and revenue soared 150% in four months (Causality Engine).

This shift—focusing on incremental customer acquisition cost (nCAC) and incremental ROAS (iROAS)—transforms budget allocation, creative planning, and growth scaling.

The Founder’s North Star: Incrementality, Not Illusion

Ultimately, moving beyond ROAS is about intellectual honesty and operational discipline. ROAS isn’t useless; it just doesn’t tell the whole story. By consistently running incrementality tests, even in basic forms, you’ll know precisely where your ad dollars are impactful—and where they merely flatter your ego.

The best operators now ask, “How much new business will this drive?” instead of “How high can I elevate my ROAS?” That’s how you steer clear of overfunding tactics that look impressive on paper but hinder real growth (LinkedIn).

Ad platform reported ROAS is a terrible north star metric for your advertising program. Here’s why: 1. Misaligned Incentives: It incentivizes focusing on the bottom of the funnel because it's easy… | Eric Sloan

Ad platform reported ROAS is a terrible north star metric for your advertising program. Here’s why: 1. Misaligned Incentives: It incentivizes focusing on the bottom of the funnel because it’s easy to measure. However, the bottom of the funnel is not how you grow a business. 2. Low-Incrementality Decisions: It drives decisions that ignore the value of channels and tactics that are hard to measure. 3. Easily Manipulated: Any smart marketer can boost ROAS at the end of a quarter by manipulating spend or media mix to make platform reports look better without actually driving impact to the business. So, what’s the solution? Don’t completely discard ROAS as a metric, but put it in context. Here are some ways to get started: ✅ Map Assumptions: Map out your assumptions about the incrementality of every media channel in your program. For example, if you’re looking at branded search ROAS, start by cutting it in half and see if it’s still profitable. Cut it even more if you have low competition. Repeat this for every channel in your media program. If you’re not confident in the assumption: run an experiment. ✅ Visualize Data: Visualize platform ROAS over the past 12 months against actual business performance metrics, like top-line revenue or marketing efficiency ratio. Did your media program improve performance in the last three months, but top-line sales stayed the same? It’s likely that the “optimizations” made just lowered incremental impact instead of driving stronger business results. ✅ Run Lift Testing: This is the most powerful way to understand the impact of your campaigns on business results. Start with your most impactful or highest investment channels, and run lift or geo experiments to better understand the incremental impact on sales.

LinkedInView Profile

Don’t be disheartened if your initial tests seem chaotic or the insights are imperfect. The true value lies in the habit: regular, rigorous, founder-driven measurement. It’s the maintenance check your growth engine requires.

Next time you review your marketing metrics, don’t ask “What’s my ROAS?” Instead, ask “What’s my incremental ROAS—and how can I boost it?” That’s the founder’s playbook for profitable scaling.

Subscribe for weekly DTC operator insights.