Understand how the holdout group enables unbiased comparisons in AI Optimize.
The holdout group is a segment of visitors in AI Optimize optimizations who always see your original page content. It provides a clean, unbiased baseline to help the AI measure the true impact of your optimizations — even during periods of fluctuation.
What the holdout group is
The holdout group is a randomly selected portion of your site's traffic that never sees optimized variations. Visitors in this group always receive your original content, regardless of how many optimizations you’ve created. The rest of your traffic — the optimization group — is eligible to see different combinations of variations.
- Baseline measurement — lets you compare optimized and unoptimized visitor behavior during the same time period
- Randomized and persistent — visitors are randomly assigned and remain in the holdout group during their session
- Fixed allocation — the default split is 20% for holdout and 80% for Optimization, but you can adjust this
Why the holdout group matters
Comparing results from before and after an optimization isn't always reliable — factors like seasonality, market changes, or external events can skew performance. The holdout group gives you an apples-to-apples comparison, isolating the impact of your optimizations.
For example, imagine a period of market instability causes overall performance to dip. Without a holdout group, you might assume your changes caused the decline. With one, you can see that optimized visitors still performed better than those in the holdout group — and confidently keep the changes in production.
Note
The holdout group helps you measure real impact, even when external factors affect sitewide performance.
Holdout vs. No Change
Both the holdout group and the No Change variation show your original content — but their purposes and scope are different:
- Holdout group — shows the original content across your entire site; used to measure global optimization impact
- No Change — acts as a control in a specific optimization; used to compare performance of variations in that optimization
Important
Visitors in the holdout group never see any optimized content making them an unbiased comparison group.
Change the holdout group percentage
The holdout group is most effective when it reflects how your overall audience behaves. If you're unsure about the right percentage for your site, contact support for personalized guidance. As a general rule, the holdout percentage should include at least 100 sessions, users, or page views per day with a conversion rate of at least 10%.
Open your Optimize site in Webflow, then:
- Click Account settings in the Navigation panel
- Go to Holdout group
- Click Edit
- Set the desired percentage for the holdout group
- Click Save