With experiments, you can A/B test multiple visual element changes on your site or application to validate or reject a hypothesis (e.g., "changing banner design will increase conversions by 10%"). Experiments allow you to apply statistical analysis methods to ensure that the primary metric change you notice is not caused by a change or error.

📘

Why experiment?

Controlled experiments are the best scientific way to establish causality between your product features and customer impact. Establishing such causality allows you to only ship features that improve customer experience. This can make experiments the driving force behind your pace of innovation.

As you grow your pace of innovation, experiments also enable you to also measure the success of the features you ship and uncover unexpected side effects with every code change. This allows you to iterate faster in the short term, establish key business drivers, and make better, evidence-driven business decisions every day.

In comparison, relationships observed in historical metrics cannot be considered structural or causal because multiple uncaptured external and internal factors influence customer behavior. Historical metrics establish correlation, not causation.

Use cases

• Trial length optimization. Compare the performance of different trial lengths to determine which leads to higher conversion rates
• Landing page optimization. Experiment with different value propositions to identify which variants drive more sign-ups or demo requests
• Product page optimization. Test different arrangements of elements on product pages to see which layout leads to higher engagement and conversion rates.
• CTA button optimization. Test variations in CTA buttons to identify which versions lead to more completed purchases
• Checkout flow optimization. Compare a multi-step checkout process with a simplified, single-page checkout to determine which leads to lower cart abandonment rates

Where to start

• Server-side vs. client-side A/B testing & personalization

Client-side experiment

• Creating client-side experiments
• Using the visual editor

Server-side experiment

• Creating server-side experiments

Experiment analytics

• Analyzing the data
• Confidence intervals
• Statistical significance
• Sequential testing
• CUPED