Experiments let you run A/B tests on your checkout content, upsells, and post-purchase funnels to find out what works best. Create two or three variants, split traffic between them, and measure which version drives more revenue, higher conversion rates, or better average order values. Once you have a winner, roll it out to 100% of your customers with confidence.
Getting Started
From the sidebar, navigate to Experiments.
Click Create Experiment to open the experiment builder.
Experiments list page
Step 1: Variants and Traffic Split
The first step is where you set up what you're testing and how traffic is divided.
Experiment creation — variants and traffic split
Name Your Experiment
Give your experiment a clear name that describes what you're testing (e.g., "Product Carousel vs. Product List — Checkout" or "Post-Purchase Discount Test: 15% vs. 25%").
Choose Experiment Type
Select what kind of content you're testing:
Content — test different checkout content blocks or upsell blocks against each other
Post-Purchase — test different post-purchase funnels against each other
Set Up Variants
Every experiment has 2 to 3 variants:
Variant A (Control) — the baseline. This can either show no content (to measure the impact of having the block at all) or show an existing block/funnel that you want to beat.
Variant B — the challenger. Select the content block, upsell block, or post-purchase funnel you want to test.
Variant C (optional) — a second challenger for three-way tests.
To add a variant, click Add Variant. You can rename variants to keep things clear (e.g., "Carousel Layout" and "List Layout").
For each variant (except the control, if you want it to show nothing), select the specific content block, upsell block, or funnel to display:
Use the page filter to narrow by surface (Checkout, Thank You, or Order Status)
Browse your existing blocks/funnels and select one
The selected block appears in the variant card
Note: Once one variant selects an upsell block, all other variants must also be upsells (you can't mix content blocks and upsells in the same experiment).
Variant setup with block selection
Configure Traffic Split
The traffic split slider controls what percentage of eligible customers see each variant. Drag the slider handles or type exact percentages.
Each variant can receive between 5% and 95% of traffic
All variant weights must add up to 100%
Click "Split Evenly" to distribute traffic equally across all variants
For a standard A/B test, a 50/50 split is recommended. If you're testing something risky, consider an 80/20 split to limit exposure of the new variant.
Traffic splitting is deterministic — the same customer will always see the same variant across multiple visits, ensuring consistent experiences and accurate data.
Traffic split slider
Step 2: Settings
Click Next to configure how the experiment runs and what it measures.
Experiment settings step
Primary Metric
Choose the main metric you want to optimize for. This determines how results are displayed and which variant is declared the winner:
For Upsell and Post-Purchase Experiments:
Conversion Rate — percentage of visitors who purchased the upsell
Revenue — total upsell revenue generated
Total Revenue — total order revenue (including original order)
Upsell Revenue per Visitor — average upsell revenue divided by impressions
Total Revenue per Visitor — average total revenue divided by impressions
Average Order Value — mean order total
Average Items per Order — mean number of items
Acceptance Rate — percentage of customers who accepted the upsell offer
For Content Experiments:
Total Revenue — total order revenue
Total Revenue per Visitor — average revenue per visitor
Click-Through Rate — percentage of visitors who clicked the content
Average Order Value — mean order total
Average Items per Order — mean items per order
Completed Checkout Rate — percentage who completed checkout
Scheduling (Optional)
Set an optional time window for your experiment:
Start Date and Time — when the experiment begins (leave blank to start manually)
End Date and Time — when the experiment automatically stops
Times are shown in your store's timezone.
Auto-Stop Conditions (Optional)
Configure the experiment to automatically stop when it reaches a target:
Impressions Target — stop after a total number of impressions across all variants (e.g., 10,000)
Days Target — stop after a certain number of days running
Orders Target — stop after a certain number of orders/conversions
These help ensure you collect enough data for statistically meaningful results without running the test indefinitely.
Launching Your Experiment
After saving, your experiment starts in Draft status. From the experiments list:
Review your experiment setup
Click Start to begin running the experiment
Status changes to Running
Once running:
Traffic is split according to your configured weights
Impressions, conversions, and revenue are tracked automatically
For post-purchase experiments, the associated funnels are automatically enabled and managed
Important: You cannot edit an experiment while it's running. If you need to make changes, stop the experiment first, duplicate it, make your changes, and start the new version.
Starting an experiment from the list
Monitoring Results
Click on any running or completed experiment to view the Results Dashboard.
Experiment results dashboard
Key Metrics
The dashboard shows performance for each variant side by side:
Impressions — number of times the variant was shown
Conversions — number of successful actions (purchases, clicks, etc.)
Primary Metric — the metric you selected (conversion rate, revenue, etc.)
Uplift — percentage improvement of each variant compared to the control
Probability of Significance — statistical confidence that the difference is real and not due to chance
Audience Breakdown
Drill into results by audience segments:
Country — performance by customer country
Province/State — regional breakdown
City — city-level data
Market — Shopify market performance
Currency — by currency code
Customer Type — new vs. returning customers
UTM Parameters — by traffic source, medium, and campaign
Customer Tags — by customer tag segments
Date Range
Filter results by time period:
Last 3 days, 7 days, 30 days, 60 days
All time
Custom date range
Experiment Lifecycle
Experiments follow this lifecycle:
Status
What's Happening
Available Actions
Draft
Configured but not yet running
Edit, Start, Duplicate, Delete
Running
Actively splitting traffic and collecting data
Stop, View Results
Completed
Stopped (manually or via auto-stop)
View Results, Duplicate, Delete
After the Experiment
Once you've identified a winner:
Stop the experiment (if it hasn't auto-stopped)
Go to the winning variant's content block, upsell, or funnel
Make it your default by activating it for all customers (remove the experiment, keep the winner live)
Optionally duplicate the experiment to run follow-up tests
Tips
Run experiments for at least 7 days to account for day-of-week variations in traffic and buying behavior
Use the auto-stop impressions target to ensure statistical significance — a good starting point is 1,000 impressions per variant
Start with high-impact tests — test whether having an upsell at all improves revenue (control = no content vs. variant = your upsell) before testing layout variations
50/50 splits give you the fastest results. Only use uneven splits (80/20) when testing something you're uncertain about and want to limit risk.
Don't run multiple experiments on the same surface simultaneously — overlapping experiments can skew results. Test one thing at a time per page.
Check the probability of significance before declaring a winner — aim for at least 95% confidence. Results below this threshold may be due to random chance.
Audience breakdowns can reveal hidden insights — a variant might lose overall but win for mobile users or customers from a specific country. Use these insights to create targeted content.
