Shopify A/B testing

Show half the shop B.
Let the carts vote.

Yes, Shopify has native A/B testing now. Rollouts, under Markets › Rollouts since 31 March 2026, runs an Experiment that splits visitors 50/50 by default between your store as it is and a changed theme, checkout configuration, catalog or discount. Experiments need the Grow plan or higher. They take theme editor changes or a whole replacement theme, not Liquid template edits. For element-level tests matched by URL, use a Shopify A/B testing app or a script in theme.liquid. Below: which route fits, a planner that says how long your test must run, and where Builderify A/B Testing fits.

Plan my test Start free
Native A/B testing Rollouts Experiments on Grow and up

By the Builderify team

Illustration · an invented shop, scored with the same z-test as the planner below. The price never changes.
01 / Shopify Rollouts

Shopify Rollouts experiments:
the native A/B test.

A rollout is a scheduled set of changes to your main theme, checkout and accounts pages, catalogs or discounts. Shopify announced it on 31 March 2026 and added checkout and customer account configurations on 5 June 2026. There are three types, and only one of them is an A/B test.

Launch

Basic and up

Merges your changes into the store permanently, to everyone at once or to a share of visitors you raise over time.

No end date. The changes stay.

Event

Basic and up

Publishes your changes for a set period, such as a seasonal sale, then rolls them back.

End date required. You can apply it permanently instead.

Experiment

Grow and up

Compares a treatment (your changes) with a control (the store as it is) on split traffic, without showing the change to everyone.

50% control, 50% treatment and an end date 90 days out, both by default and both adjustable.

What a Shopify Rollouts experiment can change, and the limits on each change
Resource The change Limits
Online store theme Edit the main theme in the theme editor One theme change per rollout. No Liquid template changes, no vintage themes.
Online store theme Replace the main theme Add the theme to your store first. Its content and translations change only through a compatible app such as Translate & Adapt.
Checkout and accounts Replace the configuration One per rollout. Create the configuration first.
Catalogs Set a catalog to Active or Draft Publishes or withdraws its prices and availability. In an experiment it applies to the online store only, so ads and feeds may show a different price.
Discounts Activate or deactivate Up to 250 per rollout. The control keeps your usual discounts. A code can be shared outside the test.

From Shopify’s Rollouts help pages, read on 4 October 2026. Each is linked in the sources at the end.

What an experiment reports

Theme experiment
Conversion rate over time, bounce rate, reached checkout rate, add to cart rate.
Checkout and accounts experiment
Checkout conversion rate.
How significance is judged
Not stated in the help pages.

Gross sales and average order value appear for Launches, not in the experiment lists. You can’t customise the metrics, and visitors on headless storefronts and POS aren’t counted.

Where

Shopify admin › Markets › Rollouts. A theme edit opens from Online Store, under Rollouts on your main theme’s card.

Traffic

Traffic allocation sets the share of eligible visitors; the split divides them between control and treatment. Under 100% of either needs an end date.

Overlaps

Several experiments on one resource each get their own visitors; a visitor is in one rollout per resource. Launches take their traffic first.

Markets

Targeting a rollout at one market is on the Advanced and Plus plans, according to the 31 March announcement.

02 / Rollouts, app or script

Rollouts or a
Shopify A/B testing app?

Start with Rollouts if you’re on Grow or higher and the change lives in the theme editor, checkout, a catalog or a discount. Reach for an app when you need revenue metrics or price tests on a smaller plan. A script in your theme is the light option for copy, images and layout on chosen pages.

Three ways to A/B test a Shopify store compared: Rollouts experiments, A/B testing apps, and a script in the theme
Question Rollouts experiment A/B testing app Script in theme.liquid
Shopify plan Grow or higher Depends on the app Any plan where you can edit theme code
How you make B Theme editor changes, or a whole replacement theme The app’s editor, duplicate templates or themes, or a second URL A visual editor on the live page, saved against CSS selectors
Prices and discounts Yes, through catalogs and discounts On some plans (see the tags below) No. Changing the number on the page doesn’t change what Shopify charges
Checkout Yes, a checkout and accounts configuration Some apps list checkout tests No. Theme scripts don’t run in Shopify’s checkout
What it measures Conversion, add to cart, reached checkout and bounce rates Usually orders and revenue too; Shoplift lists revenue per visitor Conversions on goals you set: a page, a click or a custom event
Statistics Not stated Shoplift and Elevate state Bayesian models Builderify: two-proportion z-test, two-sided 95%
Cost Included in the plan A monthly subscription Builderify: credits per launch and per 1,000 results

Shopify’s US prices on 4 October 2026: Basic $39 a month, or $29 a month billed yearly; Grow $105, or $79 yearly; Advanced $399, or $299 yearly; Plus from $2,300. Prices differ by country.

Four well-known apps, from their own pricing pages on 4 October 2026

Shoplift

From $99/mo

Core plan, up to 50k monthly store visitors. 14-day free trial.

  • Theme, template and URL split tests
  • Bayesian analysis; revenue per visitor and add to cart goals
  • Price testing listed on Advanced ($399/mo)
Shoplift pricing →

Elevate

From $49/mo

Basic plan, 0 to 1,000 monthly orders, billed monthly.

  • Theme, template and split URL tests, visual editor
  • Bayesian statistics model
  • Price and checkout flow tests from Standard ($99/mo)
Elevate pricing →

Intelligems

By orders and modules

You build a plan from monthly orders and the modules you pick.

  • Content: landing pages, on-site edits, themes, templates
  • Pricing, shipping and checkout modules
  • Reports profit, revenue and order value
Intelligems pricing →

Convert

From $399/mo

Growth plan, or $299/mo paid yearly, 100K tested users a month. 15-day free trial.

  • A/B and split URL tests on any website
  • Shopify price testing listed on Growth
  • Not Shopify-only
Convert pricing →

Not on Shopify? The Squarespace A/B testing guide covers a builder with no native testing at all.

03 / Planner

Plan a Shopify A/B test,
then check the result.

Before: how many sessions the page needs, and how many days that takes. After: whether B really won. It uses Builderify’s rule, which gives no verdict until each version has 100 visitors and 10 conversions.

1 · Before you start95% · 80% power
Visitors needed per version
13,914
Detects
3.00% → 3.60%
Both versions
27,828
Days at this traffic
35 days
In whole weeks
5 weeks

Round up to 5 full weeks, so every day of the week counts the same.

2 · When it’s doneTwo-sided z-test
A Original
B Change
Rate A
3.00%
Rate B
3.70%
Lift
+23.3%
z
2.13
p-value
0.0331
Confidence
96.7%
B wins

B converts better, p = 0.0331 (two-sided). That clears the 95% bar.

3 · Revenue per visitor

Revenue per visitor is conversion rate times average order value. If B converts more shoppers but sells them smaller baskets, conversion rate rises while revenue falls, and the reverse. Add each version’s revenue above to see both.

Revenue per visitor and average order value for each version
Measure A B B vs A
Revenue per visitor — — —
Average order value — —  

Add each version's revenue from your orders report to compare revenue per visitor and average order value.

In your store currency. Testing whether a revenue gap is real needs each order’s value, not the totals, which is why this panel compares rather than judges.

Normal curve

Φ(z) = ½(1 + erf(z / √2)), with erf from Abramowitz and Stegun 7.1.26 (error under 1.5 × 10−7). Φ(1.96) comes out at 0.9750, as in any printed z-table.

Result

Pooled rate p = (cA + cB) / (nA + nB); z = (pB − pA) / √(p(1 − p)(1/nA + 1/nB)); p-value = 2(1 − Φ(|z|)). B wins or loses when |z| ≥ 1.96.

Plan

n per version = (1.96√(2p̄q̄) + 0.84√(p1q1 + p2q2))² / (p2 − p1)². Assumes a 50/50 split, one goal, a two-sided test, 80% power, no continuity correction, and no stopping early.

04 / A product page

How to split test
a Shopify product page.

In Rollouts, edit the product template in the theme editor inside an Experiment. With a script, change the elements themselves on the URLs you choose. Either way, test one idea at a time and pick the goal before you start.

With a Rollouts experiment

  1. 1 In your Shopify admin, go to Markets › Rollouts and click Create rollout. Name it and choose Experiment.
  2. 2 Click Save, then Add changes. Under Prepare changes to your current theme, click Edit theme.
  3. 3 In the theme editor, open the product template and make the change: a heading, a block, a section. Save, then Apply.
  4. 4 In Audience, set the traffic allocation and the split. In Scheduling, set when the experiment runs; the end date defaults to 90 days.
  5. 5 Save. Results appear when you click the rollout in Markets › Rollouts. End it with Apply changes to keep B, or Roll back changes.

Mid-test edits: Shopify warns that changing the treatment or control while an experiment is active can affect its results. Parts of the theme you didn’t customise in the rollout keep syncing with your published theme.

With a script: what to change, and what not to

  1. 1 Headline. A text change. Only the words change; any markup inside the heading stays.
  2. 2 Product image. An attribute change: a different src or alt.
  3. 3 Price. Leave it. Rewriting the number on the page doesn’t change what the cart charges. Test prices with Rollouts catalogs or a price-testing app.
  4. 4 Buy button label. Careful. Many themes, Dawn among them, rewrite the button and the price when a shopper picks another variant, which can undo a change made as the page loaded.
  5. 5 Under the button. Insert after: a delivery or returns line. Only promise what your store actually does.
  6. 6 Description. Replace its HTML, or hide a block to see if less is more.
01

One product, many URLs

A product opens at /products/handle, under a collection at /collections/…/products/handle, and under a market’s language folder. Write the URL rule to match all of them.

02

Checkout is off limits

Theme code doesn’t run in Shopify’s checkout, and since 26 August 2026 the old Thank you page scripts are gone for every store. A script tool counts add to cart or reaching the cart.

03

Themes are separate

Code you add to theme.liquid lives in that theme only. If a Rollouts experiment swaps in a replacement theme, add your scripts there too.

04

Traffic is thin

A product page gets a fraction of the homepage’s sessions. Run the numbers in the planner before you commit to a small change.

05 / With Builderify

A/B test a Shopify theme
on any plan.

Builderify A/B Testing is the script route. It changes page content in the shopper’s browser, on the pages your URL rules pick, and scores the result with the planner’s z-test. It doesn’t need the Grow plan, and it doesn’t touch prices or checkout.

  1. 1 In Builderify, open A/B Testing and click Install to copy your site’s A/B tag.
  2. 2 In Shopify, go to Online Store, click … › Edit code on your theme and open layout/theme.liquid.
  3. 3 Paste the tag on the line after <head>, as high as it will go, and save.
  4. 4 Create a test. For a product, a wildcard URL rule such as */products/cold-brew-kit matches the plain, collection and language-folder URLs.
  5. 5 Make the change in the visual editor, add a goal (a click on [name="add"], Dawn’s Add to cart button, or a visit to /cart), and start.

The tag loads before the page paints, on purpose, so shoppers don’t see A flash before B. Shopify’s editor runs Theme Check, which can flag parser-blocking scripts; for this tag that is expected. It hides only the elements being changed and shows them again within 1.5 seconds whatever happens. If the regular Builderify snippet is already in your theme, keep it; the A/B tag sits beside it.

What Builderify A/B Testing can and can't do on a Shopify store
On Shopify it can It can’t
Make six kinds of change: text, HTML, an attribute (such as href, src or alt), hide, insert before, insert after Change prices, discounts or what Shopify charges
Pick pages with URL rules: exact, contains, starts with, or a wildcard Run in checkout or on the Thank you page
Send a share of traffic into the test and weight each version Edit Liquid or anything rendered on Shopify’s servers
Keep each shopper on the same version (sticky assignment) Measure revenue or order value: it counts conversions
Count goals of its own: a page visit, a click or a custom event, without User Journey switched on Re-apply a change after the theme redraws part of the page, such as on a variant switch

For a goal that sits in another part of the store, the guide to tracking conversions by URL shows how to pick a page that only buyers reach.

The planner’s test, on BuilderifyCredits
Launch the test25
Results (28,746, 1/1,000)28.7
The whole test53.7

The planner’s defaults: 27,828 visitors, each recorded once, plus their expected conversions. Every account gets 250 free credits a month. One-time packs start at $9 and never expire. See pricing.

Start free

See A/B Testing in full, follow shoppers across the store with User Journey, or browse more guides.

Questions

Shopify A/B testing, before you ask.

Yes. Rollouts, under Markets then Rollouts and announced on 31 March 2026, includes an Experiment type that compares a treatment with a control. By default it splits traffic 50/50 and ends 90 days after it starts. Experiments need the Grow plan or higher, while Launches and Events work from Basic.

Theme changes made in the theme editor or a whole replacement theme, a checkout and accounts configuration, catalogs set to Active or Draft, and discounts. You can't change Liquid templates, vintage themes aren't supported, and each rollout takes one theme change and one checkout change. A theme experiment reports conversion, bounce, reached checkout and add to cart rates.

Until each version reaches the sample size you planned, rounded up to whole weeks. A product page with 800 sessions a day, converting at 3%, needs about 13,914 visitors per version to detect a 20% lift at 95% confidence and 80% power, which is about 35 days. Builderify gives no verdict until each version has 100 visitors and 10 conversions.

Not if you are on Grow or higher and Rollouts covers your change. Apps such as Shoplift, Elevate, Intelligems and Convert add revenue metrics and price tests on their own subscriptions. A script in theme.liquid, such as Builderify A/B Testing, works on any plan for content changes on the pages your URL rules pick.

No. It changes page content in the shopper's browser: text, HTML, attributes, hiding or inserting elements. That doesn't change what Shopify charges, and theme scripts don't run in checkout. For price tests use Rollouts catalogs or a price-testing app; Builderify counts goals such as add to cart clicks or reaching the cart.

Sources

Help, changelog and pricing pages, read on 4 October 2026.

← All guides

Stop losing the visitors
you already paid for.

More leads from the traffic you already have, and follow-up that sends itself. Free to start. No card, no monthly fee.

Start free See what it does