Use A/B testing to optimize and improve your email marketing efforts and referral engagement. Send out two versions of your invite emails, then compare and analyze the metrics to see which one performs better.
Note: Email A/B testing isn't supported for campaigns using multiple languages. Testing is only available for English-only campaigns.
A/B testing best practices
Plan your tests. Start with a clear goal and define what you want to learn. Are you testing your subject line, email body, banner images, or call-to-action buttons (CTAs)?
Optimize the content. Change one variable at a time. This helps you isolate which change was most effective in capturing your customers' attention.
Decide when to end. Set a duration or recipient count that marks the end of the test. Ending too early may skew the results, while testing too long may lead to inconsistent messaging that may confuse your customers.
Monitor the progress. Compare metrics like open rates and click-through rates. Identify which version drove more engagement and referrals.
Select the winner. Pick the better-performing email and roll it out to your customers. Use your findings to refine future email content and strategies.
Pro Tip: Read the Promoting your referral program article to learn how you can draft compelling and attention-grabbing referral emails.
A/B testing emails
A/B testing in Shopify admin is fully automated and AI-assisted. The system makes the necessary edits to the variables in the "Email B" or the variant email, and compares it against the current content of your theme's Post-Enrollment Email, which is the "Email A" or the control email. How to edit your campaign's theme
Start the automation
From your Shopify admin, go to Campaigns > Select your campaign. In the Promote campaign section, click A/B testing.
If you're using an affiliate campaign, go to Other settings > Advanced settings.
In the Auto-test referral emails section, select Post enrollment email and click Run automated test.
Click Start test to immediately run A/B testing.
Manage tests
You can only do one invite or referral email A/B test. The A/B testing page contains three tabs. The Active tab contains your live tests, the Ended tab displays a history tests that you completed, and the All tab contains all your active and ended tests.
Note: The A/B testing automation for Shopify supports English-only campaigns. Non-English and multi-language campaigns aren't supported.
Automation selects the winner
50% of your customers receive "Email A" or the Control, and 50% receive "Email B" or the Variant. Monitor the page to analyze and compare the performance of the two versions. The automation picks a winner every 14 days and generates a new variant for testing. The automation will keep running until you stop the test.
To stop the automation:
Go back to the A/B testing page.
Find the active Post-enrollment email test and click to view. Use the side-by-side data comparison between the control and variant performances to pick a winner.
Click Stop test.
A popup appears. Select between Variant A (Control) and Variant B (AI-generated).
Click End test. The system sets the version you selected as the default invite email.
Once the test ends, you can create a new one. Ending a test is irreversible.
A/B testing FAQ
Can I A/B test branding elements in the invite email?
Can I A/B test branding elements in the invite email?
Only the content of the post-enrollment email is A/B tested. Branding elements such as your logo, fonts, and colors aren't part of the automated test.
Why can't I access A/B testing? (Shopify)
Why can't I access A/B testing? (Shopify)
The A/B testing page only shows up for English-only campaigns. The A/B testing automation won't work for campaigns that use another language or multiple languages.
If your campaign is English-only and you're still seeing issues with the A/B testing automation, contact us.
💬 Need help? Click the chat widget on the right to start a conversation. Learn more