Use A/B testing to optimize and improve your email marketing efforts and referral engagement. Send out two versions of your invite emails, then compare and analyze the metrics to see which one performs better.
Note: Email A/B testing isn't supported for campaigns using multiple languages. Testing is only available for English-only campaigns.
A/B testing best practices
Plan your tests. Start with a clear goal and define what you want to learn. Are you testing your subject line, email body, banner images, or call-to-action buttons (CTAs)?
Optimize the content. Change one variable at a time. This helps you isolate which change was most effective in capturing your customers' attention.
Decide when to end. Set a duration or recipient count that marks the end of the test. Ending too early may skew the results, while testing too long may lead to inconsistent messaging that may confuse your customers.
Monitor the progress. Compare metrics like open rates and click-through rates. Identify which version drove more engagement and referrals.
Select the winner. Pick the better-performing email and roll it out to your customers. Use your findings to refine future email content and strategies.
Pro Tip: Read the Promoting your referral program article to learn how you can draft compelling and attention-grabbing referral emails.
A/B testing emails
A/B testing works differently for Shopify merchants and non-Shopify merchants using the ReferralCandy dashboard:
Automated invite email A/B testing (Shopify) - Fully automated testing that Shopify merchants can turn on for their campaign. The automation continuously picks the better-performing version until you manually stop the test.
Invite email A/B testing (Non-Shopify) - Non-Shopify merchants manually create an alternative referral email and chooses the version that performs better.
Shopify
A/B testing in the Shopify admin is fully automated and AI-assisted. The system makes the necessary edits to the variables in the "Email B" or the variant email, and compares it against the current content of your theme's Post-Enrollment Email, which is the "Email A" or the control email. How to edit your campaign's theme
Start the automation (Shopify)
From your Shopify admin, go to Campaigns > Select your campaign. In the Promote campaign section, click A/B testing.
If you're using an affiliate campaign, go to Other settings > Advanced settings.
In the Auto-test referral emails section, click Setup automation.
Click Turn On Automation to immediately run A/B testing.
Manage tests (Shopify)
You can only do one invite or referral email A/B test. The A/B testing page contains three tabs. The Active tab contains your live tests, the Ended tab displays a history tests that you completed, and the All tab contains all your active and ended tests.
Note: The A/B testing automation for Shopify supports English-only campaigns. Non-English and multi-language campaigns aren't supported.
Automation selects the winner (Shopify)
50% of your customers receive "Email A" or the Control, and 50% receive "Email B" or the Variant. Monitor the page to analyze and compare the performance of the two versions. The automation picks a winner every 14 days and generates a new variant for testing. The automation will keep running until you stop the test.
To stop the automation, go to the test on the A/B testing page and click End Automation. Once stopped, the system selects the winner based on the available metrics and makes it the default invite email.
Other platforms
Set up your test email (Non-Shopify)
From the ReferralCandy dashboard, go to Campaigns > Select your campaign > A/B Testing.
Click Create a new test.
In the I want to test dropdown, select post purchase emails, also known as the post-enrollment or invite email. Only invite emails can be A/B tested.
Click Edit Email B to edit the variant. You can change the branding or the content of the invite email. Save your edits.
In the I want to start dropdown, select immediately to start the test now, or select at a specified time to schedule the testing.
Manually select the winner (Non-Shopify)
50% of your customers receive Email A, and 50% receive Email B. Monitor the performance of the two versions on the A/B Testing page. Stop the test by choosing to Keep Email A or Apply Email B based on the data captured by the system. The winning variation becomes the default invite email.
A/B testing FAQ
Can I A/B test branding elements in the invite email?
Can I A/B test branding elements in the invite email?
If you're a Shopify merchant, only the content of the post-enrollment email is A/B tested. Branding elements such as your logo, fonts, and colors aren't part of the automated test.
If you're using another platform, you can A/B test both the branding and content of the Email A and Email B variant.
Why can't I access A/B testing? (Shopify)
Why can't I access A/B testing? (Shopify)
For Shopify merchants, the A/B testing page only shows up for English-only campaigns. The A/B testing automation won't work for campaigns that use another language or multiple languages.
If your campaign is English-only and you're still seeing issues with the A/B testing automation, contact us.
💬 Need help? Click the chat widget on the right to start a conversation. Learn more