Use A/B testing to optimize and improve your email marketing efforts and referral engagement. Send out two versions of your invite emails, then compare and analyze the metrics to see which one performs better.
A/B testing best practices
Plan your tests. Start with a clear goal and define what you want to learn. Are you testing your subject line, email body, banner images, or call-to-action buttons (CTAs)?
Optimize the content. Change one variable at a time. This helps you isolate which change was most effective in capturing your customers' attention.
Decide when to end. Set a duration or recipient count that marks the end of the test. Ending too early may skew the results, while testing too long may lead to inconsistent messaging that may confuse your customers.
Monitor the progress. Compare metrics like open rates and click-through rates. Identify which version drove more engagement and referrals.
Select the winner. Pick the better-performing email and roll it out to your customers. Use your findings to refine future email content and strategies.
Pro Tip: Read the Promoting your referral program article to learn how you can draft compelling and attention-grabbing referral emails.
Using A/B testing
A/B testing works differently for Shopify merchants and non-Shopify merchants using the ReferralCandy dashboard:
Automated invite email A/B testing (Shopify) - Fully automated testing that Shopify merchants can turn on for their campaign. The automation chooses the version that performs better based on metrics.
Invite email A/B testing (Non-Shopify) - Non-Shopify merchants manually create an alternative referral email and chooses the version that performs better.
Shopify
Start the automation (Shopify)
From your Shopify admin, go to Campaigns > Select your campaign. In the Promote campaign section, click Automations.
In the Automated Email A/B Testing section, click Turn On Automation to immediately run A/B testing.
A/B testing in the Shopify admin is fully automated. The system makes the necessary edits to the variables in the "Email B" or the variant email, and compares it against the content of your theme's Post-Enrollment Email, which is the "Email A" or the control email. How to edit your campaign's theme
Note: The A/B testing automation for Shopify supports English-only campaigns. Non-English and multilanguage campaigns aren't supported.
Automation selects the winner (Shopify)
50% of your customers receive "Email A" or the Control, and 50% receive "Email B" or the Variant. Monitor the page to analyze and compare the performance of the two versions. The automation stops after 14 days. If you want to stop the test early, click End Automation. Once stopped, the system selects the winner based on the available metrics and makes it the default invite email.
Other platforms
Set up your test email (Non-Shopify)
From the ReferralCandy dashboard, go to Campaigns > Select your campaign > A/B Testing.
Click Create a new test.
In the I want to test dropdown, select post purchase emails, also known as the post-enrollment or invite email. Only invite emails can be A/B tested.
Click Edit Email B to edit the variant. You can change the branding or the content of the invite email. Save your edits.
In the I want to start dropdown, select immediately to start the test now, or select at a specified time to schedule the testing.
Manually select the winner (Non-Shopify)
50% of your customers receive Email A, and 50% receive Email B. Monitor the performance of the two versions on the A/B Testing page. Once you reach an actionable result, stop the test by choosing to Keep Email A or Apply Email B. The winning variation becomes the default invite email.
A/B testing FAQ
Can I A/B test other emails, widgets, or rewards?
Can I A/B test other emails, widgets, or rewards?
No. Merchants can do A/B testing only on invite/post-enrollment emails.
However, there are plans to roll out A/B testing for other aspects of a referral or affiliate program, such as widgets, pages, and rewards. To stay informed about upcoming features, subscribe to the product newsletter, visit the product updates page, or contact us.
Why can't I access A/B testing? (Shopify)
Why can't I access A/B testing? (Shopify)
For Shopify merchants, the Automations page where the A/B testing can be turned on only shows up for English-only campaigns. The A/B testing automation won't work for campaigns that use another language or multiple languages.
If your campaign is English-only and you're still seeing issues with the A/B testing automation, contact us.
💬 Need help? Click the chat widget on the right to start a conversation. Learn more