A/B testing allows you to test different advocate reward or friend offer amounts to see which one works best for your referral or affiliate program.
Note: Reward A/B testing doesn't support campaigns using FlexiTiers or multiple languages. Testing is only available for single-tier reward and English-only campaigns.
A/B testing rewards
Compare reward variations and use metrics to decide which reward performs better for your campaign. You can only test rewards one at a time, but a post-enrollment email A/B test can run simultaneously with a reward A/B test.
For advocate rewards, you can test cash, coupon, and store credit reward types. A/B testing doesn't support custom rewards.
For friend offers, only coupons can be tested. The Nothing reward can't be tested against the coupon friend offer type.
Create a new test
Only the reward type, value and description can be tested and analyzed during A/B testing. All other reward settings remain the same for both variants such as expiry dates, minimum order amounts, and discount combinations.
To create a new reward A/B test:
From your Shopify admin, go to Campaigns > Select your campaign. In the Promote campaign section, click A/B testing.
If you're using an affiliate campaign, go to Other settings > Advanced settings.
In the What do you want to A/B test? section, select Advocate reward or Friend offer. If you want to test the two simultaneously, select both items.
Click Create test.
In the Test settings section, enter the Test name.
In the Advocate reward or Friend offer section, your current reward setup is displayed as Variation A. Go to Variation B to set up a new reward that will be compared against Variant A.
Save the changes.
The test runs immediately once saved. You can't edit your campaign's current reward setup while A/B testing.
Manage tests
You can only do one test at a time and must end an active test to start a new one. The A/B testing page contains three tabs. The Active tab contains your live tests, the Ended tab displays a history of tests you completed, and the All tab contains all your active and ended tests.
Note: You can't make changes to your campaign's language settings while you're running an A/B test.
Review and end a test
Tests must be ended manually when enough metrics are available for you to make a decision. To manually end an active reward A/B test:
From your Shopify admin, go to Campaigns > Select your campaign. In the Promote campaign section, click A/B testing.
If you're using an affiliate campaign, go to Other settings > Advanced settings.
Find the active test. Click to view and compare the below metrics:
First referral revenue - Revenue from the first purchase referred friends made
First referred purchases - Number of first purchases referred friends made
Conversion rate - Percentage of enrolled contacts who made a referral
Advocate or Affiliate views - Total view count of pages and emails by advocates and affiliates
Advocate or Affiliate shares - Total number of referral link shares by advocates and affiliates
Store visits - Number of visits to your store
Friend views - Number of friends who visited the friend offer landing page or viewed the friend offer popup
Once you generate a sufficient and reliable data, click the Review and End test button at the top of the page.
A popup appears. Select the new reward setup for the campaign and click End test.
After the test ends, your chosen variant's reward setting is immediately applied to your campaign.
Important: You can't restart or undo ended reward A/B tests.
A/B testing rewards FAQ
I use Klaviyo to send out marketing emails. Can I A/B test rewards?
I use Klaviyo to send out marketing emails. Can I A/B test rewards?
It's not recommended to A/B test rewards if you're using Klaviyo to send out marketing emails, as this feature is not yet compatible with the Klaviyo integration. Stay tuned for updates related to this.
Why can't I A/B test my reward? Why is my reward A/B test not working?
Why can't I A/B test my reward? Why is my reward A/B test not working?
There are a couple of reasons why you can't use reward A/B testing:
Your campaign is paused or stopped. Learn more
You're using FlexiTiers or a multi-tier reward structure
Your campaign uses multiple languages
You already have a test running. You must end the active reward A/B test to start a new one.
Your friend offer is set to Nothing
The following are also not possible to test:
Different tier structures. For example, a multi-tiered structure vs. single reward. Reward A/B testing is only available for single reward campaigns.
Campaign reward setup against another campaign's reward. Testing is only available for a reward setup within a campaign.
💬 Need help? Click the chat widget on the right to start a conversation. Learn more