Shipping Tests - How to Set Up
Set up a Shipping Test with ABConvert in a few simple steps
Last updated
Set up a Shipping Test with ABConvert in a few simple steps
Last updated
To begin a new shipping test in ABConvert, click on "Create New Test" located at the top right of the screen. When running the shipping test for the first time, a message will appear at the top, prompting you to check if your Shopify store supports carrier rates. Please refer to the above information for further details about carrier rates. After the carrier rate check is complete, your store will be able to begin running Shipping Tests.
Make sure you double check if your carrier rate is successfully enabled in the Settings -> Shipping and Delivery.
📌 We normally recommend you remove the origin rate during the testing period in order to prevent conflict. However, if you have multiple rate such as Standard v.s. Express. You just have to remove the rate that you are testing.
Remember, do name the Shipping rate different than the existing rate (the rate you have not removed), otherwise Shopify will always return the rate with lower price if there are rates with the same name.
Once in the Shipping Test page you're presented with all the options required to get your shipping test up and running. We'll walk through the different blocks available for configuration in this section.
This block contains the test name which is used to easily reference this test and analytics in the future as well as the hypothesis (description) for what you're hoping to achieve. This cannot be changed once a test has started.
This section allows you to configure what % of traffic will see which variant. We recommend leaving this on 50/50 to get the most accurate results.
🎯 Initially you might notice a significant disparity in the amount of traffic between the two variants, but over time, it will even out to within 1 - 2% of the predetermined percentage.
ABConvert offers three types of tests, namely the Free Shipping Threshold, Flat Rate Shipping, and Advanced Shipping.
In the variants section, you can specify the free shipping threshold for each group if you choose the free shipping threshold test type, or set the flat rate shipping rates if you choose the flat rate shipping test type. The values are defined in your store's default currency.
The shipping countries section is where can you choose which country your test will be active for. Please note that all checkouts with products in a selected shipping country will be part of the shipping test.
🎯 Although you can conduct multiple shipping tests concurrently, each active test is restricted to one test per country. This ensures that we obtain the most precise outcomes for every test and country you're evaluating.
Before launching your shipping test, the Checkout Configuration section needs to be set up. This section allows you to customize how the rate is displayed at checkout by choosing a shipping rate name and creating a personalized shipping description.
ABConvert is the ONLY shipping test app to offer the unique feature of setting a custom transit time to measure the effects of shorter or longer shipping times. At the bottom of this step, a checkout preview is available to show you precisely what will be displayed during checkout and what rates might appear depending on the test group.
That's it! Your new test is all setup and ready to go. Just hit the next button at the bottom and we'll start providing shipping rates. If there are any errors we'll also let you know prior to the test going live.
Shopify currently returns only one rate with the lowest price if there are multiple rates with the same name. You will need to delete the rate you are testing to ensure accurate shipping.
You can either delete your rate manually in your "Shipping and Delivery" settings in Shopify and add the rate back after you end the test, or you can select the rate to delete in our final review modal. Our app will delete the rate and recover it for you after you close the test.
⚠️ Multiple rates might be displayed, or your test rate might not appear if you do not delete the rate you are testing.
After create a shipping test, you can navigate to the analytics page and click summary to preview the test. Or you can follow the below instruction.
🎯 Not seeing the right experiment on your store? To provide a fast and seamless experience for your customers we cache experiment data for a period of up to 5 minutes. This won't affect customers' checkout capabilities. If you'd like to test it yourself without waiting, simply append `?preview=0&country=<country code>` or ?preview=1&country=<country code> to your URL bar, allowing you to preview the ongoing experiment.
You can look up the country codes here:
Check out these resources to level up your shipping test:
Learn about setting up Advanced Shipping Tests:
Learn about setting up a free shipping banner:
