Shipping Tests - QA Checklist

Ensuring accurate shipping setups with essential QA steps

This article provides a comprehensive checklist for performing shipping tests. It will guide you through the preparation steps, help you conduct tests efficiently, and ensure your shipping setup works as expected.

🏁 Pre-Test Checklist

Before starting your shipping tests, ensure the following requirements are met:

Due to Shopify constraints, Carrier service API is not available for Basic Shopify store.

• Carrier service API is enabled.

• Shipping script is enabled. (see guide)

• ABConvert shipping rate is added to the shipping zone that you want to test:

  • Go to Shopify settings > Shipping and delivery > select the shipping profile that you want to test.

If ABConvert Shipping rate is NOT added, simply click "Add rate" and add it to the zone.

ABConvert shipping banner is not available for advanced shipping tests.

• (Optional) shipping banner is enabled. (see guide)

🔗 Prepare preview links

To review your test configurations and test results, follow these steps:

• Go to the Test Analytics Page and click “Summary.”

• The Summary Block will pop up, showing your test configuration and providing links to preview the test.

✅ Use preview links to QA your test

There are three types of shipping tests available. Simply scroll to the relevant test section to view its QA checklist:

Free shipping threshold

• Rate Display

  • The rate shows up correctly when the cart total meets or exceeds the threshold.

  • Free shipping does NOT show up if the cart total is below the threshold.

  • Deleted rates do NOT show up.

  • Shipping rate and description match the test configuration.

• (Optional) ABConvert shipping banner text displays correctly across different groups.

• The test does NOT impact shipping zones that are not included.

  • Enter an address from a different shipping zone to verify.

• Cart

  • Cart items are displayed correctly. If line item properties appear, please refer to this guide.
  • Cart functions correctly, including adding items, removing items, and updating item quantities.

Flat rate shipping

• Rate Display

  • The rate shows up correctly across test groups.

  • Deleted rates do NOT shows up.

  • Shipping rate and description match the test configuration.

• (Optional) ABConvert shipping banner text displays correctly across different groups.

• The test does NOT impact shipping zones that are not included.

  • Enter an address from a different shipping zone to verify.

• Cart

  • Cart items are displayed correctly. If line item properties appear, please refer to this guide.
  • Cart functions correctly, including adding items, removing items, and updating item quantities.

Advanced shipping

• Rate Display

  • The correct shipping rate is displayed based on order value, weight, or other conditions.

  • Shipping rates update dynamically when the cart value or weight changes.

  • Deleted rates do NOT appear.

  • Shipping rate and description match the test configuration.

• The test does NOT impact shipping zones that are not included.

  • Enter an address from a different shipping zone to verify.

• Cart

  • Cart items are displayed correctly. If line item properties appear, please refer to this guide.
  • Cart functions correctly, including adding items, removing items, and updating item quantities.

❓ FAQs

Q1: What will happen if the test rate’s title is the same as my original rate?

In Shopify, if multiple rates share the same title, it will return the lowest rate. Therefore, please avoid using the same title for the test rate.

For Advanced Shipping tests, make sure to create distinct conditional pricing rules to prevent conflicts.

Q2: When will deleted rates be added back to the shipping zone?

Once the test is paused or ended, the deleted rates will be restored to the shipping zone.

Q3: There are text elements displaying shipping rates in the store, such as a self-built shipping banner, policy text, or shipping progress bar. How can these be managed?

A3: Custom shipping scripts for handling text or shipping progress bars are only available for Advanced or Plus plan users. To learn more about this feature, please refer to this article.

By following the pre-test checklist and QA steps, you can ensure that your shipping settings function correctly across different scenarios. Whether you’re testing Free Shipping Threshold, Flat Rate Shipping, or Advanced Shipping, this checklist will guide you through the process to verify rates, configurations, and ensure a smooth customer experience.