Analytics Discrepancies

We outline the primary reasons for differences to Shopify's analytics

A common question from users is: Why is the numbers in the ABConvert analytics dashboard different from Shopify? While it may seem like data should match across platforms, the reality is that tracking methods differ significantly, which lead to discrepancies. Below, we outline the primary reasons for these differences.

1. Differences in Visitor and Session Definitions

Definition	ABConvert	Shopify
Visitor	A visitor is equal to a session that lasts 30 minutes. It resets after 30 minutes (no matter if there is activity or not) or when a new browser session begins. Visitors are counted once per unique purchasing journey.	A visitor is defined as a unique individual who interacts with the store. Visitors are deduplicated across sessions and devices.
Session	Sessions are the same as visitors. A session lasts 30 minutes. It resets after 30 minutes (no matter if there is activity or not) or when a new browser session begins.	Sessions are defined as 30 minutes of continuous activity, resetting after inactivity or new browser sessions.

2. Differences in Event Tracking Methods

Definition	ABConvert	Shopify
Event Recording	Events (e.g., add-to-cart, checkout) are recorded at most once per session to prevent double counting. Each visitor or session can only attribute one event per funnel step.	Events are recorded based on Shopify’s predefined tracking logic, which may include multiple events within a single session.
Data Collection Methods	Uses web pixels, JavaScript scripts, and webhooks to collect data directly from user interactions during tests.	Relies on its analytics infrastructure to collect data from various store interactions and channels.
Data Collection Limitation	Checkout event tracking: if customers use shop pay or express checkout, the events cannot be tracked in ABConvert due to the exclusion of web pixel event.	N/A

3. Inclusion/Exclusion of Bot Traffic

ABConvert employs anti-bot measures to ensure only real visitor data is recorded. Events like product views or add-to-cart actions are only captured if they are tied to genuine user interactions.

4. Privacy Settings and Browser Restrictions

• Modern browsers like Safari block third-party cookies and certain JavaScript trackers by default. Additionally, privacy tools like ad blockers can prevent analytics scripts from firing. This will result in discrepancies in conversion metrics.

• ABConvert uses first-party scripts and web pixels for tracking but may still encounter issues with blocked events in some environments (e.g., TikTok’s in-app browser).

---

Order Discrepancies Between Product Tests and Shopify Reports

Order discrepancies often occur when comparing ABConvert's product test reports with Shopify's order data:

• ABConvert: Attributes orders to a test if the visitor interacted with a tested product page at any point during their session. This includes cases where the final purchase contains products that were not part of the test. The logic is based on the idea that the tested experience influenced the visitor's behavior during their shopping journey.

• Shopify: Attributes orders directly to specific products or collections included in the purchase. It does not consider whether the visitor interacted with a tested product page earlier in their session unless that product is part of the final transaction.

For example:

• If a visitor views a tested product page but purchases an unrelated item later in their journey, ABConvert attributes that order to the test group because the tested experience influenced their behavior.

• Shopify would not attribute this order to the tested product unless it was directly included in the purchase.

---

Recommendations for Users

To minimize confusion when comparing analytics:

1. Understand how each platform defines key metrics like sessions, visitors, and orders.

2. Use ABConvert’s analytics dashboard to focus on metrics specifically designed for A/B testing performance evaluation, such as: AOV and conversion rates.

3. For deeper validation, export raw data from both platforms (e.g., order IDs) and compare manually.

If you encounter discrepancies that seem unusual or unexplained, feel free to contact our support team for assistance via the in-app support widget.