A/B Tests: How To Verify Difference In Average Order Value?
-
Hi there! When the data from an A/B test shows a difference in AOV between the variants, how do you determine if this difference is statistically probable or not? Best Regards, Martin
-
Thank you, David.
-
Perhaps this will get your on the right track, from http://flintanalytics.com/how-to-determine-if-you-should-advertise-higher-priced-goods-to-mac-users/
Average Order Value Statistical Significance
To do this you need to download the transaction data for both variants, because you need to be able to determine the variance in order value. You can download this data by going to conversions > ecommerce > transactions in Google Analytics. Once there, Google only lets you export 500 rows of transactions at one time, so to keep it simple I ensured my date ranges were under 500 transactions.
Once you get the order values for each set of customers in Excel you need to perform a t-test so that you can compare the means of the two groups. This t-test will give you the p-value that you will use to determine if the difference in average order values between the groups is significant. Yout formula in excel should look like the following:
=t.test(array of group 1, array of group 2,two-tailed distribution, two-sample equal variance homoscedastic)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Same spam traffic coming from two different sources
This week we did two big email sends from completely separate sources and separate lists. However, in GA I see spam traffic coming from the same city and the same service provider when I look at the report for each source. Screenshots attached. I'm hoping someone can help me understand what I'm looking at and what I should do in the future to prevent. Thanks! g9NTZ5z X5WhYnf
Reporting & Analytics | | eGInnovations0 -
Google Analytics Goal/Event/SOMETHING to show only Wordpress "Posts", not pages, etc
Hi all, Our site is build on Wordpress and formerly the post URL's had the typical date format at the beginning. This made it easy for me to look at, for example, all search traffic to the blog. I would just view URL's containing /2014/ and /2015/ and boom. We have since removed the dates from the URL's with proper redirects etc, which is great, but now I can't figure out a way to look at ONLY the blog in GA. I like to track a KPI of 'search visits to blog posts' and I can't figure out how to now. Can I set up a GA event that only fires when the post type template for blog posts loads? Some other solution? I'm lost here, and there's gotta be a good way to do it...
Reporting & Analytics | | 3DR0 -
Links from PA/DA of 1
My immediate thought is that a link from a website that has a PA/DA of 1 for both or either metric would be absolutely worthless, but I just wanted to make sure. Could this also be from it being recently indexed? Also is there a rule of thumb that anybody knows, or tries to follow at least? Obliviously the higher the better, but for example; A PA/DA of 15 and below is SPAM A PA/DA of 30 and above is GOOD A PA/DA of 50 is GREAT PA/DA of 80 is AMAZING PA/DA of 100 is GODTIER
Reporting & Analytics | | HashtagHustler0 -
What is the difference between External Followed Links, Total External Links and Total Links?
What is the difference between Total External Links and Total Links? Below is my metric and 3 competitors. | External Followed Links | 12 | 43 | 227 | 5 |
Reporting & Analytics | | seohlp44
| Total External Links | 19 | 51 | 296 | 11 |
| Total Links | 2,990 | 704 | 15,234 | 12,518 |0 -
Difference between GWT and GA average position
Does anyone know why there would be a difference in average position from GA and GWT? I am seeing completely different results for the same date ranges. Thank you!
Reporting & Analytics | | bloomingB0 -
Can you use Google tag manager to manage rich snippets/schema mark up?
Hi,Does anyone know if you can use the new Google tag manager to manage rich snippets? I've seen that there is an HTML section where you can edit HTML that isn't shown on the site - do you think this field could be used to add schema data?Thanks,Karen
Reporting & Analytics | | Digirank0 -
Why Webmaster tools shows different avg. positions?
Hi everyone, I am new to Seomoz, I loved this amazing SEO school 🙂 My questions is about webmaster tool. Webmaster tool shows that my site's avg. positions is 34 and its getting better every week. I even see some new queries, new avg. positions for that queries. That's great ! But when I search the avg position it is between 54 -60 and I don't see any new queries . Shouldnt we rely on what webmaster tells us about avg. positions? Thanks a lot
Reporting & Analytics | | EzgiGunyel-InfinPixels0 -
Google Analytics: Difference Between Goal Conversions & Goal Completions
When using Google Analytics, what is the difference between total goal conversions and total goal completions? We have many goals set up in a lead generation environment. Therefore, the only element of conversion is submitted a lead and arriving on the "Thank You" page. THose thank you pages are tagged accordingly. When we run reports though, the number of "Total Goal Conversions" and "Total Goal Completions" never match up.
Reporting & Analytics | | eMagineSEO0