A/B Tests: How To Verify Difference In Average Order Value?
-
Hi there! When the data from an A/B test shows a difference in AOV between the variants, how do you determine if this difference is statistically probable or not? Best Regards, Martin
-
Thank you, David.
-
Perhaps this will get your on the right track, from http://flintanalytics.com/how-to-determine-if-you-should-advertise-higher-priced-goods-to-mac-users/
Average Order Value Statistical Significance
To do this you need to download the transaction data for both variants, because you need to be able to determine the variance in order value. You can download this data by going to conversions > ecommerce > transactions in Google Analytics. Once there, Google only lets you export 500 rows of transactions at one time, so to keep it simple I ensured my date ranges were under 500 transactions.
Once you get the order values for each set of customers in Excel you need to perform a t-test so that you can compare the means of the two groups. This t-test will give you the p-value that you will use to determine if the difference in average order values between the groups is significant. Yout formula in excel should look like the following:
=t.test(array of group 1, array of group 2,two-tailed distribution, two-sample equal variance homoscedastic)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is there an efficient way to block/filter referral spam in Google Analytics for a large network of websites?
Hello, everyone - I'm looking for guidance on how to block or filter referral spam in Google Analytics. But I'm needing to block for an entire network of Wordpress websites. We have two networks which total over 2,500 websites. We are currently blocking sites we find out about via htaccess. This works, but only after we see we are getting hit with the spam. Updating 2,500+ Google Analytics accounts with filtering is not an ideal option due to the time factor and the fact that new bots coming out almost daily. We can continue the htaccess method, but does anyone have any other ideas for blocking referral spam for a large network of sites? These are the other ideas we have. 1. Blocking all traffic from Russia and China based up subnets. We know many will still get through, but it should block 50% of it, we hope.
Reporting & Analytics | | copyjack
2. Moving sites to Google Tag manager. This is a huge tasks but we have seen that sites using Tag Manager are not effected, at least for now. Other ideas are appreciated!0 -
Google Analytics Average Position
I'm looking at Google Analytics -> Acquisition -> Search Engine Optimization -> Queries reports. I'm looking at keywords and the average position. What Google reports and what I see in a Google incognito search is different (usually my search is much lower). For example, for one search term, Google reports 5.8 average position and every time I search it is 8. My local result is 4. Anyone know why this is? I'm wondering if Google is averaging the Local results into number?
Reporting & Analytics | | CalicoKitty20000 -
Conversion via Paypal not showing order value in google analytics
Hello All, I have an e commerce site, when goal conversion happens by CC(credit card) I can see order value in e commerce tab in Google analytics but when conversion happens through paypal I always see order value zero. I want see order value for Pay pal also.Please help me out. Thanks
Reporting & Analytics | | Alick3000 -
Rank #1 for a 110,000/month query search, but barely any traffic?
Hi guys, As it says in the title, we've recently reached the absolute #1 position for a certain key phrase in the travel industry which the Google Keyword Tool tells me averages 110,000 local (165,000 global) searches a month... however we have received barely any traffic at all over the past TWO months for it and I'm trying my best to determine why. We've checked on multiple different devices with all forms of personalisation off, different browsers, 3G connections as opposed to office Wi-FI etc. and it still returns us as the #1 rank. Meta descriptions and title tags are pretty much pristine if I don't say so myself, however what should be a very lucrative key phrase is currently returning little to no traffic results. Has anyone had experience in a similar situation to this? Any possible causes that I might be missing? Would greatly appreciate any help. Thanks.
Reporting & Analytics | | ExperienceOz0 -
Is Analytics Code for SubDomains different?
I am currently tracking site performance on our website with Google Analytics. Recently, we've brought a Wordpress blog into our site as a subdomain blog.accupos.com. Though we did this a few months ago, we do not have Analytics data for these pages within the wordpress platform. I've found where to paste the code, in the Footer block of the Wordpress template, just before the close body tag (is this the best recommended way?) and just have one question about WHICH code to paste. Can I add the original code, same as the rest of the site? Or does Analytics have a different code for tracking subdomains? If so, do I need to replace existing code for the rest of the site? I hope I worded this okay. Please help. I'd appreciate it! Derek M.
Reporting & Analytics | | DerekM880 -
Cover my tracks / two websites on the same Analytics
Hi, I've got 2 websites and I like one of them to make some links to the other one. These two sites are on the same Google Analytics account. They have been set up 3 month ago and they have some articles on them. I've heard that Google can see that those websites are related in my Analytics and use this information to give less power to the link I could make between them. What do think about it and what do you recommend me ? Shoud I closed the Analytics account ? Or down one of the website and rebuilt it whith a different domain / host / analytics ? The thing is if I delete one of the website i'll be force to republish the articles, because there are links on it, and they're part of a deal I've already made. Thank you
Reporting & Analytics | | Spleen0 -
Google Search Results inconsistent from different computers
Recently after some optimization activities - I do not see much movement in search rankings - my client is seeing the results on page 1 position 3 and I see page 2 for the same keyword. How does Google change ranking based on past searches and how can I get an accurate picture of what the actual rank is?
Reporting & Analytics | | devonkrusich0 -
Google vs Bing/Yahoo SERP results?
I see large differences in Google and Bing/Yahoo SERP results for many keywords. Google shows many of our primary keywords in their top ten, while Bing/Yahoo rank the same one 40-50 or above. Do you have any insight on their differences that might account for this variation? We are legitimate, long time, white hatters at a small manufacturer that is one of the leaders in our industry. The only thing I can think of that might describe this issue is PPC advertising. Their algorithms wouldn't be that inconsistent would they? (We do much more paid advertising on Google) But generally this should have no affect, right? Thanks, George...
Reporting & Analytics | | rhawk0