A/B Tests: How To Verify Difference In Average Order Value?
-
Hi there! When the data from an A/B test shows a difference in AOV between the variants, how do you determine if this difference is statistically probable or not? Best Regards, Martin
-
Thank you, David.
-
Perhaps this will get your on the right track, from http://flintanalytics.com/how-to-determine-if-you-should-advertise-higher-priced-goods-to-mac-users/
Average Order Value Statistical Significance
To do this you need to download the transaction data for both variants, because you need to be able to determine the variance in order value. You can download this data by going to conversions > ecommerce > transactions in Google Analytics. Once there, Google only lets you export 500 rows of transactions at one time, so to keep it simple I ensured my date ranges were under 500 transactions.
Once you get the order values for each set of customers in Excel you need to perform a t-test so that you can compare the means of the two groups. This t-test will give you the p-value that you will use to determine if the difference in average order values between the groups is significant. Yout formula in excel should look like the following:
=t.test(array of group 1, array of group 2,two-tailed distribution, two-sample equal variance homoscedastic)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search Console Crawl Errors/Not Found - Strange URLs
Hello, In Google Search Console under Crawl > Crawl Errors > Not found I have strange URLs like the following: https://www.domain.com//UbaOZ/
Reporting & Analytics | | chuck-layton
https://www.domain.com//UPhXZ/
https://www.domain.com//KaUpZ/WYdhZ/SnQZZ/MOcUZ/ There is no info in Linked From tab. Have you seen this type of error??
Does anyone know whats causing it??
How should it be fixed?? Thanks for reading and the help!0 -
Difference Between Android Browser & Android Webview
Hello All, In google analytic I can see traffic from android browser & android webview so android webview is also a browser? Thanks!
Reporting & Analytics | | dhisman0 -
When migrating to https should you unlink the verified http version from my GA account?
Hi We had a new site design at the end of April and at the same time changed to https. Over the last 6 months, according to our analytics account the traffic has dramatically dropped especially from mobile. We were at 3607 new visitors from mobile in April and now 32! in September. Technically (other than page speed and some meta issues that I'm aware of) there doesn't initially appear to be anything wrong. I have just been into search console and notice that the old http site still has the GA code on it, could it be that this is why the data is showing such a massive drop? Should we have taken that off when we migrated the site? The same analytics account is also on the new site. Thanks in advance Janine
Reporting & Analytics | | Janine10 -
Differences in site search revenue in GA
I just put in a piece of software to replace a really bad built in site search engine on my 3dcart website. Now I am trying to measure the change, but I am having some issues. When I check the ecom data in the conversions section of GA with the built in segment Performed Site Search, I get promising results. Approximately 5% revenue increase over LY. But if we jump to behavior, site search, usage, and then check the visits with site search, I get a decrease by 4%. And the actual revenue is off, by like double (150k compared to 80k) Anyone have any idea why I am getting these results? The site search function is set up. Tracking is enabled, query parameter is keyword and search url is /search.asp?keyword=
Reporting & Analytics | | ShockoeCommerce0 -
How do you setup a test analytics account?
I have created a test view but that is not helping with skewed data. Do you all just create a total separate profile with a different GA account for a non-indexed domain? How can I test tag manager and other things without messing with our data? I have our IP filtered out so it doesn't show up in reports but other event tags from outsiders will.
Reporting & Analytics | | Sika220 -
Cookies/Tracking/Code developement
So. I have an online application form, on the website. It's maked with php, html and css. What i like to place on the email, is a cookie tracking system. For example, when someone search on the google for " web design ", and then see my result on google organic, comes the visitor to my site, charge the fields of application forms and click send. Than, me like an admin, i receive an email with informations of a customer, i see name/surname/etc.. but i like to integrate is for example, like to see a: IP from Visitors, who charged the fields Referrerr, if google, like to see which query he typed. How can i do this please?
Reporting & Analytics | | leadsprofi0 -
Google Analytics Content Experiments don't deliver 50/50?
Our A/B test is actually delivering at about a 70/30 page view rate. 70% in favor of the original version and only 30% of the new. We are sending 100% of our traffic to this homepage test. Has anyone else experienced this? There seems to be a lot of folks experiencing this.....anyone know why?
Reporting & Analytics | | VistageSEO0 -
Cover my tracks / two websites on the same Analytics
Hi, I've got 2 websites and I like one of them to make some links to the other one. These two sites are on the same Google Analytics account. They have been set up 3 month ago and they have some articles on them. I've heard that Google can see that those websites are related in my Analytics and use this information to give less power to the link I could make between them. What do think about it and what do you recommend me ? Shoud I closed the Analytics account ? Or down one of the website and rebuilt it whith a different domain / host / analytics ? The thing is if I delete one of the website i'll be force to republish the articles, because there are links on it, and they're part of a deal I've already made. Thank you
Reporting & Analytics | | Spleen0