A/B Tests: How To Verify Difference In Average Order Value?

[TalkInThePark]

Hi there! When the data from an A/B test shows a difference in AOV between the variants, how do you determine if this difference is statistically probable or not? Best Regards, Martin

[TalkInThePark]

Thank you, David.

[David_ODonnell]

Perhaps this will get your on the right track, from http://flintanalytics.com/how-to-determine-if-you-should-advertise-higher-priced-goods-to-mac-users/

Average Order Value Statistical Significance

To do this you need to download the transaction data for both variants, because you need to be able to determine the variance in order value. You can download this data by going to conversions > ecommerce > transactions in Google Analytics. Once there, Google only lets you export 500 rows of transactions at one time, so to keep it simple I ensured my date ranges were under 500 transactions.

Once you get the order values for each set of customers in Excel you need to perform a t-test so that you can compare the means of the two groups. This t-test will give you the p-value that you will use to determine if the difference in average order values between the groups is significant.  Yout formula in excel should look like the following:

=t.test(array of group 1, array of group 2,two-tailed distribution, two-sample equal variance homoscedastic)