How to hold a variable constant for an A/B test

[LivDetrick]

For example, let's say you want to A/B test a title tag change. You are hoping to identify whether a title tag change increases CTR. But, position is always fluctuating a bit and that affects CTR, too. So, I'm interested in how you could hold position constant in order to isolate the change in CTR that is due to the title tag change.  Does anyone know of resources/tools/tutorials for how to do this?

It's been... a very long time since I took statistics (-:  I have access to Excel, MS Access, and R studio.

[LivDetrick]

Thanks for the reply. I'll check out the links you included.

I thought it might be helpful to show an example of what got me thinking along these lines. This is a study about A/B testing of title tag changes. They don't say they accounted for position fluctuation; maybe they didn't, but it seems like you'd have to in order to have meaningful results.

https://www.rankscience.com/coderwall-seo-split-test

[KevinBudzynski]

I don't see a way of doing this accurately. With organic rank fluctuations, seasonality, competition it's near impossible to "hold" a position. However, if you have a position in serps for a specific keyword with little fluctuation and get decent traffic you can establish this as your baseline.  Once the title is updated and appears in the serps, begin monitoring w/search console, analytics or etc. and see if you see any decent jumps.  Some links for additional information: Finding the ROI of Title tag changes using Google's CausalImpact R package,  SEO Split Testing & A Beginner’s Guide to A/B Testing: Effective SEO Landing Pages. Would love to hear if anyone else has a better way.