A/B Split Testing - Rankings Drop? Need an expert opinion...

[AMHC]

We are running an A/B split test (Started on 12/12), and a few days after we started the test, we fell from position 9/10 (about 3 weeks on page 1) to position 11/13, and we've been there ever since.

We are still running the test. We disallowed the test page in robots.txt, but that's all.

• no canonical
• no noindex
  -no Google Experiments code.

Theoretically, Google could crawl the site, and find the page, but then the page is disallowed. The alternate page is not indexed.

Could this explain the rankings drop?

[AMHC]

It's a big site, and the most important category page. It explains a lot.

[MichaelC-15022]

How big is your site?  Is the page being A/B tested a pretty strong page (PA, PR)?

Let's say you've got a relatively small site, with a few dozen pages, and the page in question is one of only a couple linked to from the main nav.  So it's one of the stronger pages on the site.   Blocking it in robots.txt means Google isn't going to continue to distribute link juice from that page to other pages on your site, so all of those other pages get slightly weaker.

In general, robots.txt isn't the place to block Google, as you throw away the outbound link juice from that page that's blocked.  Instead, you'd want to do a meta robots noindex,follow on the page itself.

If you've got a big site, and this particular landing page isn't a major portion of the overall site in terms of PA, then you shouldn't have seen an effect like I've described.