Site with 2 domains - 1 domain SEO opimised & 1 is not. How best to handle crawlers?

[DGAU]

---

Situation:

I have a dual domain site:
Domain 1 - www.domain.com is SEO optimised with product pages and should of course be indexed.
Domain 2 - secure.domain.com is not SEO optimised and simply has checkout and payment gateway pages.

I've discovered that Moz automatically crawls Domain 2 - the secure.domain.com site and consequently picks up hundreds of errors.
I have put an end to this by adding a robots.txt to stop rogerbot and dotbot (mozs crawlers) from crawling domain 2. This fixes my errors in Moz reports however after doing more research into 'Crawler Control' I figure this might be the best option.

---

My Question:

Instead of using robots.txt to stop moz from crawing all of Domain 2 should I use on each page of domain 2?

I believe this would then allow moz and google to crawl Domain 2 but also tell them both not to index it.
My understanding is that this would be best, and might even help my overall SEO by telling google not to give any SEO value to the Domain 2 pages?

[evolvingSEO]

Hello!

I can answer this from a Google / SEO perspective (a non-moz tool perspective).

First you want to be sure the secure subdomain content is not indexed.

• If the secure subdomain is NOT indexed, leave the robotos.txt crawl blocking in place. You don't want and don't need Google crawling secure pages and payment pages. Just be sure they truly all are private pages. If they are NOT indxed, the crawl block is best - this will prevent google from crawling, and if they can't crawl they can't index.

• If the secure pages ARE indexed

• remove the robots.txt crawl block.

• Add meta noindex on all the pages

• Wait for them to be noindexed (removed from google)

• Then, block them from being crawled with robots.txt - which will prevent them from being crawled, and thus prevent them from being indexed as well.

[dave.kudera]

Hey, Dave here from the Help Team!

Jumping in to answer the technical question, you can definitely use the meta robots tag instead of a disallow directive in your robots.txt file.  I would like to point out that Meta Noindex is something we report in Site Crawl so you would see an influx in that issue category but you can mark them as "ignored" as you see fit.