How do i stop Moz from indexing my dev site?
-
Hey,
I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google.
How do i stop MOZ including it in its reports?
Cheers everyone.
Bart -
Hi,
To stop moz crawler you have to mention below text in robots.txt for dev.domain.com
User-Agent: rogerbot Disallow: /
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz campaign for a subdomain with no home page
Hello all, I'm trying to setup a campaign in Moz for one of our subdomains that we use for marketing landing pages. Unfortunately there really is no home/landing page for the subdomain so when I enter it into the campaign settings it says the site isn't accessible. Is there anyway for me to add this? I tried entering the subdomain with a star, e.g. pages.mydomain.com but it would not accept it. Any ideas? Thanks!
Getting Started | | Brando160 -
Can I use 2 Moz Campaigns to Track more than 3 Competitors?
I would like to track 6 competitors for a new client, maybe short term, maybe long term. Can I set up 2 Moz Campaigns for the same client using the same keywords only with different competitors?
Getting Started | | cindyt-170380 -
My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?
My website does not permit all crawler to crawl website. Whether ii need to give permission to moz bot to crawl website or not? If yes what is the moz bot name?
Getting Started | | irteam0 -
How do get Moz to spider a Development site PRE LAUNCH?
Hi, Does anyone know how we could get Moz to browse a development site before launch? But without Google and other engines indexing it? Thanks
Getting Started | | bjs20100 -
How to remove campaign from Moz.com
I've accidentally created a duplicate campaign, which I am unable to delete, please help
Getting Started | | shopperlocal_DM0 -
Is the Moz Rank Tracker Broken?
Hi, I've waited a few days for data to appear in the account but all of the keywords show 'not in top 50'. When I know this isn't the case since I can see half of them myself either on page one or two (yes I have cleared cache / history and have tried an incognito window to see the results for myself; I have even tried another PC. Also I'm really not keen on the new layout of the pro account, everything is so much smaller event the text. Personally I think it doesn't look half as clean and simple as it used to. Another thing, the link analysis tab isn't showing all of my links - it's only showing 2! Which is way off, even when I check with Open Site Explorer there are a lot more than that. Thanks Ricky
Getting Started | | Thirsty-Media0
