Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

How do i stop Moz from indexing my dev site?

Getting Started

721

Tholomew1 last edited by

Hey,

I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google.

How do i stop MOZ including it in its reports?

Cheers everyone.
Bart
1 Reply Last reply
Reply Quote 0
Alick300 last edited by

Hi,

To stop moz crawler you have to mention below text in robots.txt for dev.domain.com

User-Agent: rogerbot Disallow: /

Hope this helps.

Thanks
1 Reply Last reply
Reply Quote 2

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Did Moz have a citation report ?

Did Moz have a citation report ?
Getting Started | | Vbleuwire

0
Moz campaign for a subdomain with no home page

Hello all, I'm trying to setup a campaign in Moz for one of our subdomains that we use for marketing landing pages. Unfortunately there really is no home/landing page for the subdomain so when I enter it into the campaign settings it says the site isn't accessible. Is there anyway for me to add this? I tried entering the subdomain with a star, e.g. pages.mydomain.com but it would not accept it. Any ideas? Thanks!
Getting Started | | Brando16

0
Can I use 2 Moz Campaigns to Track more than 3 Competitors?

I would like to track 6 competitors for a new client, maybe short term, maybe long term. Can I set up 2 Moz Campaigns for the same client using the same keywords only with different competitors?
Getting Started | | cindyt-17038

0
How to generate a a toxic link report for a specified site?

Hello Everyone, I am new to this I need some help. Thanks ahead
Getting Started | | kremsoft

0
My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?

My website does not permit all crawler to crawl website. Whether ii need to give permission to moz bot to crawl website or not? If yes what is the moz bot name?
Getting Started | | irteam

0
How do get Moz to spider a Development site PRE LAUNCH?

Hi, Does anyone know how we could get Moz to browse a development site before launch? But without Google and other engines indexing it? Thanks
Getting Started | | bjs2010

0
How to remove campaign from Moz.com

I've accidentally created a duplicate campaign, which I am unable to delete, please help
Getting Started | | shopperlocal_DM

0
Is the Moz Rank Tracker Broken?

Hi, I've waited a few days for data to appear in the account but all of the keywords show 'not in top 50'. When I know this isn't the case since I can see half of them myself either on page one or two (yes I have cleared cache / history and have tried an incognito window to see the results for myself; I have even tried another PC. Also I'm really not keen on the new layout of the pro account, everything is so much smaller event the text. Personally I think it doesn't look half as clean and simple as it used to. Another thing, the link analysis tab isn't showing all of my links - it's only showing 2! Which is way off, even when I check with Open Site Explorer there are a lot more than that. Thanks Ricky
Getting Started | | Thirsty-Media

0