Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

How do i stop Moz from indexing my dev site?

Getting Started

721

Tholomew1 last edited by

Hey,

I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google.

How do i stop MOZ including it in its reports?

Cheers everyone.
Bart
1 Reply Last reply
Reply Quote 0
Alick300 last edited by

Hi,

To stop moz crawler you have to mention below text in robots.txt for dev.domain.com

User-Agent: rogerbot Disallow: /

Hope this helps.

Thanks
1 Reply Last reply
Reply Quote 2

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Unsolved What would the exact text be for robots.txt to stop Moz crawling a subdomain?

robots.txt crawl disallow subdomain sub-domain

I need Moz to stop crawling a subdomain of my site, and am just checking what the exact text should be in the file to do this. I assume it would be: User-agent: Moz
Disallow: / But just checking so I can tell the agency who will apply it, to avoid paying for their time with the incorrect text! Many thanks.
Getting Started | | Simon-Plan

0
Risk of moz on website

Does Moz collect information of customers accessing my website https://tructiepbongda.site/?
Thank you!
Getting Started | | gogoanimetp

0
What is the quickest and easiest way to run an SEO audit on a Wordpress site that at least shows all the mechanical problems?

What is the quickest and easiest way to run an SEO audit on a Wordpress site that at least shows all the mechanical problems?
Getting Started | | integratedproperty

0
How can i start in moz?

I want to know what to do first, how do I start the branding, etc. Thanks!!!
Getting Started | | Gridiron236

1
My question is, when you translate your website to another language, does moz crawl both or do i have to add another campaign to moz so that they can crawl it seperately?

Hello, i recently translated my website to spanish, keywords,meta tags, content etc. All of our URL for the translated pages start "ep", which symbolizes espanol. My question is, does moz crawl those pages along with my english pages? or do i have add another campaign for moz to crawl my spanish pages seperately?
Getting Started | | prestigeluxuryrentals.com

0
Moz not reading Visit to my website

Hello , i have conencted moz to My new ecommerce website , MOZ is not reading any visits , although i can see the visit on google analytics and i have moz connected to My analytics as well, why i cant see the visits ? need your help please 🙂
Getting Started | | moeayyad

0
Can not create new campain with my site: edunet.com.vn

I can not create new campain because it always warning my site is not a right URL. I don't understand, please tell what should I do. My site is: edunet.com.vn. (When I try to use "Grade a page for keyword" for URL: edunet.com.vn or edunet.com.vn/thong-tin-tuyen-sinh, it returns "Sorry, but that URL is inaccessible".) Thank you so much. Minh Tam.
Getting Started | | toosol

0
How does MOZ assign a keyword to a page in the On Page Grades by Keyword Ranking

I'm in the beta analytics tool. In the On Page Optimization > On Page Grades by Keyword section, there is a Keyword column with a keyword assigned to each URL on my site. http://analytics.moz.com/search/on-page/572409.326269 How is the keyword that is shown here decieed upon? Is it simply based on a guess of the "best keyword" for each page based on kwp density and other factors? Can I map a new keyword to each page? or at least to my top ten most important pages? I see I can check the grade for any keyword on any page here: http://analytics.moz.com/search/manage-page-grades/572409.326269. I want to map my top ten pages to ten keywords - can I save these changes in the system so I don't have to manually check a keyword/url match each time? Shaun
Getting Started | | nuash

0