How do i stop Moz from indexing my dev site?
-
Hey,
I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google.
How do i stop MOZ including it in its reports?
Cheers everyone.
Bart -
Hi,
To stop moz crawler you have to mention below text in robots.txt for dev.domain.com
User-Agent: rogerbot Disallow: /
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Our crawler was not able to access the robots.txt file on your site
I've submitted my website to be crawled by Moz and done everything I can according to the troubleshooting guides. Please help! https://digitalbutter.co.za/robots.txt
Getting Started | | DigitalButter0 -
Can Moz Monitor a JS Site?
We are building a new site that, on the blog landing page, uses JS to populate the individual blog article links on the page. The links are not viewable in the page source, but do appear once the page fully renders. After running an on-demand crawl of the site (in QA...not indexed yet), it appears that Moz isn't indexing these pages, and it also isn't reading other page elements that load later (like an H1 that is rendered in JS but not in page source). Are we going to be able to use Moz to track this site? Is there some setting to help?
Getting Started | | Rodrigo-DC0 -
Has anyone purchased the MOZ SEO courses? Are they good?
I'm looking to learn more about SEO. Has anyone purchased the courses available through MOZ? If so, are they useful?
Getting Started | | Stevepair0 -
Why does the moz bar link count differ from the moz link analysis page
Hi all, Why does the Moz Bar show a different link count from the Moz link analysis page? For example, when I check the SERP below, for the first result, the bar shows 936 page links from 4 RDs. But when I check out the link analysis page, it tells me there are just 141 page links from 4 RDs. What gives? For the second entry, the bar shows 6 page links from 0 RDs. Not sure how that's possible. Can anyone explain these things. Thanks! Andy Reviewed SERP: https://www.google.com/search?source=hp&ei=hhRVW5yyH5C60PEP-_-isAw&q=mountain+bike+trails+near+me&oq=mountain+bike+trails+near+me&gs_l=psy-ab.3..0l7j0i22i30k1l3.645.4654.0.5322.28.19.0.8.8.0.243.2920.3j11j5.19.0....0...1c.1.64.psy-ab..2.26.2961...0i131k1.0.C4lAxLkGgH0
Getting Started | | AndyKubrin0 -
Moz Pro Warning: Redirect Chain
I have just signed up to a Moz Pro account, and after it finished crawling my website it gave me a warning about a redirection chain. http://elementpaints.com >> https://elementpIaints.com >> https://www.elementpaints.com I'm trying to find some more information about how to fix this problem but I'm not having much luck. This article I found even says it is not a problem: https://really-simple-ssl.com/knowledge-base/avoid-landing-page-redirects So now I'm even more confused, do I still need to fix this? If so, how do I do that? FYI: I have Lets Encrypt SSL cert installed on my server, and I'm using Cloudflare with Full SSL option and HSTS enabled, and "Always Use HTTPS" option is turned on. | http://elementpaints.com |
Getting Started | | elementpaints0 -
Moz not reading Visit to my website
Hello , i have conencted moz to My new ecommerce website , MOZ is not reading any visits , although i can see the visit on google analytics and i have moz connected to My analytics as well, why i cant see the visits ? need your help please 🙂
Getting Started | | moeayyad0 -
Moz Analytics - Custom Report Question
Hello Moz Forums! I'm having problems building custom reports for my clients. So, i've managed to put together a few reports and I've set these to be scheduled for weekly automated emailing! Now, here's the problem, before sending it to my clients, I was testing this by sending it to myself. This is what I receive: http://www.freelance-seo.co.uk/images/Moz.png I don't really want my clients to see 'By Moz!' everywhere when I send this out, is there a way to change this automated mail? Or a way to just just send the PDF without a link? Alternatively, have I just failed in using this product? Edit: Just to clarify! The 'simply change your report settings' option, just takes me to an option to edit the report title, not the email in any way.
Getting Started | | Paul_Tovey0 -
Trending Bugs in Moz Analytics
The FAQ’s We hope we can help! Below are the trending issues at the moment. If you don’t see your question addressed or need further help, send us a message at http://moz.com/help/contact. Our Help Team gumshoes will investigate your issue and respond shortly. Where did my Campaigns Go?
Getting Started | | Abe_Schmidt
Whenever your account is suspended, your campaigns are archived. The good news is that reviving archived accounts is really simple. Go to your Campaigns section: http://pro.moz.com/campaigns
At the top of the page there is a "Archived Campaigns" Tab, give it a click.
From here, you should easily be able to "activate" your campaigns. My Google Analytics profile won’t stay connected to my campaign and/or I am missing profiles in my GA Settings. Google goes through a process called Oauth when it comes to authorizing access of services that are linked to your Google account. Under any single Google account, there are about 20 tokens account wide. Those tokens are used to provide access for a variety of services from Gmail to apps on your phone. Once you hit the limit, which is 20 for most users, the system automatically revokes the oldest token to provide a new one. I'm not certain if this is causing the issue on your account, but it is a great place to start troubleshooting. You may be able to correct the issue by manually revoking your tokens to make room for new ones on your account and then reconnecting the account to Moz. Here are the instructions on how to do that: Follow this link https://accounts.google.com/b/0/IssuedAuthSubTokens. This page displays the current OAuth tokens you are using. Once you reach the page, simply press the revoke button (illustrated on this screenshot: http://screencast.com/t/vjh3KrjRRIe) for services that you are not using right now. Once you are done with, that simply go back to your campaign settings, disconnect your GA profile and reconnect. The below process may also fix this issue for you. Head to your campaign settings page on your overview page. Disconnect your Google Analytics connection. Go back to your settings page and click on "connect account." Please make sure you log into the correct GA account. 🙂 Hurray! This should let you grab the most current traffic data! Oops! try refreshing page! I can’t access any of my Campaign data.
This issue is normally machine specific and can be a bit complex. An individual forum has been created to address this: https://seomoz.zendesk.com/entries/28203486-Oops-Try-Refreshing-the-Page-Error-showing-on-all-pages- Unable to retrieve historical ranking CSVs. Some users did not receive their requested Historical Rankings CSVs. This has mostly been affected by a change or update to your competitors after a campaign has been setup for a while. Please send your request to retrieve this data with http://moz.com/help/contact.8