Setting up MOZ to run on Staging
-
Hi Moz,
We would like to setup Moz to run on our Staging Server. This would be extremely valuable as it alerts us to new SEO issues/risks in a controlled and secure environment that is not exposed to production.
Our internal team has recommended potentially setting up a reverse proxy server that will validate either via Moz's/Rogerbot's http header or IP and allow Moz access to our Staging environment to crawl.
Is this something that we can setup with Moz? Are there other ideas to enable Moz to crawl our Staging server?
-
I know this is old - but I was also wondering this same thing.
Our specific use case is that we would love to be able to run a on-page grader via Moz on our staging environment before going live to make sure we are not degrading on-page seo for any specific keywords when changing things around to new designs.
-
No worries!
You can add a directive to your site's robots.txt file as such:
User-agent: rogerbot
Disallow:User-agent:*
Disallow: /This will block all bots from accessing your site aside from rogerbot.
There may be other methods as well, which I am personally not aware of. Perhaps other users on the forum may be able to provide tips on what you can potentially try out.
Let me know if you have any other questions!
Eli
-
Hi Eli,
Thanks for the fast response.
Creating a public facing 'staging' instance does create a certain degree of risk as it can potentially be discovered and indexed.
Are other clients currently using this method? If so what best practices do you recommend we follow if we were to go this route to ensure that our 'Staging' site is ONLY crawled by Rogerbot and not by any other bot or service.
-
Hey!
Thanks for reaching out to us!
In order to track a new site you would indeed need to create a new campaign. You can do that here https://analytics.moz.com/campaigns/new . Each campaign tracks the exact URL entered when it's created.
Unfortunately you would not be able to validate us by IP address alone as we do not use a static IP address or range of IP addresses, as we have designed our crawler to have a dynamic approach. This means we use thousands of dynamic IP addresses which will change each time we run a crawl. We believe that this approach gives us the best dynamic view of the web!
However, one alternative I'd be able to suggest would be to identify our crawler by User-agent: rogerbot. You can read more about rogerbot in our guide. It's also worth pointing out that your site would need to be publicly available for our crawler to reach and crawl it.
I hope this helps, please do let me know if you would like more clarification or have any other queries.
You're welcome to reach out to help@moz.com if it's easier!
Eli
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does increase in DA increase Moz Trust score?
I have checked many website which in my office. Have a confusion Does increase in DA increase Moz Trust score? I have a website https://emocrest.com/ which is new one and Moz Trust score as of now is 1 only. Well, most of the projects I'm handling in our office having 2,3,4 in the range. How this MT value is calculated and does it got any relation with the change in Domain Authoiry or the page authority of a website?
Moz Pro | | nazfazy1 -
I'm New To Moz What To Focus On First
Hello, Recently signed up to MOZ for the sites we operate in the UK. I wondered what folks would recommend I focus on first when starting to use MOZ for the first time for sites SEO? Cheers Stuart
Moz Pro | | Urban331 -
Is there a tool on Moz or out on the internet that does bad link checker
I'm still pretty new to this and I was wondering if there is a free software, one of Moz or free out on the the internet that allows you to check bad links. I've done a lot of link building with citations and directories that for my clients industry. I just don't want to add their website and profile to a bad/risky directory and it penalizes my clients. I've seen a few out there, but I need one that is respectable and reliable. Any suggestions? I found one called bad neighborhood http://www.bad-neighborhood.com/text-link-tool.htm. Thanks Again, Benny
Moz Pro | | ACann0 -
How long for authority to transfer form an old page to a new page via a 301 redirect? (& Moz PA score update?)
Hi How long aproximately does G take to pass authority via a 301 from an old page to its new replacement page ? Does Moz Page Authority reflect this in its score once G has passed it ? All Best
Moz Pro | | Dan-Lawrence
Dan3 -
Best Way to Seemlessly Use Moz and Google Analytics
Hi All, I have Moz Pro and I am beginning to explore Moz Analytics. I currently use Google Analytics for pretty much everything and I am almost at expert level at using the GA interface so I'm ready to learn a new tool and have a second source of data! My question is: what's the best way to get the most of Moz Analytics and GA congruently? Off hand I know that Moz's keyword, rank, and link data, including competitive insights, is more comprehensive than GA. But how else can I use Moz and get the most out of the Moz Analytics? Looking to really dive in and discuss how we can make use of both! Thanks - Vee
Moz Pro | | Symmetri0 -
Please set somethings straight
as a recent majesticSEO subscriber, I feel like i have either been mislead or something isn't right. For some reason I was thinking it was best to have a single link from a single domain, and that by have too many links from a single domain (backlinks) the ratio between referring domains and backlinks was a factor in pagerank. In domain analysis here, it would seem that is not that case as it puts a nice big green check mark by the person with most links, whether they are from 1 domain or multiple domains. Is this a good thing? Here's an easy example. I am a website designer, and I have the ability to link to my website in the footer of every website I design either by image or by text. If I do that, obviously I would have a link from every page on the site, but only from one root domain. Is this a good thing or a bad thing. My work around is to have a link in the footer that looks like this "website design by homecutcreative.com" that links to an internal page that has one link to my site, therefore keeping my referring domain : backlink ratio to a 1 which I was under the assumption was the best possible situation. Thanks for your help.
Moz Pro | | jonnyholt0 -
Drop in number of Pages crawled by Moz crawler
What would cause a sudden drop in the number of pages crawled/accessed by the Moz crawler? The site has about 600 pages of content. We have multiple campaigns set up in our Pro account to track different keyword campaigns- but all for the same domain. Some show 600+ pages accessed, while others only access 7 pages for the same domain. What could be causing these issues?
Moz Pro | | AllaO0 -
How often does seo moz update its link data?
I have lots of recently gained good links that don't show in seo moz, how long does it take for moz trust, moz rank and PR to update?
Moz Pro | | myloseo0