Setting up MOZ to run on Staging
-
Hi Moz,
We would like to setup Moz to run on our Staging Server. This would be extremely valuable as it alerts us to new SEO issues/risks in a controlled and secure environment that is not exposed to production.
Our internal team has recommended potentially setting up a reverse proxy server that will validate either via Moz's/Rogerbot's http header or IP and allow Moz access to our Staging environment to crawl.
Is this something that we can setup with Moz? Are there other ideas to enable Moz to crawl our Staging server?
-
I know this is old - but I was also wondering this same thing.
Our specific use case is that we would love to be able to run a on-page grader via Moz on our staging environment before going live to make sure we are not degrading on-page seo for any specific keywords when changing things around to new designs.
-
No worries!
You can add a directive to your site's robots.txt file as such:
User-agent: rogerbot
Disallow:User-agent:*
Disallow: /This will block all bots from accessing your site aside from rogerbot.
There may be other methods as well, which I am personally not aware of. Perhaps other users on the forum may be able to provide tips on what you can potentially try out.
Let me know if you have any other questions!
Eli
-
Hi Eli,
Thanks for the fast response.
Creating a public facing 'staging' instance does create a certain degree of risk as it can potentially be discovered and indexed.
Are other clients currently using this method? If so what best practices do you recommend we follow if we were to go this route to ensure that our 'Staging' site is ONLY crawled by Rogerbot and not by any other bot or service.
-
Hey!
Thanks for reaching out to us!
In order to track a new site you would indeed need to create a new campaign. You can do that here https://analytics.moz.com/campaigns/new . Each campaign tracks the exact URL entered when it's created.
Unfortunately you would not be able to validate us by IP address alone as we do not use a static IP address or range of IP addresses, as we have designed our crawler to have a dynamic approach. This means we use thousands of dynamic IP addresses which will change each time we run a crawl. We believe that this approach gives us the best dynamic view of the web!
However, one alternative I'd be able to suggest would be to identify our crawler by User-agent: rogerbot. You can read more about rogerbot in our guide. It's also worth pointing out that your site would need to be publicly available for our crawler to reach and crawl it.
I hope this helps, please do let me know if you would like more clarification or have any other queries.
You're welcome to reach out to help@moz.com if it's easier!
Eli
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz crawler not working
Hi Moz crawler keep failing on my site with the error showing : Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. I'm not sure what am I missing out.. this is my robots.txt.. i don't think Im missing anything else.. https://www.wearefutureheads.com/robots.txt can the support team help ?
Moz Pro | | teikh0 -
Moz Domain Authority 27
Hi there, I have been reading up about Domain Authority being important to rank though I have a question about it. When trying a few google searches and using a tool to see the Moz Domain Authority i notice the following. I won't mention sites or keywords, happy to send them via private message if required. A site ranks 1st position for most keywords including the main keyword that related to their industry, a highly competitive keyword. Their Moz Domain Authority 27, Est. Visits: 265,000, Pinterest shares 72, Facebook shares 191 and unique domains that point to it are 55. Yet when I look at another site, optimising for the same keyword, Moz Domain Authority 50, Est. Visits: 60,000, Pinterest shares 35000, Facebook shares 20000 and unique domains that point to it are 521. I would have thought that the second site ranks 1st for the keyword and the first one mentioned which is in 1st position I would have expected to be on page two or three. as I scroll down the first page for the term, all sites except for one have higher DA, visits, shares and so on. If a high PA is required can someone help me to understand the above example? Whn a low PA site ranks much better for a primary keyword than all other sites that have a higher PA than it?
Moz Pro | | IvanaDaulay1 -
Webmaster Tools shows mystery errors that Moz does not
One of my campaigns is doing great in the sense that the website has been running fault free for a few months now. Great, of course! But... in Google Webmaster Tools errors keep coming in showing older media documents and pages. And it does not say where they are from. Probably this is more a Google question, but I thought I'd try to find some answers here first. I would appreciate any suggestions and help. Monique
Moz Pro | | MarketingEnergy0 -
Is there a tool on Moz or out on the internet that does bad link checker
I'm still pretty new to this and I was wondering if there is a free software, one of Moz or free out on the the internet that allows you to check bad links. I've done a lot of link building with citations and directories that for my clients industry. I just don't want to add their website and profile to a bad/risky directory and it penalizes my clients. I've seen a few out there, but I need one that is respectable and reliable. Any suggestions? I found one called bad neighborhood http://www.bad-neighborhood.com/text-link-tool.htm. Thanks Again, Benny
Moz Pro | | ACann0 -
Is the Moz Ranking Report correct?
Hello here, today I got my weekly report from Moz by email and it reported a strong decline of rankings on some of our major keywords. Then I went to check on Google personally, and instead I found my pages ranked much higher than reported by Moz (please note that my results wasn't personalized...). I also use a desktop program to check my rankings, and what I found personally on Google corresponded to what my desktop program reported me. Here is an example: my website is virtualsheetmusic.com, and Moz reported a rank NOT in top 50 for the following keyword: "moonlight sonata sheet music" Whereas we are actually on the 19th spot. Another example for the keyword "czardas violin sheet music", Moz reports NOT in top 50, whereas we are actually at the 5th spot on the first page of Google results! Why's this? Anyone can explain? Is there anything wrong with Moz ranking report?
Moz Pro | | fablau0 -
Who else has inaccurate reporting? SEO Moz BUG
I have been in contact with support to little avail no timelines for the correction kind of leaves me out in the cold. I have loved SEOmoz in the past but this recent bug is costing me time and money. Can someone please tell me how they are working around all these issues in rank reporting and a zero accuracy rate on page report cards? bugs are holding up my SEO efforts. SEOmoz has been in touch but has provided no timelines. My programs are suffering. I found inaccuracy in each report metric and now have very little faith that this is going to be corrected in a timely fashion. I rely greatly on the wonderful tools SEOmoz provides and now I am flying blind. Who else has been hit by the bug? And, what has support done for you to help?
Moz Pro | | ericajane0 -
Best SEO Tool Set ! What Are they ?
Hey Moz Family. I wanted to get everyones opinion on some Seo Tools Sets that they have used and have had success with or find essential to Seo Campaigns. I have been playing around with a lot of tool sets. Just need some honest input on some tools you guys use, or a combination of tools. Best of luck, Hampig M Ceo BizDetox
Moz Pro | | BizDetox0 -
About the rankings report in the Pro Dashboard, does it track the ranking of every page on a root domain, or just the home page or whichever page you set up the campaign with?
I noticed that one of the pages on my root domain has a #5 rank for a keyword, yet the ranking report says that there are no results in the top 50. So I am assuming it is only tracking the home page. That is one thing I liked about the Rank Tracker, that it would find any page that was ranking on a root domain. Thanks, Lara
Moz Pro | | larahill0