Setting up MOZ to run on Staging
-
Hi Moz,
We would like to setup Moz to run on our Staging Server. This would be extremely valuable as it alerts us to new SEO issues/risks in a controlled and secure environment that is not exposed to production.
Our internal team has recommended potentially setting up a reverse proxy server that will validate either via Moz's/Rogerbot's http header or IP and allow Moz access to our Staging environment to crawl.
Is this something that we can setup with Moz? Are there other ideas to enable Moz to crawl our Staging server?
-
I know this is old - but I was also wondering this same thing.
Our specific use case is that we would love to be able to run a on-page grader via Moz on our staging environment before going live to make sure we are not degrading on-page seo for any specific keywords when changing things around to new designs.
-
No worries!
You can add a directive to your site's robots.txt file as such:
User-agent: rogerbot
Disallow:User-agent:*
Disallow: /This will block all bots from accessing your site aside from rogerbot.
There may be other methods as well, which I am personally not aware of. Perhaps other users on the forum may be able to provide tips on what you can potentially try out.
Let me know if you have any other questions!
Eli
-
Hi Eli,
Thanks for the fast response.
Creating a public facing 'staging' instance does create a certain degree of risk as it can potentially be discovered and indexed.
Are other clients currently using this method? If so what best practices do you recommend we follow if we were to go this route to ensure that our 'Staging' site is ONLY crawled by Rogerbot and not by any other bot or service.
-
Hey!
Thanks for reaching out to us!
In order to track a new site you would indeed need to create a new campaign. You can do that here https://analytics.moz.com/campaigns/new . Each campaign tracks the exact URL entered when it's created.
Unfortunately you would not be able to validate us by IP address alone as we do not use a static IP address or range of IP addresses, as we have designed our crawler to have a dynamic approach. This means we use thousands of dynamic IP addresses which will change each time we run a crawl. We believe that this approach gives us the best dynamic view of the web!
However, one alternative I'd be able to suggest would be to identify our crawler by User-agent: rogerbot. You can read more about rogerbot in our guide. It's also worth pointing out that your site would need to be publicly available for our crawler to reach and crawl it.
I hope this helps, please do let me know if you would like more clarification or have any other queries.
You're welcome to reach out to help@moz.com if it's easier!
Eli
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does Moz takes to update DA and PA value?
I used to work daily on my office website https://quintdaily.com/ and follow link building strategy. however, since more than 40-50 days, there is no movement in my DA, as the PA has just lifted up. However, there is no changes in Domain Authority so far. Besides, other than this website, im working on location wise SEO projects, but, i can find those websites Domain Authority changed in less 20 days. Is there is time period in changing the Domain Authority once it touched DA 30+?
Moz Pro | | seochris172 -
Spam score by moz
como puedo estar seguro de que los enlaces toxicos que informa la herramienta moz spam score, son realmente malos para mi seo? Algunos de ellos parecen totalmente buenos! gracias!! as I can be sure that the toxic links moz tool reports the spam score, they are actually bad for my SEO? Some of them seem totally good!
Moz Pro | | Sbm20080 -
Rank tracking in Excel via MOZ API?
Hi there, i hope someone knows the solution to this problem. I created/ copied the "advanced Excel Keyword Analysis Report" that Dan Peskin was kind enough to share, you can find it here: http://moz.com/blog/how-to-build-an-advanced-keyword-analysis-report-in-excel. Now i m looking for a solution to rank my Keywords by using an API, preferably the MOZ API. My keywords are alligned in a column "Keword List", which will be regularly updated. Next to this column i would like to have a column with the corresponding rank. Is there a automatic solution, or do i have to do it manually?
Moz Pro | | Juri.Rabiner0 -
How to handle subdirectories in Moz and Webmaster Tools?
I have 3 websites: http://www.abc.com (English pages) http://www.abc.com/fr (French) http://www.abc.com/de (German) Webmaster Tools In Google Analytics: I created an Account called ABC and created 4 properties with relative filters ABC Default (No filter) ABC English (filter to exclude /fr, /de) ABC French (filter to include /fr) ABC German (filter to include /de) In webmaster tools, I wish to add 3 sites http://www.abc.com http://www.abc.com/fr http://www.abc.com/de How do I validate each url with Google analytics, considering I just want the specific language for folder. How do I validate the domain name http://www.abc.com to always ensure the inclusion of the default english data? MOZ Do I need to create 3 campaigns (one per language) and if yes, again how do I handle http://www.abc.com which should exclude /fr and /de? Thank you
Moz Pro | | seo12120 -
Confused on setting up my domain in a campaign
I have a website that is www.example.com. When I first setup my campaign in SEOMoz, I missed the instructions that stated I need to set up my domain as it appeared in the browser. Thus, I set it up as example.com. When I realized my mistake, I setup a second campaign as www.example.com and am running them side by side to see the differences. My question has to do with the best setup. I have a blog that is a subdomain as blog.example.com. It appears that the www.example.com campaign is not crawling the blog pages but example.com is. Are there any downsides in setting up this campaign as example.com? I have a number of other sites with the same issue so would like to get this resolved before I setup the additional sites. Thanks for any assistance or insights into ways I may be "shooting myself in the foot" with the wrong domain structure in my campaign settings.
Moz Pro | | rfwood0 -
What's with the new Q&A set up?
In the old Q&A set up your response was automatically given a thumbs up by you so that you got a mozpoint for every response you gave. Now you have to manually thumb up your responses. Why was this change made? Can we get it back to the way it was?
Moz Pro | | DanDeceuster1