I’d like to set up a Moz campaign that crawls just the primary website, not subdomains
-
Is this something you could help with? (it either bombs or crawls everything, so I assume I’m missing something in the campaign settings, or it’s just not possible.
-
Hey, thanks for reaching out to us!
You can block our crawler from crawling specific subdomains of your site by excluding our user-agent from the subdomains robots.txt file. Our user agent is "rogerbot" and the directive would look something like this:
User-agent: rogerbot
Disallow: /You will want to make sure that this is associated with the robots.txt file specific to the subdomain you want to block.
You can read more about Rogerbot in our guide.
Hope that helps, let me know if you have further questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz Free
hello guys, is there any way to use Moz free or any well-known company who would like to give me Moz login at a very cheap price. if anyone knows then please guide me shortly. Thanks With Regards, Abert sin.
Getting Started | | seochris172 -
Moz campaign for a subdomain with no home page
Hello all, I'm trying to setup a campaign in Moz for one of our subdomains that we use for marketing landing pages. Unfortunately there really is no home/landing page for the subdomain so when I enter it into the campaign settings it says the site isn't accessible. Is there anyway for me to add this? I tried entering the subdomain with a star, e.g. pages.mydomain.com but it would not accept it. Any ideas? Thanks!
Getting Started | | Brando160 -
Moz Not Pulling Monthly Data
So I set up my Moz campaigns on the 8th of this month. I can see data for the week, however, I can't see data from the previous week or for the month. So my question has two parts: a.) Is there a way to view data for the previous week, and b.) when can I expect to see data for the entire month. Seeing data for the entire month is absolutely critical for me, since this is what I'm going to use to prepare my monthly client reports. Has anyone experienced anything like this? PLEASE HELP!!!
Getting Started | | maxcarnage0 -
How to locate page with the duplicate title? (Crawl Diagnostics - Duplicate Titles Warning)
I am looking through my crawl diagnostics and one of my errors states that a page has a duplicate title. My problem is that I do not know how to find the duplicate. Any advice here?
Getting Started | | bearpaw0 -
Problem creating campaign
Hello everyone! I am a newbie here. I am trying to create a campaign, however I got stuck at the first step! It says 'We have already started building this campaign. If you entered the wrong URL and would like to track a different site, please create a new campaign." please see attached image. I thought it would be very straight forward... I have tried to create another campaign with a different website and it worked, so I don't understand what has gone wrong with this campaign. It will be much appreciated if anyone can help with this... 😞 thank you very much !! Screen%20Shot%202014-08-12%20at%2022.39.39.png
Getting Started | | sweetlime0 -
How long does it usually take Moz to populate information for a new Web site?
We recently launched (9/13/2013) an e-commerce Website and added the campaign to SEO MOZ. Week after week the Domain Rank is 1 and none of our keyword stats or link stats are populated. We have another Moz campaign that posts weekly updates and is doing extremely well. I'm just wondering how long it usually takes Moz to start populating all the analysis stats? I'm also wondering if there might be a campaign setting buried somewhere that I need to enable or maybe it just takes more than 5 weeks? Any insights would be much appreciated. Here's the new URL we need to track with MOZ: http://www.imsportshq.com
Getting Started | | Tripper0