I’d like to set up a Moz campaign that crawls just the primary website, not subdomains
-
Is this something you could help with? (it either bombs or crawls everything, so I assume I’m missing something in the campaign settings, or it’s just not possible.
-
Hey, thanks for reaching out to us!
You can block our crawler from crawling specific subdomains of your site by excluding our user-agent from the subdomains robots.txt file. Our user agent is "rogerbot" and the directive would look something like this:
User-agent: rogerbot
Disallow: /You will want to make sure that this is associated with the robots.txt file specific to the subdomain you want to block.
You can read more about Rogerbot in our guide.
Hope that helps, let me know if you have further questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Website DA Decreased
I Have A website Related to Agras T16 Drone Today DA of my website decreased what could be the reason Please help me what about issue?
Getting Started | | JerryAlexo0 -
MOZ flagging urls as longer than 75 characters when they aren't
As an example: https://www.abthermal.com/3m-70-70hdt-self-fusing-silicone-rubber-electrical-tape.html Moz has flagged this as URL too long... but guideline is that happens at 75 characters or more? I count 55..?? or does the 75 limit have to include the domain? Thanks, Adrian
Getting Started | | abtechgroup0 -
Keyword ranking in google but not in MOZ
Good day, When I type my keyword in google, the website related to this keyword appears on page 1 (7th link from the top). However in Moz report, the keyword position is #13. I don't understand why. Can someone explain the reason? Thank you,
Getting Started | | Wolfpacksweet0 -
What day of the week should I set my campaigns up on?
I am new to doing SEO, and am not an SEO professional. I'm just taking a DIY approach to learning about and doing my own SEO for some non-profit work and using Moz is my first foray into it. I have a question about which day of the week is best to run my campaigns. I have 5 campaigns running, but I set them up rather haphazardly. So one is running on Saturdays, and the remaining 4 run on Mondays. I am running these campaigns on sites that are not mine. I'm not the webmaster. This is a non-profit, sort of a research effort that seeks to investigate other non-profit websites that all advocate on a particular topic. I'd like to be able to compare the 5 campaigns (e.g., their keyword rankings especially). So I think I should get them all running on the same day. Setting aside that Moz isn't able to let me switch which day they run on, I'm needing advice on how to think about which day of the week I should choose. I haven't been running them for very long and I'm sill a newbie so I don't think it's a huge deal that I'll need to re-setup at least one of the campaigns to get them all on the same day. I've been searching Moz Q & A and google for things like "best day of the week to track SERPs" and "how often to track SERPs" but I'm not getting anything that's helping me think about what's important for choosing a day of the week. I am getting results for other rank trackers out there that let you do more frequent ranking but I'm not quite ready to branch out to other tools, so I really more need some guidance on how to think about which day of the week to choose here in Moz. Blog posts, tutorials, links to related questions, etc on this topic would be awesome, plus your general thoughts and discussion on it would be great, too.
Getting Started | | scienceisrad1 -
Why can I only create 1 campaign
I created a new account and my Moz plan says, that I can have 5 campaigns. But it does not let create me another campaign in addition to my first one. What to do???
Getting Started | | PascalKlein0 -
Why does Moz Analytics only let you look at last month / last week?
I cant think of any other reporting product that only lets you look at the last period, am I missing a setting somewhere. I need to take a look at Feb!
Getting Started | | locusm1 -
When does Moz update campaign data with new timeframe ?
Hi, I joined Moz community on last week and discovered an amazing tool. I set up my first campaign and I am getting information for a timeframe that's going from 7 to 13 of March. I believe that new data for 14 to 20 of March should be available but I didn't get any update. So, when is Moz supposed to update campaign data with fresh information ? Thanks in advance for such a dumb question. Sébastien
Getting Started | | lecercledesgourmets0 -
Quickest Way to Pro Campaigns after Log In?
For some reason every time I sign into Moz there is no sign of a button or menu for my pro campaigns. I have to click on the Moz Analytics button at the top right of the screen. As of today I now have to click a campaign within analytics before I see any sign of a pro campaign button. Can anyone tell me the quickest and easiest way to find my pro campaigns once I've signed in? I also have the same problem if I wonder off to blogs, Q&A, Products and Training. Feeling increasingly frustrated and starting to avoid using Moz.
Getting Started | | ChristinaRadisic0