Crawling issue
-
Hi,
I have to set up a campaign for a webshop. This webshop is a subdomain itself.
First question: The two subfolders I need to track are /nl_BE and /fr_BE. What is the best way to handle this? Shall I set up two different campaigns for each subfolder, or shall I just make one campaign and add tags to keywords?
**Second question: **it seems like Moz can't crawl enough pages. There are no disallows in the robots.txt. Should I try putting the following at the top into my robots.txt?
User-agent: rogerbot
Disallow:Or is it because I want to crawl only a subdomain that it doesn't work?
Thanks
-
Hey,
Thanks for reaching out to us!
You can create a Campaign solely for a subdomain or subfolder by selecting the +Advanced setting in the Campaign set-up; just click the check box there and it will limit our Campaign audit to the pages on that specific subdomain or subfolder. From there, you can see in your Campaign Setting if you've set it up for just that chunk of your site, or for the entire root domain.
I've got a guide to this process that I think may help.
With regard to your second question, feel free to reach out to help@moz.com so that we can take a closer look
Looking forward to hearing from you,
Eli
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved What would the exact text be for robots.txt to stop Moz crawling a subdomain?
I need Moz to stop crawling a subdomain of my site, and am just checking what the exact text should be in the file to do this. I assume it would be: User-agent: Moz
Getting Started | | Simon-Plan
Disallow: / But just checking so I can tell the agency who will apply it, to avoid paying for their time with the incorrect text! Many thanks.0 -
DA PA issue
Hi, I don't understand why does the DA of my site increasing but not the PA, ay reason guys?
Getting Started | | Seotools530 -
I’d like to set up a Moz campaign that crawls just the primary website, not subdomains
Is this something you could help with? (it either bombs or crawls everything, so I assume I’m missing something in the campaign settings, or it’s just not possible.
Getting Started | | blueprintatl0 -
Does MOZ pick up every issue in one crawl?
Hi, Does MOZ pick up every error/warning in one crawl? Or does it take numerous crawls? Many thanks Lee
Getting Started | | lbagley0 -
Crawl Diagnostics Help
Hi there Where can i find my campaigns crawl diagnostics? I need to find where this information can be found and specific issues? Is this possible, i cant seem to find this info. regards Ana
Getting Started | | Starsia200000 -
Creating a Campaign Issue: Not Valid URL (URL Doesn't Resolve?)
I'm unable to start a campaign. The MOZ Analytics page is telling me the URL is not Valid. the site is www.veinguys.com I'm not sure what the issue is - the site is live. Please advise.
Getting Started | | ignaciolorenzo0 -
Daily crawl reports, are they wasting my time?
I am relatively new here, I have 5 campaigns. I get new crawl complete reports almost every day for all of them. Wow great, except when I check the reports nothing has changed. Even if I have gone in and changed things or fixed errors, the same ones are still there and takes 4-7 days for that work to show up. Everytime I get one of these reports I am opening them up going through and not seeing the changes I implemented the previous days before. I'll spend 20-30 minutes going over these and checking details. So the question is, Are these reports wasting my time? Are they actually new reports or am I just getting spammed repeat notices everyday?
Getting Started | | RandyFriesen0