Crawling issue
-
Hi,
I have to set up a campaign for a webshop. This webshop is a subdomain itself.
First question: The two subfolders I need to track are /nl_BE and /fr_BE. What is the best way to handle this? Shall I set up two different campaigns for each subfolder, or shall I just make one campaign and add tags to keywords?
**Second question: **it seems like Moz can't crawl enough pages. There are no disallows in the robots.txt. Should I try putting the following at the top into my robots.txt?
User-agent: rogerbot
Disallow:Or is it because I want to crawl only a subdomain that it doesn't work?
Thanks
-
Hey,
Thanks for reaching out to us!
You can create a Campaign solely for a subdomain or subfolder by selecting the +Advanced setting in the Campaign set-up; just click the check box there and it will limit our Campaign audit to the pages on that specific subdomain or subfolder. From there, you can see in your Campaign Setting if you've set it up for just that chunk of your site, or for the entire root domain.
I've got a guide to this process that I think may help.
With regard to your second question, feel free to reach out to help@moz.com so that we can take a closer look
Looking forward to hearing from you,
Eli
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz DA Issue
Hey Community, This is my site which is the Best Drifitng in Australia. I am getting a problem and my site's DA is unable to show using. I want to see it as soon as possible, although I am using a premium tool. Thanks
Getting Started | | jndjue40 -
I’d like to set up a Moz campaign that crawls just the primary website, not subdomains
Is this something you could help with? (it either bombs or crawls everything, so I assume I’m missing something in the campaign settings, or it’s just not possible.
Getting Started | | blueprintatl0 -
We recently switched from HTTP to HTTPS and we are having crawling issues!
We switched our website from HTTP to HTTPS and we started to get an email from Moz about the robots.txt being unable to crawl our website. The website is hosted through wordpress but we haven't had any issues until we switched. We have no idea what to do or even what the problem is! If you have had a similar problem and fixed it, we need your help! Thank you.
Getting Started | | DrInfinity0 -
Crawling Website
How often does Moz crawl the website? I have a lot of high priority crawl issues but my web development team says that it fixed them. However, Moz still shows me that none of the errors are fixed.
Getting Started | | Leoni0 -
Crawl Diagnostics Help
Hi there Where can i find my campaigns crawl diagnostics? I need to find where this information can be found and specific issues? Is this possible, i cant seem to find this info. regards Ana
Getting Started | | Starsia200000 -
Campaign.crawl-seed.bad-response
I am trying to set up a new campaign for a website, but I keep getting this error message... campaign.crawl-seed.bad-response 😞 I have no idea what the problem is. Can you tell me what I am suppose to do to fix this? The URL I am trying to set up is www.aboutplcs.com
Getting Started | | ChadC0 -
Adding a campaign freezes in first step, any tips on how to resolve this issue?
I just subscribed to the 30 days trial and wanted to set up my first campaign, but it just doesn't work. I name the campaign, enter the URL and click Next >>. The page starts processing but even after 5 minutes waiting time, nothing happens. I just don't get through to the 2nd step. I already tried setting it up in Firefox, Chrome and Opera. It just won't work. 😕
Getting Started | | WindSolutions0 -
MOZ Starter Crawl not happeneing
Hi I added a new site 48hours + ago and the starter crawler has not even begun collecting data. Any help would be appreciated. cheers Isaac
Getting Started | | sodafizz0