Massive increase in Moz crawl.
-
I have a subdomain which has just started to be crawled by Moz, Previously this wasn't the case.
The sub-domain had 16,000+ issues. Why has Moz started to count sub-domains as part of the main domain, has Google started to do this aswell?
-
Hi Dave!
Thanks for reaching out! My name is Anne, and I helping the Mozzer Alliance today! There could be a few different reasons we've just started crawling this subdomain:
1. Did you recently link this subdomain to your site differently? Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. This could explain why we just started crawling the subdomain.
2. Did you change your robots.txt for this subdomain? Our crawler, Rogerbot, follows robots.txt. If you edited this site recently, this could also explain why we just started crawling your subdomain.
I hope this helps! If you have more questions, please reach out to the help team at https://moz.com/help/contact. From there we can help you troubleshoot what's going on, but keep your campaign details off the interwebz.
Have a great weekend!
Anne
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
MOZ identifies duplicate titles - one has' www' in the title
MOZ has identified duplicate titles - one has' www' in the title. - we have a few pieces of content where the same thing is happening. Not sure how this has happened. Should we do something about this? Will it cause problems for ranking? | KETAMINE GUIDE FOR DRUG WORKERS - free | Harm reduction informationhttp://substance.org.uk/harm-reduction-information/ketamine-guide-for-drug-workers-free | 13 | 2 |
On-Page Optimization | | Substance-create
| KETAMINE GUIDE FOR DRUG WORKERS - free | Harm reduction informationhttp://www.substance.org.uk/harm-reduction-information/ketamine-guide-for-drug-workers-free | 13 | 4 | 1 - 2 of 20 -
How to increase SERP for long tail keywords?
Let me describe my situation. I currently run an e-commerce site that aggregates items across various e-commerce website. I am focusing my SEO on the long tail keywords such that when user searches for 'hermes birkin bag croco black' and other specific item search, then the page that I have on my site pops up. What are some tips on increasing this kind of stuff in a general sense such that I don't have to go through all of my 10,000 items that I have in my site and optimize each and every page for keywords. Right now what I am thinking is to increase the domain authority of the overall site. Any other tips?
On-Page Optimization | | herlamba0 -
404 errors in wordpress... Pages have never existed so why is google trying to crawl them?
I've just logged into webmaster tools and have over 100 404 errors. I'm running wordpress and I recently added child pages to 2 of my categories like so. www.mydomain.com/category1/lincolnshire www.mydomain.com/category1/cambridgeshire etc... The 404 errors though are for pages or categories I've never created though. I have over 20 root categories but decided to test adding child pages to only two of them. The 404 errors are for www.mydomain.com/category5/cambridgeshire .... It seem that gogle has tried to crawl these pages that don't exist. Can anyone explain what's going on? When I click 'linked from' in webmaster tools it's showing links from pages on my site that don't exist also.
On-Page Optimization | | SamCUK0 -
How do I get rid of crawl errors?
I recently revamped all my category pages to make them Google friendly. I did a lot of keyword research and ended up deleting categories and renaming categories. Now Google Webmasters is showing a ton of 404 errors. How can I get rid of them? Do I have to remove the URL one by one with Google Webmasters? Is it something that won't hurt my rankings and will go away in time?
On-Page Optimization | | dealblogger0 -
Why is SEOMOZ Crawl Diagnostics not in sync with Webmaster Tools
Currently, my Website, according to the Crawl Diagnostics Summary, has 401 'Duplicate Page Title Errors'. But in Google Webmaster Tools, under Óptimization on the Left hand Side Toolbar, if you look up HTML Improvements, there are only 4 'Duplicate Title Tags'. I have two questions re this: A) Do 'Duplicate Page Title Errors' and 'Duplicate Title Tags' have the same meaning' ? , and B) why are there 401 errors located by the former, and just 4 by the latter?
On-Page Optimization | | ABCPS0 -
Hi Moz peeps, google said that we should have two site maps...
Hi Moz peeps, google said that we should have two site maps... one for google and one for people. Now i know having a site map to submit to google for the first time is important for SEO, but is having a site map for people visitng the site help at all in terms of google's bots crawling your site? I know it might actually help human people navigate through your site, I just want to know if by not having it or having it affects on page SEO at all, thanks guys
On-Page Optimization | | david3051 -
How to Resolve Google Crawling Issues for My eCommerce Website?
I want to resolve Google crawling issues for my eCommerce website. My website is as follow. http://www.vistastores.com/ Google have crawled only 97 webpages from my website. My website is quite old. (~More than 6 months) But, Google have indexed only 97 webpages. I have created one campaign over SEOmoz tool and found some errors over there. So, I just assumed that due to it Google did not crawled my website. But, I have created one another campaign for my competitor website to know actual status and reason behind it. I found that, my competitor website have more error compare to me but, Google have crawled maximum pages compare to me. So, What is reason behind it? How can I improve my crawling rate and index maximum webpages to Google? [6133009604_af85d29730_b.jpg](img src=) 6133009604_af85d29730_b.jpg 6133009604_af85d29730_b.jpg 6139706697_4e252fdb82_b.jpg
On-Page Optimization | | CommercePundit0 -
Why has only one page been crawled?
I went over the list of possible reasons, but I don't think any of them apply to the site in question: sarahhueniken.com. Any other reasons only one page would be crawled?
On-Page Optimization | | scottsemple0
