Massive increase in Moz crawl.
-
I have a subdomain which has just started to be crawled by Moz, Previously this wasn't the case.
The sub-domain had 16,000+ issues. Why has Moz started to count sub-domains as part of the main domain, has Google started to do this aswell?
-
Hi Dave!
Thanks for reaching out! My name is Anne, and I helping the Mozzer Alliance today! There could be a few different reasons we've just started crawling this subdomain:
1. Did you recently link this subdomain to your site differently? Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. This could explain why we just started crawling the subdomain.
2. Did you change your robots.txt for this subdomain? Our crawler, Rogerbot, follows robots.txt. If you edited this site recently, this could also explain why we just started crawling your subdomain.
I hope this helps! If you have more questions, please reach out to the help team at https://moz.com/help/contact. From there we can help you troubleshoot what's going on, but keep your campaign details off the interwebz.
Have a great weekend!
Anne
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep. I just wanted to know the possible reasons and has anyone encountered this before?
On-Page Optimization | | MoosaHemani0 -
I am trying to better understand solving the duplicate content issues highlighted in your recent crawl report of our site - www.thehomesites.com.
Below are some of the urls highlighted as having duplicate content -
On-Page Optimization | | urahul
http://www.thehomesites.com/zip_details/76105
http://www.thehomesites.com/zip_details/44135
http://www.thehomesites.com/zip_details/75227
http://www.thehomesites.com/zip_details/94501 These are neighborhood reports generated for 4 different zip codes. We use a standard template to create these reports. What are some of the steps we can take to avoid these pages being categorized as duplicate content?0 -
Domain authority decreased but rankings increased
Hello! I'm a bit confused. It seems the industry as a whole that my company operates in has taken a bash from the update a few days ago as all of the competitiors have taken a hit to their domain authority. What is confusing is that my websites rankings have increased as well. There has been a change to the robots.txt file in an effort to stop being crawled by SEMRush which is messing with my site stats but that is the only change. Thanks in advance, Ben
On-Page Optimization | | Ben-Cleaver1 -
How to fix Medium Priority Issues by mozpro crawled report??
How to resolve this issues crawled by mozpro?
On-Page Optimization | | renukishor
Some Medium priority issues like that: Missing Meta Description Tag: 2669
Title Element is Too Long: 523
Duplicate Page Title: 37 How to add missing meta description tag in these pages and how to short title element ?0 -
Moz showing 384 description duplicates on my ecommerce store.... when I download CSV, most pages are coming from my WordPress Blog, why?
Hi, I am trying to investigate why I am getting 384 description duplicates on my ecommerce store (www.doggie-diva.com)? When I download the CSV file from MOZ, the majority of the pages they refer to are pages from my Word Press blog, which is hosted on a different server (blog.doggie-diva.com). I do have a link from my website to my Word Press blog and vice versa. Can you please explain to me why this is happening when I don't have duplicate content? Example of a page flagged from www.doggie-diva.com with duplicate content (http://blog.doggie-diva.com/tag/dog-gymnastics. Thanks, Rachel <colgroup><col width="549"></colgroup>
On-Page Optimization | | doggiedivalicious
| |0 -
How to increase SERP for long tail keywords?
Let me describe my situation. I currently run an e-commerce site that aggregates items across various e-commerce website. I am focusing my SEO on the long tail keywords such that when user searches for 'hermes birkin bag croco black' and other specific item search, then the page that I have on my site pops up. What are some tips on increasing this kind of stuff in a general sense such that I don't have to go through all of my 10,000 items that I have in my site and optimize each and every page for keywords. Right now what I am thinking is to increase the domain authority of the overall site. Any other tips?
On-Page Optimization | | herlamba0 -
Error is not going away and crawling
I have fixed an error but its still showing in red as error. Im totally new to SeoMoz and to SEO in general so im not sure how this tool works. Did I fix it correctly or not if its still showing? It was a broken link and now it links up to another page. Do I just have to wait? My website only has 8 pages and on the dashboard it says crawled 8 pages but it takes up to a week for a full crawl? Im really confused. Thank you in advanced!
On-Page Optimization | | Pixeltistic0 -
Not making a change of the 100's in crawl Diagnostic
Based on the PRO crawl Diagnostics – if we don’t make a change on 1 page, does that just affect the SEO on that one page, or does it affect the SEO on all pages of the site? E.g. If we get a “Too many on page links” for a certain page that we don’t really want to rank for – does not fixing that particlaur page affect the site as a whole? Hope I explained this ok..
On-Page Optimization | | inhouseninja0